August 2023 Roundup: How scammers are using custom AI tools for fraud

Simon Taylor

min read

•

September 5, 2023

Hey Fraud & Compliance Nerds 👋,

We’re switching things up and trying a new format for this month’s newsletter.

If we’re being honest, the last one felt a bit long on the product updates so we wanted to tighten it up and try to lead with the value and link to the product updates.

So today’s newsletter is organized into 3 chunks: a rant, a new fraud pattern, and some tactical advice to upgrade your output at work.

Rant: Fraudsters using generative AI (like ChatGPT) for scams
Pattern: Voice Cloning in Account Takeovers
Tactical: 5 mental models for fighting fraud

Before we jump in, here’s the link to last month’s product updates for Risk and Sponsor OS. TLDR, we shipped way more KYB depth, more batch and SQL style rule querying and generation. Oh and if you’re a sponsor bank, the rules and queues that give you oversight of your programs just got a whole lot better.

Let us know what you think about the new format!

PS: Are you going to fintech_devcon? I’m giving a presentation about the mistakes everyone makes in fintech. Pass by and say hello. You’ll come for the insights, and stay for the hot takes.

Alright, let’s talk fraud.

📣 Generative AI is Scaling Up Fraud

Generative AI is a classic example of a “dual use” technology. It is both good and bad.

The scientists at the Manhattan project wanted to create unlimited cheap energy for the world. They didn’t foresee the dual use of splitting the atom.

The release of the movie Oppenheimer is a timely reminder that some technology breakthroughs can be used for good and bad.

Generative AI (GenAI) is capable of delivering us a world where we become superhuman, 10x more productive and have a thoughtful, infinitely patient army of assistants at our beck and call. It’s also capable of generating images of child exploitation, instructions for deadly chemical weapons and now, scaling scams.

Now the scammers are using GenAI.

The last two weeks saw the release of DarkBERT, WormGPT and FraudGPT tools to help scammers scale up their attacks. Historically scammers had a choice, go wide or go deep. Go wide with generic scams hoping to get some hits, or go super deep but narrow and specific with higher conversion.

These new tools mean we already have scams being generated that are highly personalized, localized and properly formatted. They’re believable and much more effective than the traditional “spray and pray” attacks fraudsters would use.

Some financial institutions are banning the use of large language models like ChatGPT internally, but that won’t help us. Genie’s don’t go quietly back into bottles. The early adopters are those that can make money quickly exploiting the tools.

The question now is what do you do about it.

Here’s my playbook

Stay informed. What you don’t know will hurt you. Grow bigger ears, build systems of intel gathering, and knowledge sharing internally. Attend conferences and community events in fraud and compliance like ACAMS and MRC. Setup dinners with your peers, have knowledge sharing sessions, listen to podcasts like Fraudology and follow this guy called Soups Ranjan on LinkedIn.
Get more data. Soups always says “all risk problems are data science problems.” He’s right, and all data science problems need more data, and higher data quality. You cannot under invest in this space. Look past the transaction to the device, user behavior, email history, open banking and consortia data (like EWS and SardineX).
Get friends. Fraudsters share information on what is effective on the dark web. As the legitimate industry we have to be careful to color inside the lines of data privacy. But we have frameworks to share insights and data. We’re also building a “Fraud Squad” community, drop us a line if you’d like to be involved and get early access.

Sharing is caring.

That’s why I got so excited by SardineX. It’s a genuine attempt to become the anchor for trust across all payment networks and payment types. Whether you’re a SaaS business that does payments, a financial institution or a Crypto exchange.

As the song in Moana goes “You’re welcome…” To join SardineX

(Yes I have a toddler who’s obsessed with that song, you’re welcome).

ST.

🕵️ Voice Cloning in Account Takeovers (ATO)

A new Fraud just dropped

Voice cloning in account takeovers 🏴‍☠️

Voice based authentication is not reliable anymore for authentication into a bank or financial services industry.

It's now trivial take a few seconds of someone’s voice and train a GenAI model to answer any questions during the bank login process.

Traditional methods of authenticating users are less reliable.

Increasingly we need more factors, and more patterns of behavior to have confidence a user is who they say they are.

That’s why I’m convinced device and especially behavior are such powerful signals. Your voice, your face and even short videos of “liveness” can be faked.

But how you type, swipe and tap are as unique as every strand in your DNA.

🔑 The keys to prevention

Ensure every login and authentication is also checking for the user device and behavior changes
If you see different devices, IPs, bots or geolocations add additional friction such as requiring a PIN code
If the user behavior has also changed, layer on further friction like a hold on the account for 24 hours until an agent can verify a customer

The more signals you watch during authentication, the more your team (or our AI) can identify the patterns fraudsters are using and their attack pattern. These patterns are often unique to the product you offer and the markets you operate in.

There is no one size fits all. Often one size fits nobody. If you want something tailored like an in-house team spent 5 years and $XXm building it, get in touch. That’s what we do.

Stay safe out there Fraud Squad 🐟🐟

Soups.

➕ Fraud Squad: Lessons from Evolution

Fraud is a classic adversarial problem - the fraudster vs. the provider of goods/services.

As each side's strategy evolves the other adjusts - ad infinitum.

Just like Predator vs Prey.

That can be instructive. I’ve derived derived mental models from nature to help you fight fraud - here are a few traits that you can use to be successful:

Develop a diverse and sophisticated sensory system. Rather than using a single sense like sound, sight or smell, the combination is most powerful. Get more data, get a more complete picture of the landscape and you can better defend your users. We use Machine Learning to spot new anomalies and patterns, needles in the haystack.
Pattern recognition is key to spotting danger. Prey’s ears notice a rustling in the long grass and sense that usually means danger. Likewise the fraud squad spots patterns and creates rules like “Email Doesn’t Have History + IP is a VPN + Browser is in Incognito Mode + VOIP Phone.”
Use poison to lower the expected value of predation. Pufferfish and Skunks have a deterrent from attack. Similarly the fraud squad will make life harder with “step up” verification or additional friction to make life harder for attackers.
Protect yourself against ambushes. Attacks and night or from the long grass happen in fraud too. The fraud squad uses alerts and always-on machine learning models looking for danger. When your team is out of hours or least expects it the alert can create faster reactions.
Develop a network ‘alarm system.’ Just as dogs bark sensing danger, so too can the fraud squad can share information between good actors. Financial institutions, merchants and payments companies can alert each other to danger.