Is AI Making Payment Fraud Easier?

Unless you’ve been living under a rock for the past few years, you’ve definitely heard about the changes AI is bringing to nearly every industry. From text to hyper-realistic images and even audio and video content, AI can generate basically anything from scratch. 

Sure, it’s not without its flaws, but with every passing day AI algorithms get smarter and more advanced. The Artificial Intelligence Index Report 2024 describes the advancement of AI as “startlingly rapid.” As the report outlines, AI systems like ChatGPT “have become so advanced that they now very nearly match or exceed human performance in tasks including reading comprehension, image classification and competition-level mathematics.” 

But what does that mean for your business? The answer to that question is: potentially everything. Depending on what you’re trying to do, there’s really no limit to how AI could help your business grow and thrive. But, on the other hand, that growth also applies to the people looking to hurt your business.

Deepfakes are one of the most dangerous types of fraud made possible by AI. Using deepfakes – incredibly realistic fake images, audio and video – can convince people that a fraudster’s message is the real deal. In Hong Kong recently, for example, a finance worker was tricked into paying out $25 million after viewing a staged video call with the company’s CFO and their colleagues. This type of deepfake is also known as synthetic media generation – and it’s especially dangerous when it comes to payment fraud. 

In 2023, the National Security Agency, FBI and Cybersecurity and Infrastructure Security Agency published a paper warning companies to be on the lookout for this type of payment fraud. The call looked so real that the worker didn’t see any issue with authorizing the payments, costing their company millions. If you aren’t aware of the risk of deepfake scams, it becomes incredibly easy to fall for them. 

The key to avoiding deepfake scams is to stay informed. Once you understand that deepfakes are a potential threat to your business, you’ll be more likely to consider the possibility when something doesn’t quite feel right. 

Stop and think before actioning anything that could potentially have come from a deepfake. Have you just come off a video call where people were acting unusually or it felt like the call came from out of nowhere? Speak to the people who were on the call to make sure it really was them. It might feel silly, but I promise it won’t feel sillier than having to admit you lost your company millions of dollars.  

AI isn’t all about new, futuristic scams. It’s also helping fraudsters perfect and grow their tried-and-true tricks. 

You’re familiar with phishing scams – we've talked about them on the blog before and you’ve likely been trained to spot them at work. But with the help of AI, phishing can become harder to detect, harder to stop and much more common. 

For example, a previous way to figure out whether an email was a scam or not was to reply to it. In the past, scammers wouldn’t be able to respond well enough to convince their would-be targets that the scam was legit. Maybe their English wasn’t strong enough or perhaps they didn’t know the answers to key questions asked in the reply. With AI, that language barrier disappears and the scammers instantly become much “smarter” with their industry knowledge. 

Thankfully, even though automated phishing scams can seem more real with the help of AI, you can still take steps to avoid falling for it. Remember to refer back to original invoices and contracts every time – every bit of information from the numbers to the company address should match. And, when in doubt, call the company and speak to them yourself to confirm whether you’re dealing with the real deal or not. 

In the past, scammers had to either understand hacking well enough to access someone’s email accounts or use email cloning methods to try and trick their targets. Now, they just have to be patient. 

If a fraudster is looking to take over an email or other account, they can look to AI tools like PassGAN, an AI tool developed last year that cracks passwords in the blink of an eye. A recent study found that PassGAN’s AI can crack 51% of passwords in less than a minute, 65% in less than an hour, 71% within a day and 81% within a month. That means that practically no one is safe from very sophisticated technology attempting to crack their passwords. 

Now, that doesn’t mean that you’re helpless against AI-fueled identity theft. It just means that, if your business is targeted, there are tools fraudsters can use to help them succeed. It’s your job to be prepared for new and sophisticated fraud attempts. Consider setting other passwords or “code phrases” with your business partners. That way, if a scammer succeeds in stealing their identity, you have another layer of security while speaking to them.

A picture is worth a thousand words. But could a fake picture end up costing your business? Scammers can get even more creative with the help of generative AI, creating realistic images to try and back up any claims they might make to squeeze money out of your business. Let’s say, for example, your manufacturing company receives an email from a dissatisfied customer. Your customer claims that they received damaged products from your company and as a result, they’d like a refund. To back up their claim, your customer attaches photos of the damaged products. 

But are you 100% sure that those photos are real? At the time of writing, AI photos can sometimes appear “dream-like” or “soft,” with blurred edges and other markers that point to AI. But with the speed at which AI is advancing, that’s likely to change in the near future. How are you supposed to tell the difference?

The key here is to stop and think carefully about what you’re seeing. Before you process refunds or make unexpected payments, double check all the information in front of you. Even things like the email signature or specific turns of phrase the email uses could alert you to potential fraud. If you have someone local who can do a physical check of the goods, that’s another possibility. 

There’s no question that AI is transforming the landscape of nearly every industry right now. Whether you’re extremely familiar with everything AI can do or a total beginner, the key to protecting your business from the new risks AI can pose is to do your homework. Stay updated on what AI is capable of, recent fraud attacks and the methods fraudsters use to get to your business. If everyone on your team is consistently and regularly trained on how to spot the signs of AI in payment fraud, your business has an extra layer of protection against a very tricky area of security.