Why human behaviour remains key to scam prevention

Most people believe they could spot an AI-generated scam.

Recent research from Commonwealth Bank’s Behavioural Science Centre of Excellence suggests that confidence may not always match reality. Nearly nine in ten Australians (89%) believe they could spot an AI-powered deepfake scam, but only 42% were accurate when tested, highlighting a gap between confidence and ability . It’s a finding that challenges assumptions about how effectively people can identify deception online. 

For the payments industry, that finding matters because every scam ultimately relies on a human decision. Someone needs to trust the message, believe the story and approve the payment. Compounding the issue, the point at which a scam begins is often far removed from where financial losses occur. 

James Roberts, Executive General Manager, Fraud and Scams at Commonwealth Bank, explains that "by the time you're making the payment, the deception has already been successful." 

It’s the final stage of a much longer chain of manipulation, and one reason why Commonwealth Bank is focused on embedding intelligence, verification and intervention upstream in the payment journey.

Where scams begin

Whether a scam begins with an online advertisement, a messaging platform or another digital channel entirely, the pattern is increasingly familiar. Criminals impersonate trusted brands and public figures and create convincing digital content to build credibility before any money moves.

That shift is forcing the industry to expand its field of vision. Scam prevention is no longer confined to payment rails or transaction monitoring. It spans social media platforms, messaging services, telecommunications networks, mobile operating systems, identity systems, payment service providers and cryptocurrency exchanges.

And criminals are highly adaptive. Where SMS was once a common channel for scam campaigns, Roberts says he sees the activity shifting to iMessage and RCS. As controls improve in one part of the ecosystem, activity migrates elsewhere.

A similar dynamic is playing out in the technologies enabling scams.

Building smarter defences

AI is creating new opportunities for fraud and scams at scale. The ability to produce high-quality deepfake audio and video, combined with real-time face swapping technology, mean criminals can impersonate individuals in ways that are almost imperceptible to victims.

Roberts refers to these as “crime-as-a-service” tools, explaining that bad actors can access these technologies on the dark web. Increasingly, these tools pose a threat through identity theft, allowing someone’s likeness to be co-opted to bypass identity verification systems.

Yet the same underlying technologies are transforming scam defences. Across the financial services sector, organisations are deploying agentic AI to identify emerging fraud patterns, strengthen controls and respond more quickly to changing criminal tactics.

One example is Commonwealth Bank’s ‘Pollen Team’, a specialised unit managing a swarm of AI agents that lure and engage scammers in automated conversations. Since launching last year, these agents have conducted more than 350,000 message and voice-based interactions with scammers.

These bots distract scammers and tie up their resources, but they also extract real-time intelligence about their methods. This can then be used to block further scam attempts and protect other consumers, as well as strengthen broader industry responses.

As Roberts points out, scammers are constantly evolving, and the speed at which you can keep up with scammers’ changing modus operandi is directly proportionate to how many people you can help.

The benefits of speed and scale are also enhancing internal controls. Armed with the latest intelligence, AI agents are helping improve Commonwealth Bank’s fraud detection system. The Fraud Detection Agent (FDA), for instance, is analysing millions of transactions each day and suggesting changes to rules governing the identification of suspicious activity.

By working alongside the FDA, Commonwealth Bank teams have updated the majority of the approximately 3,000 fraud rules in recent months, and reduced implementation times from days to minutes.

An ecosystem response

Banks and payments providers have also moved to introduce scam protection measures within the payments process itself.

Since 2023, Commonwealth Bank’s NameCheck solution has compared account details customers enter when making a payment against available data to help customers avoid scams or mistaken payments. Confirmation of Payee complements this by matching the name entered by a payer with the details held by the receiving bank.

However, while AI is strengthening fraud detection for individual institutions, the scale of scam activity requires collaboration across the broader ecosystem.

Australia’s approach reflects that thinking. Industry initiatives such as Scam-Safe Accord and the Fraud Reporting Exchange (FRX) system have focused on improving coordination between institutions, accelerating intervention and enabling more effective recovery efforts. Regulatory reforms have also sought to bring multiple sectors into the conversation, recognising that scam losses often originate far from the point where a payment is made.

At the same time, banks are using an expanded set of data sources to assess payments risk in real time. As Roberts says, “more data basically means more accuracy,” and this has long applied to the customer device and behavioural telemetry banks see on the payer’s side that helps determine whether a payment should proceed.

Roberts says that collaborations and intelligence sharing across the ecosystem have extended what banks like Commonwealth Bank can see on the recipient side. The BioCatch Trust Network is one such initiative.

“With BioCatch Trust, the behavioural device telemetry that we see on the sender side is seen pre-payment on the recipient side before a payment is released,” Roberts says. “It allows you to consider behavioural attributes and score social engineering and account takeover risk.”

Roberts believes that increased visibility will become more important as scammers pivot to fast international money transfers as tighter regulation and policies in many jurisdictions gradually reduces the attractiveness of cryptocurrency as a method of moving scam proceeds.

“When we get to a world of productionised real-time settlement on international electronic transfers, that will become a replacement for the crypto outflows because the criminals need to get it out, and get it out quickly,” Roberts says.

As financial institutions adopt faster cross-border payment capabilities, Commonwealth Bank also continues to extend NameCheck beyond its own banking platform. The solution can be embedded into the payment processes of banks and other organisations to help validate account details used in domestic and international payments.

The sharing of capabilities and data across the domestic market and global ecosystem reflects Australia’s unusually collaborative approach to mitigating scams. Roberts points to growing interest from the United States, where industry bodies have been benchmarking Australian initiatives to strengthen their own scam prevention activity.

Augmenting human decision-making

Despite advances in AI, behavioural analytics, and intelligence-sharing networks, the industry’s biggest challenge remains fundamentally human. After all, scams succeed because they commonly exploit trust and urgency.

Research from Commonwealth Bank’s Behavioural Science Centre of Excellence suggests many people are more confident in their ability to identify deception than they should be. It also indicates that almost one in three (30%) of scam victims suspected something was wrong but went ahead anyway.  In many cases, the issue is not a lack of information, but how decisions are made under pressure.

This is shaping the way fraud and scam prevention is designed. AI-powered fraud detection and prevention, verification data and targeted friction embedded into payments processes can help prevent losses before a payment is completed.

The systems being built reflect a simple reality that the industry cannot assume that every customer will recognise a scam before it’s too late. The payment may be where losses occur, but the deception often begins much earlier.