AI is becoming capable of doing more than generating an answer. Increasingly, systems can use tools, interact with other systems and take actions on behalf of people.
That shift raises a new set of questions around security, customer choice and human oversight, speakers told Commonwealth Bank’s internal BrighterTech technology event in Sydney and Bengaluru.
Across the event, a common thread was how organisations must design the controls, experiences and work around AI’s growing capability so people can use it with confidence.
Five key themes provide a guide to what that response could look like.
1. More autonomy needs strong controls
The challenge changes when an AI system can act rather than simply provide information.
Anthropic Deputy CISO Jason Clinton described a personal AI agent as a “brain in a jar”, connected to other systems through a series of virtual arms.
Those connections provide opportunities to control what an agent can do.
Clinton told BrighterTech that Anthropic allows an internal agent to work with email and prepare a draft, for example, but does not give it permission to send the message. A person must take the final action.
The principle is to match controls to the potential impact of an action rather than trying to remove AI capability altogether, he said.
“An airplane that never flies is 100% safe, but it’s also completely useless,” Clinton said.
The same issue was raised at CommBank’s BrighterTech event in Bengaluru, where Snyk’s Pas Apicella, spoke about keeping people accountable for AI-driven work and ensuring agents operate within defined boundaries.
“Let the agents do the work, but accountability remains with humans. And don’t let the same agent validate the code that it writes,” Apicella told the event.
2. The next customer could be an AI agent
AI agents could also change the way customers deal with banks and other businesses.
Futurist Katja Forbes told BrighterTech that agents were beginning to act on behalf of people and organisations, including making payments, negotiating and interacting with products and services.
“Your next customer will be an algorithm,” Forbes said.
She described a spectrum ranging from agents that help people make decisions through to delegated and more autonomous agents that can carry out tasks themselves.
That creates practical questions for businesses about what external agents can access, which actions they can take and the terms on which organisations will deal with them.
Forbes described that ability to set boundaries as “commercial sovereignty”.
CommBank’s AI approach recognises that customers may increasingly interact with its services through external agents or systems, with appropriate safeguards, limitations and controls considered where material risks are identified.
3. People may want AI help without giving up the decision
Greater automation does not mean customers will want every choice made for them.
Customer experience strategist Dan Monheit told BrighterTech that people were likely to move between three broad types of experience: asking technology to “do it for me”, doing something themselves, or asking AI to “help me choose”.
He expects that middle ground to become increasingly important, with AI helping people search, analyse and narrow their options while leaving the final decision to them.
That distinction is particularly relevant in banking, where trust can depend on both the technology and the people using it.
Research published by CommBank and Melbourne Business School in February found 69% of surveyed Australians would be more likely to use a bank if they knew its staff had been trained in the safe, ethical and responsible use of AI.
4. Cyber security has to prepare for what AI will become
Security teams also need to look beyond what current models can do.
Clinton told BrighterTech that AI was already being used across traditional security functions at Anthropic, including automated security reviews, vulnerability remediation and detection and response.
But he cautioned against designing security systems solely around the capability of today’s models.
His advice was to “skate to where the puck will be”, anticipating that the systems organisations deploy now will become considerably more capable over time.
CommBank’s own approach reflects the two-sided nature of the challenge. It says AI can introduce new cyber vulnerabilities and help bad actors scale attacks, while also providing tools to strengthen defences and identify signs of suspicious activity.
5. Work needs to keep building human judgement
AI can make experienced workers more productive while reducing some of the opportunities through which less experienced people learn.
University of California researcher and author Matt Beane told BrighterTech that he had observed that pattern across more than 30 occupations, including situations where automation enabled experts to do more while junior workers lost access to parts of the job that previously helped them develop.
His research identifies challenge, complexity and connection with other people as three important ingredients in building skill.
Rather than preserving older ways of working, Beane said organisations needed to redesign work so they could gain the productivity benefits of AI while continuing to build human capability.
“We can have both and we must,” he said.
