The email looked perfect. It opened with an internal project name that only senior staff would know. The writing style matched the CEO's — the cadence, the occasional em dash, the way she always signed off with "Best, SJ."
It requested an urgent wire transfer of ₹47 lakh to a new vendor account. It explained that the board approval had already happened on a call. It asked the finance manager to move quickly and keep it quiet until the announcement.
The transfer was made before anyone thought to call the CEO directly. She had not sent the email. Nobody had. An AI had written it.
What Changed When Hackers Got AI
The phishing email used to be obvious. Bad grammar. Strange sender domains. Generic salutations. 'Dear Valued Customer.' You could spot it in seconds. The advice was simple: if it looks wrong, it probably is.
That advice is now dangerous. Modern AI-powered phishing attacks are trained on the target's actual writing — scraped from LinkedIn posts, public emails, company announcements. The AI analyses the communication style and produces emails stylistically indistinguishable from the real person.
What used to require a skilled social engineer spending weeks researching a target now takes minutes. An attacker can describe the target, provide writing samples, specify the pretext, and receive a highly convincing message that references real projects, real relationships, and real events.
The Deepfake Video Call — A Documented Case
In January 2024, a finance employee at Arup — the London-based multinational engineering firm — received an email that appeared to come from the company's UK-based CFO, requesting a series of confidential transactions. The employee was suspicious. So they asked for a video call to verify.
On that call were the CFO — and several familiar colleagues. They looked real. They sounded real. They addressed the employee by name and confirmed the transfers. The employee complied and executed fifteen transactions totalling HKD 200 million — approximately USD $25.6 million.
Every person on that call was an AI deepfake. The CFO had not sent any email. None of those colleagues had joined any call. The attackers had built the deepfakes from publicly available video and audio of Arup executives scraped from online conferences and company recordings. Arup confirmed the incident in May 2024 after CNN reported it. Hong Kong police reported no arrests had been made as of early 2025. (Source: CNN, February 4, 2024; Fortune, May 17, 2024; Hong Kong Police briefing)
"The attack surface is no longer just your software. It is your psychology — your trust, your urgency, your instinct to defer to authority."
How Defenders Are Fighting Back With AI
Security teams are not standing still. AI is being deployed on the defensive side at the same pace as the offensive. Behavioural anomaly detection systems now establish a baseline of normal behaviour for every employee — when emails, login times, data access patterns, or transaction authorisations deviate from that baseline, the system flags it automatically.
Email security platforms powered by AI now analyse writing style, sender graph analysis, metadata, and request patterns. An email asking for a financial transfer from someone who has never made such a request before triggers a flag regardless of how convincing the content is.
Voice biometrics authentication is being deployed for high-value phone transactions — the system continuously verifies that the voice on the call matches the authenticated voiceprint, and flags if the match drops below a confidence threshold.
What Every Individual Can Do
Technical defences matter. But the most effective defence against AI-powered social engineering remains human awareness and protocol. First: always verify urgent financial requests through a second, independent channel. If the CEO emails you asking for a transfer, call the CEO on a number you already have saved — not a number provided in the email.
Second: slow down on urgency. "Move quickly." "Before end of day." "Keep this confidential." These phrases should trigger scepticism, not compliance. Legitimate urgent requests can withstand a two-minute verification call.
Third: trust patterns, not people. If a request breaks normal patterns — different payment channel, new vendor, unusual amounts, requests for secrecy — apply extra scrutiny regardless of how familiar the source seems.
The Bigger Picture
This arms race has no finish line. As defensive AI improves, offensive AI adapts. As voice cloning detection gets better, voice cloning gets more sophisticated.
The only sustainable advantage in this environment is not a specific technology — it is a culture of security awareness that evolves with the threat. Organisations where every employee understands the attack vectors, follows verification protocols regardless of apparent source, and feels comfortable slowing down on suspicious requests are significantly harder targets than those relying solely on technical defences.
The technology will keep changing. The human judgment to question and verify is the one defence that does not have an obvious AI bypass.
