How Does AI Help Cybersecurity Teams — And How Do Attackers Abuse the Same Tools?

The Real Case: A $25 Million Video Call That Never Happened

In January 2024, a finance worker at a multinational firm in Hong Kong joined a video conference call with who appeared to be the company's CFO and several colleagues. They discussed a confidential transaction. The CFO gave direct instructions to transfer $25.6 million USD to specified accounts. The employee complied.

Every single person on that call was a deepfake.

Attackers had scraped publicly available video footage of the real executives, trained AI models on their faces and voices, and staged an entire fake meeting in real time. No blurry, glitchy cartoon faces — convincing, moving, talking replicas. The fraud wasn't discovered until the employee checked with headquarters days later.

This wasn't a sophisticated nation-state operation requiring a team of elite hackers. The tools used — video synthesis models, voice cloning software, and real-time face-swapping applications — are commercially available. Some are free. That's the part security professionals don't say loudly enough: the barrier to running this attack is now shockingly low.

How the Attack Actually Works — Step by Step

Here's the exact attack chain for a deepfake executive fraud — broken down so it's clear what you're actually up against:

1. **Target selection.** Attackers identify a company with publicly available executive content — LinkedIn videos, earnings call recordings, YouTube interviews, conference presentations. Thirty seconds of clean audio is plenty. Three seconds can be enough for basic voice cloning using tools like ElevenLabs or open-source alternatives.

2. **Data harvesting.** They download footage, extract audio tracks, and feed them into AI voice and video synthesis models. This process now takes hours, not weeks.

3. **Phishing the entry point.** Before the big call, attackers often send a convincing AI-written email — no grammar errors, perfect tone, correct company terminology — to establish trust and set up the meeting.

4. **The call.** Using real-time deepfake software (HeyGen, DeepFaceLive, or custom setups), they join a video call. The fake face tracks head movements. The cloned voice responds to questions with slight natural delays. It reads as human.

5. **The ask.** A wire transfer. A credential reset. Access to a system. Always urgent. Always confidential. Always framed as coming from authority.

On the defensive side, security teams use AI differently: anomaly detection systems like Darktrace monitor network traffic patterns and flag deviations in milliseconds. AI-driven email filters from companies like Abnormal Security score incoming messages for behavioral signals no keyword list could catch. Same underlying technology. Opposite intent.

Why Smart, Careful People Still Fall For It

Most guides say the solution is 'slow down and think critically.' Here's why that's often not enough.

The psychological mechanism isn't gullibility — it's authority combined with manufactured urgency. When your brain registers your boss's face, your boss's voice, and your boss's communication style simultaneously, skepticism gets overridden. This is how humans are wired. The attack is designed specifically to exploit the trust shortcuts our brains take with familiar faces.

Here's the surprising part: research from University College London found that people correctly identify deepfake audio only about 73% of the time — even after being warned they might hear one. Without warning, that number drops sharply. Your ears are not a reliable detector.

And the quality gap is closing fast. Early deepfakes had tells: unnatural blinking, blurry ear edges, audio sync issues. Current models running on consumer GPUs produce output that professional video editors struggle to flag. When the Hong Kong employee was later interviewed, they described the call as entirely normal — no red flags, no hesitation.

That's the trap. You won't feel suspicious because the attack is engineered specifically so you won't.

Your Defense Checklist — Do These This Week

If you're waiting for your company to protect you, you're wasting time. Individual action matters here.

**1. Create a family safe word — today.** Pick a word or short phrase that only your immediate family knows. If anyone calls claiming to be a family member in an emergency, they must say the word. No word, no trust. No exceptions. This defeats AI voice cloning completely.

**2. Call back on a known number.** If you receive an urgent call or message — from a boss, banker, or family member — hang up and call them back using a phone number you already have saved. Never use a number provided in the message itself.

**3. Establish a company code protocol.** If you handle financial transfers at work, push for a verbal confirmation code system for any transaction over a set threshold. One short out-of-band confirmation call to a pre-registered mobile number stops wire fraud cold.

**4. Treat video as unverified.** Seeing someone on video is no longer proof of identity. Ask an unexpected question only the real person would know. Make them wave with a specific hand. Introduce unexpected variables the attacker didn't prepare for.

**5. Don't post long voice or video clips publicly if you can avoid it.** Executive LinkedIn videos, podcast appearances, and conference recordings are training data. Limit what you put out there, especially clean, quiet audio.

**6. Report suspected attempts immediately.** Contact your IT security team and file a report with the FBI's IC3 (ic3.gov) or your national cybercrime unit. Reporting creates the data trail that helps researchers track evolving attack patterns.

Key Takeaways

• A single deepfake video call cost one Hong Kong company $25.6 million in January 2024 — the attack used publicly available AI tools, not classified technology.
• Voice cloning models like ElevenLabs can replicate a person's voice from as little as 3 seconds of clean audio — any podcast, video, or voicemail you've posted publicly is fair game.
• People correctly identify AI-cloned audio only ~73% of the time even when warned — your instincts are not a reliable defense against well-executed deepfakes.
• Create a family safe word today — a single pre-agreed code phrase that anyone calling in an 'emergency' must provide. It costs nothing and defeats voice cloning entirely.
• By 2026, real-time deepfake video will be accessible via standard consumer hardware to virtually any attacker — the Hong Kong case will not be an outlier, it will be routine.

FAQ

Q: How do I know if a call from a family member is real?
A: Use a pre-agreed safe word that only your family knows — if the caller can't provide it, hang up and call back on the number saved in your phone. Don't let urgency or emotional pressure override this step; that pressure is often part of the attack design.

Q: Can AI voice cloning be detected by phone companies?
A: Honestly, not reliably yet — carriers can flag some patterns, but real-time voice synthesis moves faster than detection infrastructure currently can. A few startups like Pindrop are working on voice authentication tools, but they are enterprise products not available to individual consumers today.

Q: What should I do if I think I was targeted by a deepfake scam?
A: If money moved, call your bank immediately to attempt a recall — speed matters more than anything in the first 24 hours. Then file a report at ic3.gov (US) or your national cybercrime unit, and notify your company's security team so they can investigate and warn others.

Conclusion

The same AI that helps a security analyst detect a breach at 2 a.m. is also helping a scammer impersonate your CEO on a video call at 9 a.m. the next morning. That symmetry is what makes this moment genuinely dangerous — the tools are neutral and the attackers got there first. Do one thing today: text your family group chat and agree on a safe word. That single action makes you meaningfully harder to attack than the vast majority of people who will read this and do nothing.