AI Security

Deepfakes and Synthetic Media: The New Social Engineering Problem

Criminals have used AI-cloned voices to impersonate executives and authorise large transfers, and the FBI has warned that synthetic media will increasingly be used in fraud. We explain how deepfake social engineering works, why detection alone will not save you, and the process controls that stop it.

By EthicalHat Research · · 4 min read
Infographic: How a deepfake payment fraud unfolds

Business email compromise (BEC), in which criminals impersonate executives or suppliers to redirect payments, is already one of the costliest forms of cybercrime. The FBI's Internet Crime Complaint Center consistently reports BEC losses in the billions of dollars each year.

Now the impersonation is moving beyond email.

  • In 2019, the Wall Street Journal reported that criminals used AI-generated audio mimicking the voice of a German parent company's chief executive to persuade the head of a UK energy firm to transfer about $243,000.
  • Earlier this month, Forbes reported on court documents describing a 2020 case in which a bank manager in the UAE was persuaded to transfer $35 million, partly through a phone call using a cloned voice of a company director he knew, backed up by forged emails.
  • In March, the FBI issued a private industry notification warning that malicious actors "almost certainly will leverage synthetic content for cyber and foreign influence operations" in the coming 12 to 18 months, and described synthetic media as an extension of business email compromise.

What deepfakes are

"Deepfake" refers to synthetic media (audio, images or video) generated or manipulated with deep learning, usually to make someone appear to say or do something they did not.

  • Voice cloning can now produce a convincing imitation of a person's voice from recordings of them speaking. Executives who give interviews, earnings calls and conference talks have plenty of public audio.
  • Face swapping and reenactment can put one person's face on another's body in video, or animate a still photo.
  • Synthetic identities combine generated profile photos and fabricated histories to create fake people, useful for fake recruiters, fake suppliers and influence campaigns.

Quality varies. Real-time, interactive deepfakes are still harder than pre-recorded ones. But for a short, urgent phone call over a poor line, the bar for "convincing enough" is low.

Why it works

Deepfake fraud exploits the same psychology as all social engineering, with an extra layer of false confidence:

  • Authority: the voice of the CEO or a senior director.
  • Urgency: a confidential acquisition, a regulatory deadline or a supplier about to walk away.
  • Secrecy: "don't discuss this with anyone yet".
  • Corroboration: a follow-up email from a spoofed or compromised account appears to confirm the call.

Hearing a familiar voice disarms the scepticism people have learned to apply to unexpected emails.

Why detection alone is not enough

Researchers and vendors are building deepfake detection tools that look for artefacts in audio and video, and there are useful signals: unnatural blinking, lighting inconsistencies, audio compression artefacts and odd prosody. But:

  • generation quality is improving quickly, and detection is an arms race,
  • phone calls are low-fidelity, which hides artefacts,
  • most organisations cannot run detection on every call their finance team receives, and
  • a successful fraud needs only one convincing moment.

Process is a far more reliable defence than detection.

Controls that stop deepfake fraud

Out-of-band verification. Any request to transfer funds, change bank details, share sensitive data or grant access must be verified through a different, pre-established channel, such as a call back to a known number from the company directory rather than the number that called you.

Dual authorisation. Payments above a threshold require approval from two people, one of whom did not receive the original request.

No exceptions for urgency or seniority. The policy must explicitly say that the CEO will never ask anyone to bypass it, and senior leaders must model that behaviour themselves.

Shared verification phrases. Some organisations use pre-agreed code words for high-risk requests between executives and finance teams.

Supplier change controls. Changes to supplier bank details are verified with a known contact and held for a cooling-off period.

Training with real examples. Update security awareness training to include voice and video impersonation. Let staff hear what a cloned voice sounds like.

Reduce what attackers can learn. Consider what executive audio and video is publicly available, and what organisational details (who approves payments, who reports to whom) are easy to find.

Beyond payments

Synthetic media is also being used for:

  • fake job candidates in remote interviews, to gain access to company systems,
  • reputation attacks using fabricated statements or videos, and
  • bypassing voice-based authentication at call centres.

Each needs its own verification controls: identity checks for remote hires, rapid response plans for fabricated media, and stronger authentication than voiceprints alone.

The bottom line

Deepfakes turn "I recognised their voice" from evidence into a vulnerability. The good news is that the defences are not exotic: verify high-risk requests through an independent channel, require more than one person to approve, and train people that urgency and authority are exactly what attackers fake. Those controls work whether the impersonation is by email, phone, video or something not yet invented.

Our security awareness programme includes deepfake and voice-phishing scenarios, and our spear phishing simulations test whether verification processes hold up under pressure.

EthicalHat Research

The EthicalHat research team: penetration testers, SOC analysts and engineers writing about security, software engineering and AI.

Want an expert opinion on your own systems?

Email info@ethicalhat.com or call +1-844-838-4422. Our security engineers are happy to talk it through.