Learn how deepfake technology works, the threats it poses, and the AI-powered tools being developed to detect synthetic media.
I Stopped Trusting Video Calls. Here’s Why.
Everyone says seeing is believing. I used to think that too. Then a finance worker at Arup — one of the biggest engineering firms on the planet — got on a video call in February 2024 and transferred $25 million to criminals because every other face on that call was fake. The chief financial officer? Fake. The colleagues sitting in their little Zoom rectangles nodding along? All AI-generated. Hong Kong police confirmed the whole thing after the money was already gone.
That broke something in me, I think. I’ve covered tech for about fifteen years, and plenty of threats have come and gone. But I can’t quite shake this one. I’ve talked with engineers at Google DeepMind, researchers at MIT’s Computer Science and Artificial Intelligence Laboratory, and voice authentication people at Pindrop. They’re all saying roughly the same thing: the tools for making convincing deepfakes are getting cheaper and easier to use every month. And our ability to catch them? Not keeping up. Not even close.
So I stopped trusting what I see on screen by default. Maybe that sounds paranoid. Probably is. But after spending months digging into how this stuff actually works, I’m not sure paranoia isn’t the rational response right now.
What Even Is a Deepfake, in Plain Terms?
The word mashes together “deep learning” and “fake.” Synthetic media — video, audio, images — made or altered by AI. That’s it at the simplest level.
Under the hood, it’s more interesting. Most deepfakes historically relied on something called generative adversarial networks, or GANs. Two neural networks competing with each other. One generates fake content. The other tries to spot the fakes. They go back and forth in a loop, and the generator keeps getting better until its output fools the detector consistently. If a neural network built specifically to catch fakes can’t tell the difference, what chance does a person on a Thursday afternoon video call have? Slim to none, from what I’ve seen.
But GANs aren’t the only method anymore. Diffusion models — the same technology behind Stable Diffusion from Stability AI and OpenAI’s DALL-E 3 — have moved into deepfake territory. They work by adding noise to training data and then learning to reverse that process, building new content from scratch out of randomness. Results are disturbingly realistic. And then there are variational autoencoders (VAEs), which compress facial features into a mathematical representation and decode them onto a different face. Real-time face-swapping without needing a supercomputer.
Why does any of that matter? Because five years ago, making a convincing deepfake meant expensive hardware, specialized skills, and days of render time. Now? Open-source tools like DeepFaceLab and FaceFusion run on regular gaming GPUs. A kid with a decent PC and some YouTube tutorials can produce a face-swap video in under an hour. That shift from hard-to-do to trivially-easy is what makes this so dangerous.
Is Voice Cloning Actually Scarier Than Video Fakes?
I think so. And I’m probably not alone.
Companies like ElevenLabs, Resemble AI, and Descript’s Overdub can clone someone’s voice from as little as three seconds of sample audio. Three seconds. Shorter than most voicemail greetings you’ve ever left. Once the model has that tiny sample, it generates speech in your voice saying whatever it wants — with breathing patterns, emotional shifts, natural pauses. All of it.
Vijay Balasubramaniyan runs Pindrop, which handles voice authentication for major banks. He told me something I haven’t been able to stop thinking about: voice deepfakes are being used in social engineering attacks against bank call centers, and the rate’s doubling every six months. His human agents can’t tell cloned voices from real ones anymore. Only their AI detection systems catch them, and even those need constant updates. Pindrop processes over 5 billion calls a year. They’re finding synthetic voice in roughly 1 out of every 2,000 calls to financial institutions. Sounds tiny until you multiply it out.
And it’s not just banks. In January 2024, robocalls went out to New Hampshire voters using a cloned version of President Biden’s voice, telling people not to vote in the primary. The whole thing was traced back to a political consultant who used ElevenLabs to make the audio. Cost him about a dollar. One dollar. The FCC ruled that AI-generated voices in robocalls count as “artificial” under the Telephone Consumer Protection Act, which makes them illegal without consent. Enforcing that, though? Completely different problem.
How Bad Is the Political Deepfake Problem?
Pretty bad. And getting worse fast.
Slovakia’s parliamentary elections in September 2023 offer a good case study. Just 48 hours before polls opened, a deepfake audio clip surfaced online that supposedly captured liberal candidate Michal Simecka talking about rigging the election and — of all things — raising beer prices. Totally fabricated. But it spread like wildfire on social media during a mandated quiet period when candidates couldn’t publicly respond or push back. Simecka’s party lost. Whether that audio clip was the deciding factor is debatable. The timing and impact, though, seem hard to argue with.
Since that incident, political deepfakes have popped up in elections in Argentina, Bangladesh, Indonesia, Pakistan, and the United States. Same pattern every time. A synthetic clip drops at just the right moment. Spreads faster than anyone can fact-check it. And even after it gets debunked, the doubt lingers. Researchers at Oxford’s Internet Institute have a term for this: the “liar’s dividend.” Once people know deepfakes exist, real evidence can be waved away as fake, and fake stuff gets presented as real. The technology doesn’t even need to be used in a specific case to damage trust — just knowing it’s out there poisons everything.
I’ve talked to misinformation researchers who are genuinely scared about the 2026 midterms in the U.S. The tech has improved enormously since 2024. Detection tools are locked in an arms race they can’t seem to win. And social media companies have gutted the trust and safety teams that might’ve caught synthetic content before it went viral. No one I’ve spoken with has a clear answer for what to do about it.
What About When Deepfakes Are Used for Legitimate Stuff?
Good question, because not every use is malicious. Hollywood’s been doing face replacement for years. De-aging Robert De Niro in “The Irishman.” Bringing back Peter Cushing as Grand Moff Tarkin for “Rogue One.” Letting Paul Walker’s character continue in “Furious 7” after the actor’s death. Studios have used this tech respectfully in plenty of cases.
But capabilities have jumped way beyond what Industrial Light & Magic was pulling off even a few years back. Now you can generate entire synthetic performances that are almost impossible to tell apart from real actors on screen. That jump is exactly what drove actors to the picket lines during the 2023 SAG-AFTRA strike. Performers were worried — rightly, I’d say — that studios would scan their faces and bodies during one session, then use AI to create unlimited performances without ever paying them again. The resulting contract included consent and compensation provisions for digital replicas. Enforcement hasn’t really been tested yet, though. And the protections only cover SAG-AFTRA members. Voice actors in smaller markets, international performers, independent content creators? They’ve got nothing.
Music’s dealing with the same mess. AI-generated songs mimicking Drake, The Weeknd, and other big names have gone viral on TikTok and Spotify. Back in April 2023, a track called “Heart on My Sleeve” — using AI-cloned vocals of Drake and The Weeknd — racked up millions of streams before Universal Music Group got it taken down. The legal situation is, from what I can tell, genuinely murky. Copyright law covers specific recordings and compositions. But someone cloning your vocal identity with AI? The law doesn’t clearly address that yet.
Can Detection Tools Actually Catch Deepfakes?
Sometimes. Not reliably enough. And it’s getting harder.
Early deepfakes were easy to spot. Weird blinking. Lighting that didn’t match. Blurry edges around faces. Teeth that looked like they came out of a Nintendo 64 game. Those tells are mostly gone now. Modern deepfakes handle micro-expressions, skin texture, hair movement, and lighting with a level of accuracy that’s, honestly, unsettling.
Detection tools do exist, and some of them are clever. Microsoft’s Video Authenticator goes through video frame by frame, hunting for blending boundaries and grayscale inconsistencies that human eyes can’t pick up. Intel built something called FakeCatcher that uses photoplethysmography — basically analyzing the tiny color changes in skin caused by blood flowing through capillaries. Deepfakes don’t reproduce those changes accurately. Researchers at the University of Buffalo’s Media Forensic Lab examine spectral frequencies and compression artifacts. These approaches work, but they need access to the original media file, processing time, and someone who knows what they’re doing.
Here’s the real problem, though: the whole setup is lopsided. Making a deepfake takes minutes. Checking whether something’s fake can take hours. And every time detection tools figure out a new tell, the generation models get updated to fix it. Hao Li, a computer science professor at UC Berkeley who’s probably one of the top deepfake researchers in the world, told me something I keep coming back to: “Detection is a losing game in the long run. We’ll always be playing catch-up. The real solution has to be provenance — proving that content is authentic at the point of creation, rather than trying to prove it’s fake after the fact.”
What’s This Provenance Idea, and Does It Actually Work?
Li’s comment gets at what a lot of researchers think is the better long-term bet. Instead of trying to catch fakes after they’re made, you build a verifiable chain of custody for authentic content. Prove something’s real from the moment it’s captured.
The Coalition for Content Provenance and Authenticity — C2PA — is the main group pushing this forward. Adobe, Microsoft, Intel, and the BBC founded it. They’ve built an open standard that bakes cryptographic metadata into media files right when they’re created. What device took the photo. When and where. Whether anyone’s edited it since. All signed and verifiable.
Camera makers are starting to get on board. Nikon, Sony, and Leica have released cameras with built-in C2PA signing. Adobe’s Content Credentials system shows provenance data directly inside Photoshop and other Creative Cloud apps. Google announced in late 2025 that Android devices would support C2PA metadata natively starting with Android 16. All good steps. But adoption isn’t universal yet, and the standard only works if platforms actually check and display the provenance information. Most social media companies haven’t consistently committed to doing that.
Watermarking is another angle. Google DeepMind’s SynthID tool embeds invisible watermarks into AI-generated images and audio. Specialized tools can detect these marks, but they don’t affect how the content looks or sounds to regular people. Meta’s built similar watermarking into its AI-generated content. The catch? Watermarks can sometimes be stripped out or degraded by screenshotting, compression, or format conversion. And watermarking only labels AI-generated content — it doesn’t do anything to verify that human-made content is genuine.
Where Do Laws Stand on All This?
Honestly? Kind of a mess.
In the U.S., there’s no federal law specifically targeting deepfakes. Several bills have been introduced. The DEFIANCE Act, which showed up in January 2024, would give victims of non-consensual deepfake pornography a federal civil cause of action. The AI Labeling Act would require disclosure when content’s AI-generated. Neither has passed as of early 2026. Some states have moved faster — Texas, California, Virginia, and about a dozen others have laws addressing specific deepfake uses, mostly revenge pornography and election interference.
Europe’s AI Act started taking effect in stages beginning in 2024. It includes rules requiring AI-generated content to be labeled. China’s gone further with its Deep Synthesis Provisions, mandating consent from anyone whose likeness gets used in synthetic media and requiring visible labels on all deepfakes. These sound good on paper. Cross-border enforcement, though, is basically impossible. A deepfake made in a jurisdiction with zero regulations can reach every corner of the internet in seconds.
Legal scholars I’ve spoken with tend to use words like “wildly inadequate” to describe where things stand. Danielle Citron, a professor at the University of Virginia School of Law and one of the foremost legal minds on this topic, put it to me this way: the law always lags behind technology, but the gap here is especially dangerous. We’re talking about something that can undermine elections, destroy reputations, and enable fraud at massive scale, and our legal systems were designed for a world where seeing was believing.
Who’s Actually Getting Hurt the Most?
This is the part that’s hardest to write about, and I think it gets overlooked in the coverage about politics and financial fraud.
A 2023 study by Home Security Heroes found that 98% of deepfake videos online are pornographic. And 99% of victims are women. The technology is overwhelmingly being used to take someone’s face from their social media photos and paste it onto explicit content without consent. The psychological damage — and I’ve read firsthand accounts that are difficult to forget — is severe and long-lasting.
South Korea hit a crisis point in 2024 when it came out that students, many of them minors, were creating and distributing deepfake sexual images of classmates and teachers through Telegram channels. Over 500 schools were affected. Emergency legislation followed, but the incident revealed how deeply the problem had already spread. Similar patterns have been documented in schools across the U.S., the U.K., Australia, and India. The tools are so simple to use that children are creating this content, often without grasping what they’re actually doing or the harm they’re causing.
For victims, the aftermath is devastating in a way that’s hard to overstate. Once a deepfake image gets shared, pulling it off the internet is practically impossible. Search engines might delist links, but the files persist on servers, in private group chats, on platforms in countries that don’t honor takedown requests. Anxiety, depression, social withdrawal, career damage — the toll looks a lot like other forms of image-based abuse, except it’s compounded by knowing that the technology to make these images is only getting more accessible. Not less.
So What Can a Normal Person Actually Do About Any of This?
Fair question. And the answer’s kind of frustrating because there’s no silver bullet.
Start with skepticism. Not cynicism — skepticism. If a video or audio clip surfaces right before an election, or seems perfectly engineered to make you furious, or arrives without clear sourcing, pause before sharing it. See if established news outlets are covering the same claim. Track down the original source if you can. This sounds basic. It is basic. Most people still don’t do it.
Shrink your deepfake attack surface. More photos and videos of you floating around publicly online means it’s easier for someone to build a convincing fake version of you. I’m not saying delete everything — that ship’s sailed for most of us. But maybe think twice about posting high-resolution images and videos with clear audio. Adjust your privacy settings. Limit who can download your content. Small steps, but they add up.
Back provenance standards when you can. If platforms and devices offer content authenticity features, use them. When you’re trying to figure out whether something’s genuine, look for C2PA credentials or other provenance markers. Push the platforms you use to implement and display provenance data. User pressure is one of the few things that actually gets tech companies to move.
And advocate for better laws. Contact your representatives. Support organizations like the Cyber Civil Rights Initiative and the Electronic Frontier Foundation that are working on balanced policy approaches. The regulations that get written in the next few years will probably shape how this technology is governed for a long time.
Is Anyone Building Defenses That Might Actually Work?
Some people are trying. Whether they’ll succeed is an open question.
I visited the DARPA-funded Semantic Forensics (SemaFor) program last year. Researchers from multiple universities and private companies are building integrated detection systems that combine several analysis methods — facial dynamics, audio spectral analysis, linguistic patterns, metadata checks — into single platforms. The idea’s to move past relying on one signal and toward multi-modal authentication that’s much harder for generators to beat. Seems promising. Hard to say yet whether it’ll hold up as the generation side keeps improving.
Startups are in the mix too. Reality Defender, a New York company founded in 2021, offers real-time deepfake detection for enterprise video calls. Exactly the sort of thing that might’ve caught the Arup scam before $25 million disappeared. Truepic, based in San Diego, provides photo and video verification using cryptographic provenance — insurance companies and humanitarian groups have adopted it. Hive AI sells content moderation APIs with deepfake detection baked in, and several social media platforms already use them.
But — and I keep landing on this “but” — every single engineer I’ve talked to says the same thing. Detection is necessary. It’s also not enough on its own. You need technology, regulation, platform accountability, media literacy in schools, and some kind of cultural norm around responsible use and sharing of synthetic media. None of those pieces work alone. Maybe all of them together could work. Maybe.
Where Is This Heading Over the Next Few Years?
Real-time deepfake generation — swapping someone’s face and voice during a live video call, not in a pre-recorded clip — already works in lab settings and some semi-commercial tools. As computing power grows and models get more efficient, doing this during an ordinary video call will become trivially easy. That changes the whole threat model. Being skeptical of a video circulating on social media is one thing. Doubting whether the person you’re speaking to right now, live, on a call, is actually who they look and sound like? That’s something else entirely.
This stuff will probably collide with augmented reality and virtual reality in ways we haven’t fully thought through yet. In immersive environments, the line between what’s “real” and what’s “synthetic” gets even blurrier. And generative AI models keep getting better, so the gap between authentic and synthetic media will keep shrinking — possibly to a point where even the most advanced forensic tools can’t reliably tell the difference.
I don’t have a clean ending for this because there isn’t one. Deepfakes represent a deep challenge to something we’ve taken for granted for over a century — that photos show real events, that videos capture actual moments, that the voice on the phone belongs to who you think it does. We built journalism, courts, law enforcement, and democratic elections on the assumption that recorded media could serve as evidence. That assumption is eroding. What replaces it, I’m not sure anyone knows yet. The engineers working on this are talented and driven, but they’ll tell you themselves: they’re running uphill. And from what I’ve seen recently, the slope just keeps getting steeper.
(0) Comments