Tech tools to fight deepfakes

Sachin Tendulkar's fans were surprised when they found the Master Blaster, known to be a choosy endorser, promoting a gaming app. The cricketer, too, was dumbfounded — for he had nothing to do with the advertisement circulated on social media in January 2024. It turned out to be a deepfake video.

The word deepfake, a portmanteau of deep learning and fake, refers to media — text, audio or visual — that has been digitally altered to create a false narrative. In recent times, several noted figures — including the Pope — have been targeted in deepfakes (see box:'Rishi and the Pope'). The problem is not confined to the movers and shakers. In December 2023, an elderly couple living in Faridabad, on the outskirts of Delhi, received a call on WhatsApp from someone who said he was a U.S. police officer. He said their 35-year-old son, an engineer in the U.S., had been arrested in a rape case and demanded ₹46 lakh to set him free. The couple sent the money after receiving an audio file in which they could hear their son sobbing. The call was fake, as was the audio. The police said a voice-cloning technique had possibly been used to con the parents.

Companies and researchers are working on tools to detect fake images and posts, for deepfake has the potential of scamming people, creating disharmony, or triggering violence. While mechanisms for video, audio, and text alteration and editing have been around for a while, the widespread access to artificial intelligence (AI) tools has made it easier for anyone to alter or manipulate media. "There's going to be more persuasive content, a whole lot more of this (deepfake) content, and it can be done by anybody," says Mohan Kankanhalli, Provost's Chair Professor of Computer Science at the National University of Singapore (NUS).

Anyone, for example, can use online tools such as Deepswap and FaceMagic to swap one face for another in a video; Stable Diffusion can generate realistic-looking, false images. OpenAI's DALL - E can create images from text-based inputs — it can, for instance, conjure up an image of someone getting a coveted award. All that one has to do is type in the relevant words.

Although these images look authentic, there are some detection tools that can see through the deception. The tools look for a range of discrepancies that can be contextual, spatial, textural — and more. One typical example of an inconsistency that detectors look for is the direction of a gaze — if a person's gaze does not match the direction the eyes face, then it is a sign of manipulation. When a person faces a camera, the gaze is towards the camera, too. But when an actual face is replaced by a fake one, it is possible that the look is not towards the camera. Another red flag is the disparity between a mouth opening and shutting and the intensity and frequency of the speech. When an image is swapped with another, there may be a mismatch between the compression factors of the two images, since JPEG images are often compressed.

Intel's FakeCatcher detects deepfakes by using the remote photoplethysmography technology to analyse blood flow in the pixels of an image. Signals from multiple frames are analysed before a video is pronounced fake. Facial movements and a change in expressions and speech result in concomitant blood-flow changes in the face. The method can detect these changes by measuring the amount of light absorbed or reflected by different parts of the face. In January 2024, computer security software company McAfee released an advanced detector to specifically target audio deepfakes.

FakeBuster, a fake detector released by the Indian Institute of Technology (IIT) Ropar in 2021, detects if videos of participants in a video call are real or manipulated. FakeBuster uses screen recordings of video conferences to train deep-learning models to identify fake videos and participants. In a January 2024 paper in Engineering Applications of Artificial Intelligence (bit.ly/deepfake-dtu) researchers from the Delhi Technological University (DTU) describe a novel detector named Face-NeSt that dynamically chooses between a range of features based on their relevance to the context to identify face manipulation. The researchers extract information on inconsistencies in the image from different domains — spatial, textural, optical flow, and so on. This information helps distinguish between the real and the fake. They are now in the process of patenting this tool and working out the modalities of its release, says Dinesh Vishwakarma, Professor and Head, Department of Information Technology, DTU.

Start-ups — including Sensity AI of the Netherlands and Sentinel of Estonia — have come up with detectors, too. Gujarat-based start-up Kroop AI has developed a detector called VizMantiz. "It is positioned for BFSI (Banking, Financial Services, Insurance) and social media platforms. Current users are large organisations in these spaces," says Jyoti Joshi, Founder and CEO of Kroop AI.

With the rise of AI-generated deepfakes, the variety and numbers of fake posts have increased. "From the algorithm perspective, this means we have to up the game now, because there is so much variety possible in defects. It is a more complex problem now, but because data (from many different deepfakes) is easily available, there is a silver lining, too," says Abhinav Dhall, Associate Professor, Flinders University and IIT Ropar.

At NUS, Kankanhalli is trying to understand how the new breed of AI-generated deepfakes differs from camera-generated videos and images. He uses a traditional analysis technique called Fourier analysis, in which signals from the images are decomposed into sine and cosine waves. The high-frequency signatures correspond to details in the image. The analyses show that videos and images created with generative AI have fewer high-frequency components than actual images.

Labelling content at source is another way of battling deepfake. When a camera creates a picture or a video, it also generates a label or signature certifying that it has not come from any AI tool. Each time the content is modified or edited, additional sets of signatures are added to establish the authenticity of the media — text, audio or video. These signatures, also called content credentials, help investigators and others figure out when and how something was generated, and whether it had been edited or AI-generated.

In February 2021, software giants Microsoft and Adobe formed an alliance with three other technology companies, Arm, Intel and Truepic, to launch the Coalition for Content Provenance and Authenticity (C2PA). C2PA is building an open-source technical standard that can be used to assess the authenticity of different types of digital media. Any image or video marked with C2PA's icon — a pin with the letters 'cr' — is C2PA-compliant. By clicking on the icon, a user will learn about the origin and history of that piece of media. This information is kept secure and tamper-proof through cryptography.

In October 2023, Leica launched a new M11-P camera with built-in hardware for content credentials. Each time an image is captured, data on where, when and who took the image, along with information on the make and model of the camera, will be stored with the image in a secure format. A digital signature is generated for each image, which can be later used to retrieve the stored data about the image. If the image is modified, the changes will show up. Also in October, Qualcomm, in collaboration with Truepic, released a new chipset that stores and secures provenance data.

Content credentials enable internet users and media consumers to distinguish between authentic and AI-generated images. It will not only help identify fake media but also enable content creators to establish the authenticity of their work in cases where genuine media is being labelled as fake. Currently, C2PA is an opt-in facility — that is, it's still optional. Its full benefits will only be unlocked when it becomes mandatory. Social media platforms are yet to opt for it.

According to the 2023 State of Deepfakes (bit.ly/deepfake-state), a report by the U.S.-based Home Security Heroes, there's been a 550% increase in online deepfake videos from 2019 to 2023. Europe-based Sumsub's identity theft report (bit.ly/identity-sumsub) states that the number of deepfakes increased tenfold from 2022 to 2023.

Most researchers stress the need to spread awareness about deepfakes so that people are more discerning about what they see. An 'always verify first' policy can curtail the limitless spread of unsavoury and misleading deepfakes. However, the 'always verify first' policy can also lead to an overall reduction in trust among the masses, even in the case of genuine news or posts.

"Deepfake detection is like an arms race," Kankanhalli says, referring to the fact that the detectors need to keep evolving to take on newer and more sophisticated deepfakes. Problems in today's AI-generated deepfakes could get ironed out in the future, and the detectors would have to be updated. Which is why he believes a more holistic approach is needed to resolve the problem. He is now part of a project on disinformation and its mitigation, in which human behaviour, psychology and patterns of information consumption are being studied for more robust solutions. Detectors must factor in that deepfake creators are skilled at exploiting human psychology. As solutions evolve, laws are also needed to deter people from making deepfakes. "If somebody has been found to use these generative AI tools or any such deepfake tools to create disinformation there are consequences for it, you can't just get away with it," Kankanhalli says.

Dhall points out that there can't be an "ideal universal" detector. "Detectors will have to be case-specific." For example, something that works well in identifying fake identities and data in a banking environment may not be suitable for social media.

It is unlikely that a one-shot solution to this problem will become available in the near future. But the quest is on, as the need for solutions is imperative. It's worth recalling that 'deepfake' nearly made it as Merriam-Webster's Word of the Year 2023. Providentially, it lost out to 'authentic'.

See also:

Fighting an infodemic