Deepfakes are finally here and we just became pioneers in using deepfake technology for election campaigning. In the recently held Delhi elections, a BJP leader delivered a speech in Hindi. This was then converted to a speech in Haryanvi not by simply dubbing it but by replacing the original audio as well as using software to modify the leader’s lips to fit the sounds of Haryanvi as well. The result is a proper lip sync that makes the actually misleading video seem quite real.
Such feats don’t scratch the surface of what deepfake technologies are capable of. The powerful idea in such a video is that the original actor’s face in the entire video clip can be replaced with that of another. The replacement is ‘perfect’ in that the ratio of the size of the face to the body looks realistic throughout the video, even if from different angles and lighting conditions, and all movements look authentic. A suitably talented engineer can also alter the tone and the pitch to make the audio match that of the new actor. Thus, we have a seemingly ‘natural’ video whose true colours become visible only with stricter scrutiny.
Deepfake software works using machine learning. Faces are converted to data (0s and 1s) preserving a record of the various facial features such as the size of the nose, the curve of the eyes, the distance between the lip and the nose, etc. These details represent the different expressions a particular face makes in the video. Using this information, purpose-built programmes can ‘learn’ how the face will look under different circumstances, while also keeping track of posture, motions and movements, dimensions and perspectives. The ‘deep’ of ‘deepfake’ stands for for deep learning, the corresponding didactic technique the machine employs to teach itself about the video, and thus produce the ‘fake’.
This technology is useful for creating satirical videos, spoofs or to generate footage when an actor is not available (or dead). However, the nature of the technology is such that it is primed for misuse.
One of the first prominent examples of a deepfake video appeared in 2018: of US President Barack Obama using foul language. The comedian Jordan Peele had produced it as well as voiced Obama. As with the Delhi election video, the face is that of Obama and the voice is Peele’s, and the two were stitched ly together by a deepfake tool, creating the spectacle of a foul-mouthed Obama. This is probably the simplest way to abuse the technology: get a politician to “say” things she never said. If you’re not able to locate a good mimic, software could fix that too through simple audio morphing.
So we can, for example, grab historical footage and reprocess them to produce sensational videos of leaders saying things completely opposed to what they originally said. Imagine Jawaharlal Nehru appreciating superstitious beliefs; Adolf Hitler declaring his love for the Jews; or a pro-Nazi rant delivered by Winston Churchill. History is thus easily falsifiable. Maybe even the present: imagine a leader of a global power declaring war on video against another power today, or a “seditious, anti-national” speech being planted on a political leader’s person or office to have her arrested.
Here is an example of a full deepfake with faces swapped and modified audio: