Deepfake

Poetry of reality Science

We must know. We will know.

Physics Chemistry Biology Astronomy Geology

A view from the shoulders of giants.

Science and religion Politicization of science Higgs boson Absolute zero Radiometric dating Confidence interval Steven Dutch Leap year v - t - e

“”In the future, entertainment will be randomly generated.

—Larry the Cucumber, VeggieTales ("The Wonderful World of Auto-Tainment!")

Deepfakes (a term combining the machine learning technique "deep learning", and "fake") is essentially the creation of media using machine learning techniques. There is widespread concern that the use of deepfakes will make it a lot easier to spread disinformation, by allowing users to create fake media (including media that was much more difficult to alter in the past, such as videos) that would be much more difficult to discern from the legitimate product.

Techniques

Deepfakes can be seen as a subset of machine learning computer techniques.

One way to make a "DeepFake" is to use a generative adversarial network, or GAN. GANs work by creating two neural networks. The first network, known as the generator, looks at training material fed into it (eg, photographs of the subject in question that you would want to fake) and attempts to produce artificial outputs by trying to mimic them. The second neural network (the discriminator) determines whether the output of the generator is "real". If the discriminator rejects the attempt, the generator tries again. Eventually, the generator learns the pattern well enough to fool the discriminator; thus, the algorithm is now trained and ready to generate the fake media.

This is just one example; other machine learning algorithms can also be applied to produce a deepfake.

Limitations

There are limitations to any machine learning technology, and that includes deepfakes. The key is that machine learning algorithms typically need to be trained somehow, and this is not typically trivial. To produce a convincing video deepfake as of this writing (2019), you typically need a significant amount of computational power, a large set of photographs to train with (ideally within a narrow scope of parameters -- ideally the characteristics of the face of the person you are using to fake is pretty similar to what is in the image you doctor), and fairly good computer skills. Deepfake applications typically require a significant amount of hand holding and data massaging in order to obtain reasonable quality. This tutorial on one of several applications to assist in making a deepfake, DeepFaceLab, perhaps demonstrates the relative difficulty of the current process.

On the other hand, the technology is currently mature enough where much of the complexity can be hidden from the user and processed on the backend, if developed properly. Recently, a Chinese mobile application called Zao made waves by creating an app that made creating a deepfake video a lot easier to do, allowing you to insert a face image (either from an uploaded file or a selfie) on short video clips. With Zao, there are limitations in the image you can use (the image you upload has to be a certain quality and characteristic, and at present the application tries to block any image it recognizes as being from a public figure), and the results are reportedly not perfect (occasionally somewhat jerky or occasionally odd). Still, it does show the potential power deepfakes may have for agents with enough backend computing power and professional development.

Alternatives to deepfakes

It is not necessary to go through the trouble of making a deepfake to spread disinformation. Many other techniques exist to spread fake media, including:

• Photoshop and other digital editing programs.
• Audio editors, using techniques such as time stretch and pitch shifting.
• Selective editing.
• Using look-alikes or sound-alike actors or actresses in staged productions.
• Fake news clickbait.
• Fake news websites.
• Fake news Youtube videos.

As many hoaxters have shown over the years, you don't need the most accurate media quality or plausibility to spread disinformation. So while there is much concern in media of lesser repute about this "shocking" technique that will "start plaguing your social media" with fakes and will be "accessible to everyday people within months", the truth is the current technological barriers are at this moment rather high, too high for most disinformation agents to bother with, especially given the much easier alternatives. As a result, very few political, conspiratorial, or disinformation deepfakes exist currently. There certainly is good justification for raising concerns over privacy and disinformation and other ethical and moral issues to be applied as the technology improves, but the technology currently is too immature for major concern just yet, with one major exception.

Current applications of deepfake technology

Pornography

Being a fiddly, time-consuming technology, deepfake applications are of limited use for most people, for now. However, they have found a useful conduit among a certain type of early technology adopter who have a lot of time on their hands and, tangentially, have a strong perverted desire to insert their favorite A-list celebrity or K-pop star into a pornographic video. Of the relatively small amount of deepfake videos on the web, 96% of them are pornography.

The most notorious deepfake application that was actually easy enough for the masses to use was, likewise, pornographic oriented: DeepNude. Instead of swapping faces (like Zao), DeepNude swapped out bikinis that women were wearing on a beach picture that the user uploaded, and replaced it with AI generated naughty bits. Like Zao, this was relatively easy to use. The programmer quickly took down the application one day after launching it, after figuring out that, you know, an application like this is going to be pretty popular with unsavory types eager to use it to create revenge porn. Oops.

Concern on the ability to create non-consensual pornography with deepfake technology therefore is very legitimate, and even more so when enterprising pedophiles take advantage of AI algorithms to churn out sexualised imagery of children and youths for them to lust upon. While most AI technology companies such as OpenAI and ChatGPT have taken steps to curb AI child porn by detecting suspicious prompts and blocking them on sight, these have been circumvented from time to time; that being said, the mere fact that AI image generation algorithms have used datasets kanged from countless images all over the web, some of them containing photos of minors who definitely did not consent to their use, made some people leery of such art generators.

Many online communities and legislatures have already reacted to this phenomenon, with Reddit banning pornographic deepfake subreddits, several legislatures rushing to make non-consensual pornographic deepfakes illegal, and Pornhub taking steps to block deepfake videos… poorly.

AI voice cloning

Similar concerns were also expressed regarding deepfake audio of musicians and celebrities. Besides the obvious risk of abusing deepfake voice models for disinformation and hate speech–case in point 4chan edgelord trolls using samples of Emma Watson's voice to make it appear as though she is reciting portions of Adolf Hitler's Mein Kampf–a number of voice actors and musicians have voiced (pun intended) their objections to third parties synthesizing samples of their voice without their consent, such as with veteran actor Rick Pasqualone who took offense at YouTuber Mafia Game Videos using the voice of Pasqualone's character Vito Scaletta from Mafia II on a series of videos about the Mafia series, labelling the faux-Scaletta deepfake voice as "soulless" and "without any artistic merit" as well as Hitman voice actors David Bateson and Jane Perry who both voiced their opposition to an exception to a UK law effectively giving AI creators carte blanche to synthesize someone else's voice without prior approval. Conversely, there are entertainers who either approve of their voice being synthesized (provided that the voice model is not used out of malice) or have had their voice cloned, such as the singer Grimes who specifically demanded she gets part of the revenue from the use of her voice, and Snoop Dogg who lent his distinctive sing-song voice to the text-to-speech service Speechify. Disney animator and Phineas and Ferb series creator-slash-voice actor Dan Povenmire also publicly expressed his approval of his character Doofenshmirtz being deepfaked by enthusiasts as well. Doug Cockle of The Witcher fame also expressed concern about AI voice cloning, but was also appreciative of modders who produce otherwise good-faith AI clones of his character Geralt of Rivia, stating "It's not that I want to punish modders. What they put out is, for the most part, interesting, cool fan-made stuff that just celebrates the world that they love, and that's to be celebrated. But there's a line there somewhere, and where exactly that line is what the AI debate is really, really starting to tease out of the industry." Fellow voice actor Cissy Jones came up with a company known as Ethovox (formerly Morpheme.ai) which aims to provide a compromise between entertainers and generative AI and has partnered with acting unions to ensure that their talents' voices aren't simply used indiscriminately.

As a result of the controversies surrounding AI voice models, some sites such as ElevenLabs have paywalled their voice cloning feature to mitigate potential abuse, or in the case of the community-run AI voice synthesis site Uberduck, put up a blacklist of personalities who either requested to have their voices removed from the site due to their (personal) objections, or in the case of politicians such as Donald Trump, were banned outright due to even more serious risks of misinformation. Uberduck previously hosted voice models of politicians on their website for the purposes of making satirical parodies using their voices, but was forced to remove them for fairly obvious reasons.