Researchers in the field of cybersecurity reveal an advanced artificial intelligence audio attack known as deepfake, which has the ability to seize control of real-time conversations.

IBM Security researchers have recently made a disturbing discovery regarding the ease with which live conversations can be hijacked and manipulated using artificial intelligence (AI). This new attack method, known as “audio-jacking,” utilizes generative AI and deepfake audio technology.
During their experiment, the researchers instructed the AI to process audio from two sources in a live communication, like a phone conversation. Once the AI detected a specific keyword or phrase, it was programmed to intercept the related audio, manipulate it, and then send it to the intended recipient.
According to a blog post by IBM Security, the experiment successfully demonstrated that the AI could intercept a speaker’s audio when prompted by the other person to provide their bank account information. The AI replaced the authentic voice with deepfake audio, providing a different account number. The victims in the experiment remained unaware of the attack.
The blog emphasizes that while executing this attack would require some level of social engineering or phishing, developing the AI system itself presented little challenge. In the past, creating a system capable of autonomously intercepting specific audio and replacing it with dynamically generated audio files would have required a complex computer science effort. However, modern generative AI is capable of handling this process with ease. The blog post states that only three seconds of an individual’s voice are needed to clone it, and the creation of deepfakes can now be done through APIs.
The threat posed by audio-jacking extends beyond tricking unsuspecting individuals into depositing funds into the wrong account. The researchers also highlight the potential for this technique to serve as an invisible form of censorship, allowing for real-time alteration of content in live news broadcasts or political speeches.