Text to speech systems nowadays are capable of cloning a person's voice directly from sampled recordings in real time (meaning you just have to get someone talking and it will replicate with sufficient accuracy their voice). It has come along because of the progress of deep learning — specifically, wavelet versions like WaveNet and Tacotron. These systems work by utilizing massive datasets, recorded conversations running into thousands of hours, for the engines to learn from and mimic human speech as closely and accurately as possible. This includes WaveNet, which is a Google initiative developed by DeepMind capable of producing speech with details like the original speaker's voice color, intonation and emotional nuances given to an extensive volume of data exceeding 44k hours.
Industries involving customer service, entertainment are making good use of real-time voice cloning. This is not magic: voice cloning for a smart assistant like Siri, Alexa, or IBM Watson actually enables more intimate interactions that lets a user get closer to a humanistic experience. These models are so streamlined that no more than five seconds of recorded speech is required to generate a highly accurate voice model. Another is NVIDIA's real-time voice synthesis platform, which can generate cloned speech in less than 50 milliseconds in applications such as video games, virtual environments and live customer service systems.
But the emergence of voice cloning sparked concerns around privacy and ethical use. Some of such attempts, dating back to 2020, included cloned voices in a financial scam — hackers used a generated voice of the CEO to conduct several fraudulent operations that resulted in losses in thousands and even millions of dollars. Summarily, attention increased and the calls for more regulation/oversight followed suit. The fact that even the FBI is issuing warnings around real-time voice cloning being used to steal identities and commit fraud shows just how quickly these technologies have managed to outstrip regulatory frameworks.
The tech continues to be developed, however: as the Verge points out, big tech companies have now invested millions into making voice cloning better — and making sure it's safe. Based on an analysis done by Intelzona, the global market for voice cloning is expected to achieve a compound annual growth rate (CAGR) of 17% during 2023–2028 period. This growth spikes as more mature markets across healthcare and education help drive market expansion in personalized voice interfaces, where the relative statistical inference accuracy of a voice assistant instantiates closer to an individual. Moreover, companies like Google and Microsoft have implemented security processes that prevent unauthorised clonning voices, by means of encrypted voice data.
Real-time voice cloning is an embodiment of technological advancement which presents vast possibilities, but its dangers merit a thought. Properly done, this might transform how we approach digital systems in every vertical industry. Read more on the real time voice cloning and go through it.