Chinese police have detained a man for creating a fake news story using an AI-powered chatbot. The arrest comes as one of the first enforcement actions under China’s newly enacted law regulating AI-generated “deepfakes,” fabricated digital media that appears realistic. The individual, identified only as Mr. Hong, allegedly used ChatGPT to create a fake news article about a train crash in Gansu province, falsely claiming the deaths of nine construction workers. The story quickly spread through 21 social media accounts owned by a southern Chinese media company.
Although ChatGPT is typically inaccessible in China due to internet censorship, individuals can bypass these restrictions using virtual private network (VPN) software. The police report did not specify how Hong managed to access ChatGPT. Gansu security officials discovered the article’s falsehood after it had already garnered 15,000 views. Consequently, the police conducted a raid on Hong’s residence to gather evidence and took measures to restrict his freedom, known as “criminal coercive measures.”
China’s new deepfake law came into effect on January 10 to combat fake media generated through deep synthesis technologies like machine learning and virtual reality. The law prohibits various categories of deepfakes that endanger national security, harm the nation’s image, disrupt societal public interest, or disturb economic and social order. It explicitly bans the use of such technologies for producing, publishing, or transmitting fake news. However, the law’s definitions for many forbidden classes of deepfakes remain somewhat vague.
The detention of the ChatGPT user and the enforcement of the new deepfake law demonstrate China’s efforts to regulate the dissemination of fake news and fabricated media. By taking action against those who create and spread false information, the Chinese authorities seek to protect national security, maintain social order, and safeguard the credibility of media and information sources.
China’s state-run media has recently emphasized the importance of combating fake news and false information. President Xi Jinping has called for more extensive efforts to combat “rumors, fake news, and illegal publications.” Authorities have implemented various measures to crack down on the spread of misinformation, including a ban on the use of anonymous social media accounts and strict regulations on live streaming platforms.
China’s efforts to regulate AI-generated deepfakes and combat fake news align with similar efforts by other countries. The United States has enacted legislation to regulate deepfakes, including the Malicious Deep Fake Prohibition Act and the Deepfake Report Act. The European Union has also considered regulations on deepfakes and other forms of manipulated media. The growing threat of disinformation and the potential harm it can cause to individuals, societies, and national security has prompted increased efforts to combat the spread of false information.
China’s recent detention of a ChatGPT user for creating a fake news story using an AI-powered chatbot underscores the country’s efforts to regulate AI-generated deepfakes and combat fake news. The new deepfake law, which came into effect earlier this year, aims to combat various categories of deepfakes that endanger national security, harm the nation’s image, disrupt societal public interest, or disturb economic and social order. By taking action against those who create and spread false information, the Chinese authorities seek to protect national security, maintain social order, and safeguard the credibility of media and information sources. The detention of the ChatGPT user reflects the growing global concern over the spread of disinformation and the need for increased efforts to combat it.
