In a major legislative step to combat the growing misuse of artificial intelligence, South Korea has passed a bill criminalizing the possession or viewing of sexually explicit deepfake images and videos.
Lawmakers approved the measure on September 26, 2024, following rising public concern over illegal content circulating on online platforms, particularly Telegram.
The Scope of the Law
Under the new legislation:
Anyone purchasing, saving, or watching sexually explicit deepfakes can face up to three years in prison or fines reaching 30 million won ($22,600).
Creating such material with intent to distribute has already been punishable by five years in prison or fines of 50 million won under the Sexual Violence Prevention and Victims Protection Act.
Once enacted, the maximum sentence for producing such content will increase to seven years, regardless of intent.
The bill now awaits approval from President Yoon Suk Yeol before taking effect.
Also Read: South Korea Moves to Enforce Six-Month OTT ‘Holdback’ Window to Protect Theatrical Film Industry
Rising Number of Deepfake Sex Crimes
South Korean authorities have reported a dramatic increase in deepfake-related crimes:
In 2024 alone, police handled over 800 cases, compared to 156 cases in 2021, when data collection began.
Most victims and perpetrators are teenagers, highlighting the vulnerability of young individuals to both producing and being targeted by these materials.
The proliferation of deepfakes, particularly sexual content, has prompted urgent calls for tougher legal measures.
Telegram group chats, where such material was widely shared, were a major trigger for the legislation.
Global Context: How Other Countries Are Responding
South Korea is not alone in addressing deepfake-related crimes:
- United States: Congress is debating bills allowing victims of nonconsensual sexual deepfakes to sue perpetrators. Another proposal would criminalize the publication of such imagery and require tech companies to remove offending content.
- Social Media Platforms: Earlier in 2024, X (formerly Twitter) blocked users from searching for Taylor Swift after sexually explicit deepfake images of the singer circulated online, demonstrating platforms’ growing role in content moderation.
The global trend indicates an urgent need for legal frameworks, tech enforcement, and public awareness to combat AI-driven sexual exploitation.
Implications for Privacy, Technology, and Society
Legal experts argue that South Korea’s law could serve as a model for other countries grappling with deepfake misuse. Key considerations include:
- Privacy Protection: Criminalizing the consumption of nonconsensual deepfakes reinforces the importance of consent in digital spaces.
- Technology Regulation: AI-generated content is becoming increasingly sophisticated, requiring proactive legislation to prevent exploitation.
- Youth Awareness: With teenagers both creating and consuming such content, education on digital ethics and consent becomes critical.
Police investigations into platforms like Telegram are ongoing, assessing whether encrypted services have been complicit in distributing illicit material.
Looking Ahead
The enactment of this law is expected to:
- Deter the creation and consumption of sexually explicit deepfakes in South Korea.
- Provide a legal recourse for victims seeking justice against nonconsensual content.
- Encourage tech companies and social media platforms to implement stricter moderation and detection systems.
However, challenges remain. Experts caution that enforcement is difficult due to the anonymity of the internet, the ease of content sharing, and the cross-border nature of AI content creation.
South Korea’s legislation marks a significant step in regulating the misuse of AI technology in the sexual domain.
By criminalizing the possession and consumption of sexually explicit deepfakes, the country aims to protect individuals’ privacy, especially vulnerable youth, and curb the exploitation potential of AI-generated content.
As other nations monitor South Korea’s approach, the law may become a global reference point in balancing innovation with ethical responsibility in the age of artificial intelligence.
