Grok Restricted in Malaysia and Indonesia over Deepfake Misuse Concerns

Elon Musk’s xAI chatbot faces regulatory action after authorities flagged its image-generation tool for producing non-consensual and sexually explicit deepfake content.

The artificial intelligence chatbot Grok, created by Elon Musk's company xAI, has been restricted in Malaysia and Indonesia due to growing concerns about its use in producing sexually explicit and non-consensual deepfake photos, according to authorities.

Both countries in Southeast Asia claimed to be taking action after learning that Grok's image-generation capabilities were being abused to create altered images of actual people, including women and children, in explicit or sexually suggestive situations without their consent. Such use, as per officials, is a breach of legal safeguards, human dignity and privacy.

Grok's access was temporarily prohibited, according to Indonesia's Ministry of Communication and Digital Affairs, which claimed the chatbot lacked adequate security measures to stop the creation and distribution of illegal content. According to the ministry, the action was required to safeguard citizens online while further assessments are conducted.

After the AI tool was repeatedly used illegally, especially to produce offensive or non-consensual altered photographs, Malaysia's Communications and Multimedia Commission soon blocked access to it. Kuala Lumpur regulators claimed to have sent notifications to X Corp. and xAI requesting more stringent moderation procedures, but the replies were judged inadequate.

A larger worldwide outcry against Grok coincides with the restrictions. In order to protect users from illegal content, including intimate deepfakes, authorities in the UK have opened an investigation into whether the platform's output violates national laws. If Grok does not put in place sufficient controls, some UK officials may consider imposing more limitations.

In response to criticism, Grok, which is available through a interface and integrated into the social media platform X, recently restricted its image creation and editing capabilities to paying subscribers. Regulators and advocates of digital rights contend that these guardrails are insufficient.