Author: americalatinanews.com

Fourteen-year-old Carolina has been on Roblox since she was 10, chatting and playing with other gamers on the platform. When the site rolled out mandatory age checks at the start of the year, Carolina was afraid she would lose access to some of her friends and group chats. She needn’t have worried — the software determined she was 16 or 17.

“Without any makeup, I did what the app asked: turned my head this way and that for the photo,” Carolina, who lives in São Paulo, told Rest of World. “The app said I was 16 to 17 years old. I was able to go back to my chats.”

Roblox is among the growing number of social media platforms installing age checks as governments around the world act to keep young users off social media sites and limit their access to content deemed inappropriate. The most commonly used methods include verification with a government-issued identity; estimation through biometrics such as facial recognition; and inferring the age of the user from their online behavior.

The methods are not foolproof. Facial recognition technology is known to be less accurate for women and people of color. On Roblox, the age check is meant to restrict who users can chat with. The 16 to 17 age group, which Carolina was assessed as belonging to, can chat with users in their group, as well as the ones immediately above and below theirs, or those in the 13–15 and 18–20 age groups. Being able to chat with older users exposes her to the risk of harassment and abuse — something California-based Roblox has come under scrutiny for. Of its more than 100 million daily users, nearly 40% are under the age of 13, some estimates show.

Age-verification technologies are not infallible. … Minors are using increasingly sophisticated techniques, including VPNs and AI-generated deepfakes or selfies.”

As a new child safety law comes into effect in Brazil from March, requiring platforms providing gambling, pornography, and other such content to verify ages, users are learning to circumvent the tech, Simone Lahorgue Nunes, founding partner at law firm Lahorgue Advogadas Associadas in Rio de Janeiro, told Rest of World.

“Age-verification technologies are not infallible,” she said. “Minors are using increasingly sophisticated techniques, including VPNs [virtual private networks] and AI-generated deepfakes or selfies. Conversely, overly stringent measures may drive underage users toward the dark web or services hosted in so-called digital havens, thereby exacerbating the very risks such regulation seeks to mitigate.”

Circumventing age verification doesn’t even need much sophistication. In a subreddit on bypassing the Roblox checks, users recommend fake IDs and AI-generated photos. “I used a fake ID of [Joseph] Stalin and it got accepted,” one user wrote. Another user posted a video on X that showed a crude face painted on a thumb going through the age check on Roblox and being assessed to be 13 to 15 years old.

The age check process “is designed for accuracy,” Roblox said in the statement announcing the rollout. The technology has been “tested and certified by third-party laboratories,” and the company constantly evaluates user behavior to determine if someone is “significantly older or younger than expected,” it said. TikTok’s new age checks use a combination of profile data, content analysis, and behavior to infer whether an account belongs to an underage user.

Since Australia last year barred those under 16 years from social media platforms such as Facebook, Instagram, X, and TikTok, at least a dozen countries including Malaysia, Spain, France, and the U.K. have said they are planning similar rules. The chief economic adviser in India, one of the biggest markets for social media companies, has also made a recommendation for such a law. 

Effective child safety online doesn’t require identifying every internet user. … Platform design choices around recommendation algorithms, data harvesting practices, and addictive features cause far more harm than anonymous access to information.”

With the focus on technology, there isn’t enough discussion around how these measures can disproportionately harm minorities and others who lack documentation or prefer anonymity online, Shivangi Narayan, who teaches sociology at the Thapar School of Liberal Arts and Sciences in Patiala, India, told Rest of World.

“Age verification would end anonymous accounts and anonymity on the internet as we know it,” Narayan said. “It is a potent tool to kill dissent and end the last vestige of protection that a lot of marginalized identities have online — people who cannot express themselves fully on social media because they could be harassed, trolled, become a victim of hate speech or in extreme circumstances, even killed.”

Privacy campaigners are also worried that the identification documents and biometric data used to determine a user’s age could be compromised, exploited, sold, or used for surveillance.

Last year, San Francisco-based Discord said the government photo IDs of about 70,000 users worldwide had been exposed through a third-party vendor. The compromised data included their names, email addresses, contact information, and payment history. Earlier this month, Discord — with over 200 million monthly active users — said it was rolling out “enhanced teen safety features” for users over the age of 13. These include deleting identity documents submitted for verification “quickly — in most cases, immediately after age confirmation.”

Some countries are opting for their own age verification systems rather than U.S.-based Jumio or Yoti from the U.K. Malaysia has its own technology, and Brazil plans to build one. India’s Signzy and Accura Scan have several global clients.

The focus on age checks diverts attention away from the more pressing concerns around social media platforms, Apar Gupta, founder-director of Internet Freedom Foundation, a digital rights organization in India, told Rest of World.

“Effective child safety online doesn’t require identifying every internet user,” he said. “Platform design choices around recommendation algorithms, data harvesting practices, and addictive features cause far more harm than anonymous access to information.”

In Malaysia, where the government has said it will soon introduce a law to restrict under-16s from accessing social media platforms, long-time Discord user Adam is concerned about the security of its facial recognition requirement, he told Rest of World.

“If your face is compromised, you can’t replace it,” the 17-year-old said. “Imagine having your biometric identity exposed forever.”

When the lights dimmed at Jaideep Sharma’s wedding reception in the north Indian city of Ajmer, guests expected to see a cheesy montage of the young couple in various attractive locations. Instead, they saw Sharma’s father — dead for more than a year — on the screen, smiling and blessing the newlyweds.

The video was created using artificial intelligence by a local creator Sharma found on Instagram. Using pictures of Sharma’s father, the creator produced a minute-long video in about a week, and charged about 50,000 rupees ($600), Sharma told Rest of World. It was worth it, he said.

“It was like a bombardment of emotions for everyone,” said the 33-year-old garment trader, who felt his father’s absence keenly at his wedding. “He was like a central force in the entire family. So when the video played, everyone was very happy and emotional at the same time.”

Sharma is among a growing number of Indians discovering the power of AI deepfakes to resurrect dead family members, create voice clones of the departed, and add absent guests to family celebrations. AI tools such as OpenAI’s Sora, Google’s Nano Banana, and Midjourney have made it easier to create images and videos that can fool even experts. Cashing in are entrepreneurs in small towns and cities, who have learned how to use these tools from YouTube tutorials and online forums.

Like Akhil Vinayak, a film buff, who posts deepfake videos of popular dead actors on Instagram for fun. A client in the south Indian city of Thiruvananthapuram approached him with an unusual request: Could he create a deepfake video of her dead mother-in-law blessing her baby?

“She wanted to surprise her husband,” the 29-year-old told Rest of World. “Her mother-in-law had passed away before the baby was born.”

Vinayak created a video showing the dead woman stepping down from heaven and visiting her son, then holding the baby she hadn’t met. The client was thrilled, and sent Vinayak a recording of the family’s stunned reaction. That video has more than 1 million likes on Instagram.

Such uses — and reactions — stand in sharp contrast to the growing pushback to AI-generated videos and voice clones, which are most commonly used for harassment, extortion, financial scams, political misinformation, and election manipulation.

For Vinayak’s clients, though, the deepfakes are not just practical but also deeply emotional, he said. Vinayak uses open-source models like Stable Diffusion and editing systems such as Adobe Premiere Pro to create them, charging about 18,000 rupees ($200) on average for  minute-long videos.

• 

• 

• 

They can be a challenge when clients have only old, damaged, or black-and-white photos of the person they wish to recreate, he said. It also requires an effort to get to know the person they are recreating.

“We have to work with the client to know about them and their behavior to create their closest online version,” said Vinayak, who now has a five-member team at his firm Kanavu Kadha, which means “stories from dreams.” He plans to launch an AI film institute.

Some creators are aware that deepfakes may not be the best way for their clients to deal with grief. Divyendra Singh Jadoun, who taught himself to use Photoshop, video editing, and generative AI tools during the Covid-19 lockdowns, has a thriving business creating AI-generated content. He began by posting “what if” parody videos on Instagram. Then came a message from a woman asking if he could recreate her deceased father for a family gathering.

Jadoun, who is based in the north Indian town of Pushkar, created a short video from the photos, videos, and audio clips the client had sent him, he told Rest of World. He now runs The Indian Deepfaker, an outfit that creates “hyper realistic deepfakes” including those of politicians, and what he calls grief tech — AI-generated avatars of dead people that can speak, text, and even video-chat in real time.

“Being able to talk to someone who is no longer alive, even in a limited way, is deeply meaningful,” Jadoun said. That AI avatars of the dead might “take people into a deep depression” is something he is aware of. “I make sure I convey to them that they should not get too attached, as they are not real,” he said.

India is among the world’s largest markets for generative AI platforms, and for AI-generated videos and voice clones. Several celebrities have sued YouTube and Google for hosting deepfake videos, and the government’s new rules to curb the flood of deepfakes go into effect on February 20. They require all AI-generated content to be clearly labeled, and place new obligations on platforms to remove such content within hours when directed to.

But in small towns, deepfakes are helping meet vital cultural needs, Bhaskar Malu, a Delhi-based behavioral scientist, told Rest of World.

“In cultures like ours, where social rituals demand physical, or at least symbolic presence, especially during weddings and funerals, AI-generated stand-ins are a response to real emotional pressures,” he said. Deepfakes can help with dealing with the loss in the short term, but they also create “an artificial reality,” where the dead are “alive and dead at the same time in your mind,” he said. The long-term effects are unclear. “Technology should be a partner, not a substitute for human connection and emotional reckoning,” Malu said.

A growing number of people say they are in a relationship with their AI chatbots, with some getting engaged to or marrying the AI characters they create on ChatGPT, Character.ai, Replika, and other platforms. As the debate around deepfakes — like the sexualized images produced by X’s Grok chatbot — continues, in wedding halls and living rooms in India, AI-generated avatars are quietly lodging themselves into ceremonies and rituals that have long insisted on presence.

For the creators, the AI technology is a way to earn an income by offering a service, much like wedding photographers and videographers do. For clients, the technology makes it easier to manage social expectations, avoid uncomfortable questions about absent relatives, and maintain continuity during rituals where presence carries deep symbolic weight.

Besides the money to be made, Jadoun values creating something that is cherished. The woman who asked for a video of her dead father thanked him when he sent her the deepfake: It was the “best thing” anyone had given her, she said.