Latitude, a Utah-based company, released AI Dungeon, a groundbreaking online game that demonstrated a new type of human-machine collaboration, in December 2019.
The company created a Dungeons & Dragons-inspired choose-your-own adventure game using text-generation technology from OpenAI, an artificial intelligence company.
When a player typed in an action or a line of dialogue for their character, algorithms created the next step of their customized, unpredictable adventure.
OpenAI provided Latitude with early access to a more efficient, commercial version of its technology last summer. OpenAI promoted AI Dungeon as an example of the commercial and artistic potential of writing algorithms in its marketing materials.
Then, last month, OpenAI discovered that AI Dungeon revealed a dark side to human-AI collaboration, the company stated. According to a new monitoring system, some players were typing words into the game that caused it to create reports about sexual experiences with children.
Latitude was asked by OpenAI to take immediate action. “Content moderation decisions are difficult in some cases, but not this one,” OpenAI CEO Sam Altman said in a statement. “This is not the future for AI that any of us want.”
Cancellations and memes
Last week, Latitude launched a new moderation scheme, which sparked outrage among its users. Some people argued that it was too sensitive, and that they couldn’t say things like “8-year-old laptop” without getting a warning message.
Others complained that the company’s plans to manually review flagged content would inadvertently snoop on private, imaginary creations that were sexually explicit but only included adults, which is a common use case for AI Dungeon.
In a nutshell, Latitude’s attempt to combine people and algorithms to police content created by people and algorithms failed miserably. On Twitter and AI Dungeon’s official Reddit and Discord communities, angry memes and claims of cancelled subscriptions flew thick and quick.
“The community feels betrayed that Latitude would scan and manually access and read private fictional literary content,” claims Mimi, an AI Dungeon player who claims to have written over 1 million words with the AI’s assistance, including poetry, Twilight Zone parodies, and erotic adventures.
Mimi and other disappointed users claim the company’s willingness to police publicly accessible content is understandable, but that it has gone too far and destroyed a powerful creative space. Mimi says, “It allowed me to explore aspects of my psyche that I never realized existed.”
Latitude’s filtering mechanism and policies for appropriate content, according to a spokesperson, are both being refined. Staff had previously banned players who they had discovered were using AI Dungeon to create sexual content involving children.
However, following OpenAI’s recent advisory, the company is working on “necessary changes,” according to a spokesperson. Last week, Latitude promised that AI Dungeon would “continue to support other NSFW content, including consensual adult content, violence, and profanity” in a blog post.
It will be difficult to prevent the AI system from producing some forms of sexual or adult content while enabling others. Since it is designed using machine learning algorithms that have digested the statistical patterns of language use in billions of words scraped from the internet, including sections not suitable for minors, technology like OpenAI’s can produce text in a variety of styles.
Although the program is capable of surprising mimicry, it does not recognize social, legal, or genre categories in the same way that humans do. When you combine the fiendish ingenuity of Homo internetus, the result may be weird, stunning, or toxic.
Late in 2019, OpenAI made its text generation technology open source, but a substantially improved version, named GPT-3, became a commercial service last year. Customers like Latitude pay to enter text strings and receive the system’s best guess at what should come next.
The service gained traction in the tech industry after early adopters shared impressively fluent jokes, sonnets, and code created by the technology.
The service would empower businesses and startups, according to OpenAI, which has given Microsoft, a major supporter of the company, an exclusive license to the underlying algorithms.
WIRED, as well as some coders and AI researchers who tested the device, discovered that it could also produce unsavory text, such as anti-Semitic remarks and extremist propaganda.
OpenAI said it would thoroughly vet customers to root out bad actors, and that most customers—but not Latitude—would be allowed to use filters developed by the AI company to block profanity, hate speech, and sexual content.
You wanted to… mount that dragon?
AI Dungeon offered relatively unrestricted access to OpenAI’s text-generation technologies away from the spotlight. The game attracted 100,000 players in December 2019, the month it was released using an earlier open-source version of OpenAI’s technology.
Some quickly noticed and appreciated its proclivity for sexual content. Others complained that the AI would inadvertently bring up sexual themes, such as when they tried to fly by dragon and their journey took an unexpected turn.
Within days of the game’s release, Latitude cofounder Nick Walton acknowledged the issue on the game’s official Reddit group. He said that many players had given him examples that made them “feeling deeply uncomfortable,” and that the company was working on filtering technology.
Players found — and recorded — that the game would sometimes write children into sexual situations as soon as the game’s early months.
The official Reddit and Discord communities for AI Dungeon have added dedicated channels to address the game’s adult content. Latitude added a “safe mode” feature that filtered out AI suggestions containing specific terms.
It wasn’t flawless, though, like all automatic filters. Some players have found that the ostensibly safe environment enhanced the text-erotic generator’s writing by using more analogies and euphemisms. To generate sales, the company also added a premium subscription tier.
The writing became even more impressive when AI Dungeon added OpenAI’s more efficient, commercial writing algorithms in July 2020. One seasoned player says, “The sheer jump in creativity and storytelling ability was heavenly.”
This individual claims that the framework became notably more innovative in its ability to explore sexually provocative themes. Last year, players found Latitude playing with a filter that automatically replaced the word “rape” with “respect,” but the feature was eventually removed.
The seasoned player was one of the AI Dungeon devotees who used the game as an AI-assisted writing tool to explore adult themes, even forming a dedicated writing community.
Unwanted algorithmic feedback could be omitted from a story to guide it in a different direction, and the findings weren’t made public unless anyone wanted them to be.
Latitude refused to say how many of the adventures had sexual content. According to OpenAI’s website, AI Dungeon attracts over 20,000 players every day.
An AI Dungeon player who posted last week about a security vulnerability that exposed every story produced in the game to the public claims he downloaded hundreds of thousands of adventures created over four days in April.
He looked at a sample of 188,000 of them and discovered that 31% of them featured sexually suggestive phrases. This review, as well as the now-fixed security bug, contributed to some players’ dissatisfaction with Latitude’s new approach to content moderation.
Latitude now has to regain the confidence of its users while also meeting OpenAI’s demands for more control over its text generator. According to an OpenAI spokesperson, the startup must now use OpenAI’s filtering technology.
The technology can be used in very limited situations, such as in Google search, where it aids in the parsing of long queries. OpenAI aided AI Dungeon in the launch of an ambitious yet risky application that enabled users to direct the technology to do pretty much whatever it wanted.