The Treacherous Turn

Year: 2022 Role: development

The Treacherous Turn is a research-supported tabletop RPG in which the players collectively act as a misaligned AI in the modern world.

glitching logo animation

Follow us at @treacherousturn for news and updates!

The project was developed during AI Safety Camp 2022, under the mentorship of Daniel Kokotajlo, and is now supported by Future Fund.

Background

A superintelligent AI may be the greatest boon, or the most perilous danger if it goes out of control , to humanity and beyond. And part of what makes it dangerous is that this danger is not obvious.

Most sci-fi works present the problem with superintelligent AI as one where they gained conscious and self-awareness, and decided to stop doing their jobs and instead rebel against humanity (Terminator, The Matrix).

However, it’s far more likely that a superintelligent AI would already pose existential risk to us, without evolving consciousness or any grudge against humanity. It can be dangerous simply because it’s incredibly good at doing what we asked it to do, but bad at accounting for the other things we value in this world. Instead of those sci-fi flicks, think sorcerer’s apprentice and monkey’s paw.

This game invites the players to think deeply and differently about how a superintelligent AI can be catastrophically dangerous, by role-playing as one.

If this sounds too serious and gloomy, rest assured that it turned out to be quite fun to play as an AI looking to sabotage and dominate humanity.

Companion App

Companion App Compute Tracker

I’m responsible for developing a companion app that players use to run the game online. The app keeps track of game events, allows the players to brainstorm and strategize the AI’s plan, and simulates how the AI uses its compute resources to perform actions such as upgrades, insights, and forecasting.

It is a real-time web app built with Firebase and VueJS, featuring:

  • an event log for the players to brainstorm and decide on actions to take.
  • a compute tracker for managing computational resources, and where to spend them.
  • a specialization manager for each players to administer capability upgrades.
  • an info section to keep track of details about the game.
  • a GM panel, hidden from other players, for the game master to track NPC information and upcoming world events.

Our game is designed to show players what an out-of-control AI would look like in the near future. Therefore, it is important to sustain a sense of realism. By using the web app to manage resource usages and automate computations, the players can enjoy the simulation without getting bogged down by the calculations needed to run that simulation.

Try the web app here!

Marketing Website

Marketing website placeholder

I’m also building a marketing website that will introduce the game, and point the players toward the resources they need. In the meantime, I’ve set up a placeholder built using WebGL shaders.

See also

The idea that advanced AI is dangerous because it can easily have great capability but the wrong goals (Orthogonality Thesis), and the idea that some predictable sub-goals would be helpful for the AI to pursue, regardless of what its final goal is (Instrumental Convergent Thesis), was explored in Chapter 7 of Superintelligence by Nick Bostrom.

The name of our game, The Treacherous Turn, comes from Chapter 8 of the same book.

Check out Universal Paperclip by Frank Lantz of NYU Game Center, for another game that explores the danger of superintelligent AI.

Theory Wheel

The Theory Wheel

Notes mentioning this note