Block announces Dessa, a speech-to-speech AI model trained on human conversation

What happened

Block, the company led by Jack Dorsey, has announced an AI model called Dessa. According to The Verge, the company says Dessa was trained from scratch using only human conversation. The model works by voice rather than text. On its website, the Dessa team describes it as a true speech-to-speech model, and says everything it knows came from human speech.

The team says Dessa can run locally on a MacBook Pro M4 Max or better. The Verge reports that Block shared sample conversations to show the model off. Some of them are described as a little awkward or uncanny. In one shared by Dessa lead Ragavan Thurairatnam, the model at first mistakes the speaker for someone named “Ronaldo”, though the rest of the exchange is described as mostly natural.

The Dessa website includes other examples:

• A conversation about Christopher Nolan’s films, in which Dessa goes on an unexpected tangent about Bryan Singer’s Superman films.
• A person telling Dessa about a surprise trip to Paris. The website describes Dessa’s reply as sounding like teary gratitude.
• A conversation between OpenAI’s GPT-Live-1 voice model and Dessa. The Verge says Dessa sounds more natural in this one.

The Verge also shared a video of Dessa playing a guessing game over voice.

The team gives three reasons for building Dessa: curiosity (described as fun), better understanding between humans and AI, and safety. On safety, the team says voice AI is still nearly deaf to the emotion in people’s voices, and that Dessa is a step toward fixing this. The team says that speech is the most natural interface for any intelligence, and that current voice AI turns speech into text, which loses detail and cannot keep up with human timing.

Who is affected

The Dessa team says it will give “a small group of AI safety researchers” early access to the model. The sources do not say how that group will be chosen, or when access will begin.