Block Dessa speech model processes raw audio to detect emotions

Block, the financial services company behind Square and Cash App, has unveiled Dessa, a new approach to voice AI that skips text altogether. The company says Dessa is the first AI model trained entirely on human speech, without any text data or pretrained models. Instead of the standard pipeline that converts audio to text before processing, Dessa works directly with raw audio signals.

This architectural choice matters because it lets the model preserve information that text-based systems lose. According to the company, Dessa detects emotions, sarcasm, laughter, and pauses in real time, handling elements of human communication that convert-then-process pipelines typically miss or flatten.

What changed

Block has released details about Dessa, a speech-to-speech AI model built from the ground up on human conversation audio. The model supports full-duplex communication, meaning it can listen and speak simultaneously. Block says it built the system out of curiosity, a desire for better human-AI communication, and safety concerns, since current voice AI systems ignore emotional cues that matter in real conversations.

The system runs locally on a MacBook Pro M4 Max and was trained using a single NVIDIA GB200 NVL72 GPU. Dessa is not currently available to the general public. Instead, Block is taking a careful rollout approach, first seeking AI security researchers to test the model and surface any unexpected risks before opening access wider.

Why it matters

For professionals building voice AI applications, Dessa represents a different architectural choice. Rather than asking how to make text-to-speech models sound more human, Block is asking what happens if you skip text entirely and train on speech directly.

The distinction has practical implications. Voice interfaces for customer service, health care, counseling, or education often need to handle nuance. A voice AI that detects when a user is frustrated, joking, or hesitant could adapt its response differently than one that processes only word-level meaning. Systems that ignore emotional tone risk misreading intent, offering canned solutions when users need genuine engagement.

Block’s choice to prioritize safety and security researcher feedback before public release also signals something about how the company views voice AI risk. The focus on unexpected risks suggests awareness that audio-in, audio-out systems may behave differently from text-based pipelines in ways that are not obvious from the outset.

However, the source notes an important caveat: “sounding emotional isn’t the same as being emotional or intelligent.” For advanced speech AI, capabilities like reasoning while responding matter more than how natural the voice sounds, because they actually affect the quality of the answer. Dessa’s emotional sensitivity is a feature, but it is not a substitute for reasoning ability.

What to test

Before integrating Dessa into production systems, professionals should verify several vendor claims and practical constraints:

• Local performance claims: Does Dessa actually run reliably on MacBook Pro M4 Max hardware at acceptable latency? Test under realistic conversational load and compare against cloud-based alternatives.
• Emotional detection accuracy: How often does the model correctly identify emotions, sarcasm, or pauses? Does it misread neutral or ambiguous speech? Test on diverse speaker types and accents.
• Full-duplex performance: Does simultaneous listening and speaking create latency or accuracy degradation compared to turn-based systems? Check response times during overlapping speech.
• Safety in practice: What kinds of adversarial inputs or unusual speech patterns could cause failures? Security researchers are testing this, but document any findings relevant to your use case.
• Integration maturity: What tooling, APIs, and documentation are available? Dessa is in early testing; expect friction compared to established voice platforms like ElevenLabs vs Murf AI.
• Cost and scaling: How does local deployment scale if you need multiple concurrent conversations? What are compute costs compared to cloud services?

Common questions

What is the Dessa speech model trained on?

According to Block, Dessa was trained entirely on human speech data with no text data or pretrained models. The company built the system from scratch specifically to preserve speech information that text-based pipelines lose.

How does Dessa differ from other emotional voice AI models?

Dessa processes raw audio directly instead of converting to text first. Other systems like Hume AI’s EVI, Kyutai’s Moshi, and OpenAI’s Advanced Voice Mode focus on sounding emotionally rich through synthesis improvements. Dessa’s audio-in, audio-out approach is positioned as a different technical path to handling emotion and timing.

Can Dessa listen and speak at the same time?

Yes, the model supports full-duplex communication, allowing it to listen while it speaks simultaneously. This is a capability Block emphasizes as important for natural conversation flow.

The conclusion

Dessa represents a genuine architectural alternative in voice AI, not just incremental refinement of existing approaches. By training on speech alone and preserving audio signals end-to-end, Block is testing whether there are better ways to handle the parts of human communication that text reduces to silence.

The company’s decision to prioritize security researcher feedback before public release is sensible caution. Audio-based systems behave differently from text pipelines, and unexpected failure modes are plausible. This also means the model’s real-world performance and reliability remain unproven at scale.

For professionals evaluating voice AI, the relevant question is not whether Dessa “sounds better.” It is whether directly processing audio gives meaningful advantages for your specific use case: customer service, accessibility, counseling, or something else. That requires testing once the model becomes available.

Watch for how Block evolves Dessa’s availability, pricing, and integration paths. The early focus on security research also suggests the company may publish findings about risks in audio-based voice AI systems, which would be valuable for the broader industry.

Related reading

• Voxtral 3 voice agent SDK: 50+ languages and emotional speech
• Grok Voice 3.0 vs Voxtral 3: Voice Agent Comparison