Offline Speech-to-Text: Why I Ultimately Chose Handy

Speaking to your AI coding assistant is becoming a natural way to interact. But there’s an often-overlooked question in the speech-to-text pipeline: who gets your voice data?

This article researches cross-platform, fully offline speech-to-text solutions, explains why I ultimately chose Handy, and documents the complete selection process with real-world measurements.

Why Speech-to-Text Should Run Locally

Local (offline) speech-to-text provides two independent values:

1. Privacy — Your voice is a biometric characteristic, and its exposure carries real risks of cloning and abuse. When audio never leaves the device, the risk is closed at the source.
2. Performance — No network round-trip of “upload audio → cloud transcription → return text”. For push-to-talk interaction, the latency difference is perceptible.

The Risk of Voice Exposure

Voice is a biometric characteristic (official definitions):

• NIST SP 800-63B lists voice prints among biometric characteristics
• EDPB Guidelines 02/2021 (para. 31): “voice data is inherently biometric personal data”

Exposure is a scaled, real-world threat:

Cases in China:

• The Paper 2025-02: AI voice impersonation of a grandson defrauded an elderly person of ¥20,000
• Jinshipin 2026-06: AI face-swap + voice impersonation of a friend, ¥4.3M defrauded (5–10 seconds of audio is enough to extract a voiceprint)
• China’s Personal Information Protection Law (PIPL) classifies voiceprints as sensitive personal information, consistent with GDPR’s treatment of biometrics

Implication: An online speech-to-text service means voluntarily handing your audio to an uncontrollable third party, which decides retention, training, and security. Risk paths:

• Cloning and impersonation (deceiving people, not technical systems)
• Entering training datasets (nearly impossible to remove)

Tool Selection: Why Handy

Requirements: Cross-platform / fully offline / push-to-talk / terminal paste compatibility / Chinese support

Handy vs. Shandianshuo (both local solutions, key difference in terminal adaptation):

Two key points:

1. Terminal paste: Windows Terminal’s paste shortcut is Ctrl+Shift+V, not Ctrl+V. Shandianshuo only simulates Ctrl+V, so transcribed text cannot be pasted into terminals that require Ctrl+Shift+V. Handy lets you configure the paste method — the only one that works in terminals.
2. Multiple models: Handy supports switching between several local models. In testing, SenseVoice gave the best Chinese quality, better than Whisper.

Other candidates eliminated: Zhipu Input Method has no Linux version; Voisty is paid.

Measurements (2026-05)

Scenario: push-to-talk real-time voice input (press to speak, release to get text immediately), where real-time performance is a hard requirement. Hardware baseline: Intel Core Ultra 7 255H (Arc 140T iGPU, no discrete GPU). Model conclusions must be tied to hardware and use case — without these, “which model is better” is meaningless.

Model conclusion (Intel Core Ultra 7 255H + Arc 140T iGPU, push-to-talk scenario):

• SenseVoice wins: ~240 MB, converts 1 minute of speech in under 3 seconds; simple commands appear instantly
• vs. Whisper large v3 (over 1 GB):
  • Better Chinese quality
  • Comparable for mixed Chinese-English and pure English
  • Much faster

Boundaries

• Voice input suits natural-language intent; passwords and full code are unsuitable for voice input (precision issues)
• Local solutions prevent “audio → third party”; they do not prevent recording by a compromised machine or physical eavesdropping
• Transcribed text is still sent to the AI model — local processing protects the audio, not the content itself

References

• Handy website
• Handy GitHub
• Handy issue #1691 (global hotkey unusable in background on Ubuntu 24.04, with workaround)
• NIST SP 800-63B
• EDPB Guidelines 02/2021 on Virtual Voice Assistants (PDF, para. 31)