Microsoft’s Vibe Voice, a free open-source AI voice generation tool, was pulled by Microsoft just three weeks after its release due to deepfake concerns, but community members preserved copies of the original GitHub repository. This video demonstrates how to restore its text-to-speech functionality locally, offering realistic synthetic voices without subscription fees, comparable to paid alternatives like ElevenLabs.
🎯 Key Features
- Generates realistic, expressive speech via a hybrid architecture combining autoregressive language modeling with diffusion-based audio generation
- Supports up to four simultaneous speakers, enabling full AI podcast production similar to NotebookLM
- Handles multiple languages—including English and Chinese—within a single audio output
- Enables voice cloning from just a 30-second recording of a user’s voice
⚙️ Installation Steps
- Microsoft’s official version stripped out text-to-speech, so the creator uses a community-saved repository cloned via Claude Code desktop
- Command-line instructions clone the project and install it locally, with copy-paste prompts provided in the description
- Users download either the smaller 1.5B or larger 7B parameter Vibe Voice model, with a roughly 30-minute download time
- Claude Code then spins up a local server, where users select speakers, preloaded voices, and generate sample audio
🧪 Demonstration & Results
- Audio samples demonstrate natural intonation, rhythm, pauses, and emotional expression—visibly indistinguishable from paid ElevenLabs output
- A multi-speaker basketball debate podcast showcases distinct personality and pacing across voices
- A multilingual test seamlessly transitions between English and Chinese, confirming cross-language fluency
- The cloned custom voice closely resembled the creator’s real voice, which she planned to test with her mother
đź’ˇ Use Cases
- Convert blog posts into audio versions for accessibility
- Generate audiobooks and multi-speaker podcasts without recording equipment
- Translate content across languages while preserving speaker identity
- Produce custom audio for business events, like purchase notifications or client greetings
- Replace ElevenLabs subscriptions with free, fully local generation
đź“Ś Additional Tips
- For voice cloning, record at least 30 seconds of quiet speech, then use Claude Code to convert M4A files to WAV format and add the new voice as a speaker option
- Qwen, a Chinese open-source audio model, is recommended as a high-quality alternative
- Two model sizes allow users to balance quality against storage and performance
Final Takeaway: Vibe Voice represents a paradigm shift in accessible synthetic audio, showing that open-source AI can match proprietary paid services. Its preservation through community copies highlights the tension between deepfake safety concerns and democratized AI tooling. For creators, this means high-quality voice generation, cloning, and multilingual podcasting are now freely available locally—provided users navigate the ethical considerations of voice synthesis responsibly.