Yossi Adi, Apoorv Vyas, Bowen Shi, Wei-Ning Hsu, Matthew Le, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Jay Mahadeokar
We lifted 10 functions out of this paper's own repositories and ran 10 of them in a sandbox. "Ran" means the function executed on a synthesized input and returned a value. It is not a reproduction of the paper's results.
| Repository | Role | Ran |
|---|---|---|
| lucidrains/voicebox-pytorch | pwc_unofficial | 10 of 10 |
| Function | Status | Where it lives |
|---|---|---|
| cast_tuple | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/data.py code served (permissive licence) · get_code("f70dcb896366697c") |
| cast_tuple | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/trainer.py code served (permissive licence) · get_code("5cfcc1d3230706ff") |
| collate_one_or_multiple_tensors | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/data.py code served (permissive licence) · get_code("0e0470ad1da271f6") |
| default | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/attend.py code served (permissive licence) · get_code("60fff7c3c400d7ff") |
| exists | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/attend.py code served (permissive licence) · get_code("aa5486a3650902d8") |
| get_optimizer | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/optimizer.py code served (permissive licence) · get_code("9354ff16b3840873") |
| identity | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/voicebox_pytorch.py code served (permissive licence) · get_code("f3232418205f7cbd") |
| once | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/attend.py code served (permissive licence) · get_code("52e10a7d6b0fa036") |
| separate_weight_decayable_params | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/optimizer.py code served (permissive licence) · get_code("fdc11aefc97f3abb") |
| yes_or_no | Ran | lucidrains/voicebox-pytorch/voicebox_pytorch/trainer.py code served (permissive licence) · get_code("d95562c398a6b25e") |
Some links come from the archived Papers with Code dataset (CC BY-SA 4.0): attribution and licence.
Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster. Audio samples can be found in https://voicebox.metademolab.com.
The same record, over MCP at https://syntology.ai/mcp:
get_harvested_code_for_paper("2306.15687")
get_code_for_paper("2306.15687")
have("2306.15687")
Connect an agent — have() is free.