mirror of
https://github.com/wassname/TTS.git
synced 2026-10-29 12:51:00 +08:00
a3d5801c44
* initial commit * Tortoise inference * revert path change * style fix * remove accidental remove * style fixes * style fixes * removed unwanted assests and deps * remove changes * remove cvvp * style fix black * added tortoise config and updated config and args, refactoring the code * added tortoise to api * Pull mel_norm from url * Use TTS cleaners * Let download model files * add ability to pass tortoise presets through coqui api * fix tests * fix style and tests * fix tts commandline for tortoise * Add config.json to tortoise * Use kwargs * Use regular model api for loading tortoise * Add load from dir to synthesizer * Fix Tortoise floats * Use model_dir when there are multiple urls * Use `synthesize` when exists * lint fixes and resolve preset bug * resolve a download bug and update model link * fix json * do tortoise inference from voice dir * fix * fix test * fix speaker id and remove assests * update inference_tests.yml * replace inference_test.yml * fix extra dir as None * fix tests * remove space * Reformat docstring * Add docs * Update docs * lint fixes --------- Co-authored-by: Eren Gölge <egolge@coqui.ai> Co-authored-by: Eren Gölge <erogol@hotmail.com>
95 lines
3.1 KiB
Markdown
95 lines
3.1 KiB
Markdown
# Tortoise 🐢
|
|
Tortoise is a very expressive TTS system with impressive voice cloning capabilities. It is based on an GPT like autogressive acoustic model that converts input
|
|
text to discritized acouistic tokens, a diffusion model that converts these tokens to melspeectrogram frames and a Univnet vocoder to convert the spectrograms to
|
|
the final audio signal. The important downside is that Tortoise is very slow compared to the parallel TTS models like VITS.
|
|
|
|
Big thanks to 👑[@manmay-nakhashi](https://github.com/manmay-nakhashi) who helped us implement Tortoise in 🐸TTS.
|
|
|
|
Example use:
|
|
|
|
```python
|
|
from TTS.tts.configs.tortoise_config import TortoiseConfig
|
|
from TTS.tts.models.tortoise import Tortoise
|
|
|
|
config = TortoiseConfig()
|
|
model = Tortoise.inif_from_config(config)
|
|
model.load_checkpoint(config, checkpoint_dir="paths/to/models_dir/", eval=True)
|
|
|
|
# with random speaker
|
|
output_dict = model.synthesize(text, config, speaker_id="random", extra_voice_dirs=None, **kwargs)
|
|
|
|
# cloning a speaker
|
|
output_dict = model.synthesize(text, config, speaker_id="speaker_n", extra_voice_dirs="path/to/speaker_n/", **kwargs)
|
|
```
|
|
|
|
Using 🐸TTS API:
|
|
|
|
```python
|
|
from TTS.api import TTS
|
|
tts = TTS("tts_models/en/multi-dataset/tortoise-v2")
|
|
|
|
# cloning `lj` voice from `TTS/tts/utils/assets/tortoise/voices/lj`
|
|
# with custom inference settings overriding defaults.
|
|
tts.tts_to_file(text="Hello, my name is Manmay , how are you?",
|
|
file_path="output.wav",
|
|
voice_dir="TTS/tts/utils/assets/tortoise/voices/",
|
|
speaker="lj",
|
|
num_autoregressive_samples=1,
|
|
diffusion_iterations=10)
|
|
|
|
# Using presets with the same voice
|
|
tts.tts_to_file(text="Hello, my name is Manmay , how are you?",
|
|
file_path="output.wav",
|
|
voice_dir="TTS/tts/utils/assets/tortoise/voices/",
|
|
speaker="lj",
|
|
preset="ultra_fast")
|
|
|
|
# Random voice generation
|
|
tts.tts_to_file(text="Hello, my name is Manmay , how are you?",
|
|
file_path="output.wav")
|
|
```
|
|
|
|
Using 🐸TTS Command line:
|
|
|
|
```console
|
|
# cloning the `lj` voice
|
|
tts --model_name tts_models/en/multi-dataset/tortoise-v2 \
|
|
--text "This is an example." \
|
|
--out_path "/data/speech_synth/coqui-tts/TTS/tests/outputs/output.wav" \
|
|
--voice_dir TTS/tts/utils/assets/tortoise/voices/ \
|
|
--speaker_idx "lj" \
|
|
--progress_bar True
|
|
|
|
# Random voice generation
|
|
tts --model_name tts_models/en/multi-dataset/tortoise-v2 \
|
|
--text "This is an example." \
|
|
--out_path "/data/speech_synth/coqui-tts/TTS/tests/outputs/output.wav" \
|
|
--progress_bar True
|
|
```
|
|
|
|
|
|
## Important resources & papers
|
|
- Original Repo: https://github.com/neonbjb/tortoise-tts
|
|
- Faster implementation: https://github.com/152334H/tortoise-tts-fast
|
|
- Univnet: https://arxiv.org/abs/2106.07889
|
|
- Latent Diffusion:https://arxiv.org/abs/2112.10752
|
|
- DALL-E: https://arxiv.org/abs/2102.12092
|
|
|
|
## TortoiseConfig
|
|
```{eval-rst}
|
|
.. autoclass:: TTS.tts.configs.tortoise_config.TortoiseConfig
|
|
:members:
|
|
```
|
|
|
|
## TortoiseArgs
|
|
```{eval-rst}
|
|
.. autoclass:: TTS.tts.models.tortoise.TortoiseArgs
|
|
:members:
|
|
```
|
|
|
|
## Tortoise Model
|
|
```{eval-rst}
|
|
.. autoclass:: TTS.tts.models.tortoise.Tortoise
|
|
:members:
|
|
```
|