diff --git a/README.md b/README.md old mode 100644 new mode 100755 index ff32b14..a7c68b4 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Tacotron 2 (without wavenet) -Tacotron 2 PyTorch implementation of [Natural TTS Synthesis By Conditioning +PyTorch implementation of [Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions](https://arxiv.org/pdf/1712.05884.pdf). This implementation includes **distributed** and **fp16** support @@ -11,9 +11,7 @@ Distributed and FP16 support relies on work by Christian Sarofeen and NVIDIA's ![Alignment, Predicted Mel Spectrogram, Target Mel Spectrogram](tensorboard.png) -[Download demo audio](https://github.com/NVIDIA/tacotron2/blob/master/demo.wav) trained on LJS and using Ryuchi Yamamoto's [pre-trained Mixture of Logistics -wavenet](https://github.com/r9y9/wavenet_vocoder/) -"Scientists at the CERN laboratory say they have discovered a new particle." +Visit our [website] for audio samples. ## Pre-requisites 1. NVIDIA GPU + CUDA cuDNN @@ -24,11 +22,9 @@ wavenet](https://github.com/r9y9/wavenet_vocoder/) 3. CD into this repo: `cd tacotron2` 4. Update .wav paths: `sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt` - Alternatively, set `load_mel_from_disk=True` in `hparams.py` and update mel-spectrogram paths -5. Install [pytorch 0.4](https://github.com/pytorch/pytorch) +5. Install [PyTorch 1.0] 6. Install python requirements or build docker image - Install python requirements: `pip install -r requirements.txt` - - **OR** - - Build docker image: `docker build --tag tacotron2 .` ## Training 1. `python train.py --output_directory=outdir --log_directory=logdir` @@ -37,17 +33,22 @@ wavenet](https://github.com/r9y9/wavenet_vocoder/) ## Multi-GPU (distributed) and FP16 Training 1. `python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True` -## Inference -When performing Mel-Spectrogram to Audio synthesis with a WaveNet model, make sure Tacotron 2 and WaveNet were trained on the same mel-spectrogram representation. Follow these steps to use a a simple inference pipeline using griffin-lim: - -1. `jupyter notebook --ip=127.0.0.1 --port=31337` -2. load inference.ipynb +## Inference demo +1. Download our published [Tacotron 2] model +2. Download our published [WaveGlow] model +3. `jupyter notebook --ip=127.0.0.1 --port=31337` +4. Load inference.ipynb +N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2 +and the Mel decoder were trained on the same mel-spectrogram representation. ## Related repos -[nv-wavenet](https://github.com/NVIDIA/nv-wavenet/): Faster than real-time -wavenet inference +[WaveGlow](https://github.com/NVIDIA/WaveGlow) Faster than real time Flow-based +Generative Network for Speech Synthesis + +[nv-wavenet](https://github.com/NVIDIA/nv-wavenet/) Faster than real time +WaveNet. ## Acknowledgements This implementation uses code from the following repos: [Keith @@ -61,3 +62,7 @@ We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang. +[WaveGlow]: https://drive.google.com/file/d/1cjKPHbtAMh_4HTHmuIGNkbOkPBD9qwhj/view?usp=sharing +[Tacotron 2]: https://drive.google.com/file/d/1c5ZTuT7J08wLUoVZ2KkUs_VdZuJ86ZqA/view?usp=sharing +[pytorch 1.0]: https://github.com/pytorch/pytorch#installation +[website]: https://nv-adlr.github.io/WaveGlow