Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It is built on Paddle Fluid dynamic graph, with the support of many influential TTS models proposed by [Baidu Research](http://research.baidu.com) and other academic institutions.
Parakeet aims to provide a flexible, efficient and state-of-the-art text-to-speech toolkit for the open-source community. It is built on PaddlePaddle Fluid dynamic graph and includes many influential TTS models proposed by [Baidu Research](http://research.baidu.com) and other research groups.
<divalign="center">
<divalign="center">
<imgsrc="images/logo.png"width=450/><br>
<imgsrc="images/logo.png"width=450/><br>
</div>
</div>
In particular, it features the latest [WaveFlow] (https://arxiv.org/abs/1912.01219) model proposed by Baidu Research.
- WaveFlow can synthesize 22.05 kHz high-fidelity speech around 40x faster than real-time on a Nvidia V100 GPU without engineered inference kernels, which is faster than WaveGlow and serveral orders of magnitude faster than WaveNet.
- WaveFlow is a small-footprint flow-based model for raw audio. It has only 5.9M parameters, which is 15x smalller than WaveGlow (87.9M) and comparable to WaveNet (4.6M).
- WaveFlow is directly trained with maximum likelihood without probability density distillation and auxiliary losses as used in Parallel WaveNet and ClariNet, which simplifies the training pipeline and reduces the cost of development.
### Setup
### Setup
Make sure the library `libsndfile1` installed, e.g., on Ubuntu
Make sure the library `libsndfile1` is installed, e.g., on Ubuntu.
Paddle implementation of clarinet in dynamic graph, a convolutional network based vocoder. The implementation is based on the paper [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](arxiv.org/abs/1807.07281).
PaddlePaddle dynamic graph implementation of ClariNet, a convolutional network based vocoder. The implementation is based on the paper [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](arxiv.org/abs/1807.07281).
## Dataset
## Dataset
...
@@ -30,7 +30,7 @@ Train the model using train.py, follow the usage displayed by `python train.py -
...
@@ -30,7 +30,7 @@ Train the model using train.py, follow the usage displayed by `python train.py -
train a clarinet model with LJspeech and a trained wavenet model.
train a ClariNet model with LJspeech and a trained WaveNet model.
optional arguments:
optional arguments:
-h, --help show this help message and exit
-h, --help show this help message and exit
...
@@ -54,12 +54,12 @@ optional arguments:
...
@@ -54,12 +54,12 @@ optional arguments:
```
```
5.`--device` is the device (gpu id) to use for training. `-1` means CPU.
5.`--device` is the device (gpu id) to use for training. `-1` means CPU.
6.`--wavenet` is the path of the wavenet checkpoint to load. if you do not specify `--resume`, then this must be provided.
6.`--wavenet` is the path of the wavenet checkpoint to load. If you do not specify `--resume`, then this must be provided.
Before you start training a clarinet model, you should have trained a wavenet model with single gaussian as output distribution. Make sure the config for teacher matches that for the trained model.
Before you start training a ClariNet model, you should have trained a WaveNet model with single Gaussian output distribution. Make sure the config of the teacher model matches that of the trained model.
Paddle implementation of deepvoice 3 in dynamic graph, a convolutional network based text-to-speech synthesis model. The implementation is based on [Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning](https://arxiv.org/abs/1710.07654).
PaddlePaddle dynamic graph implementation of Deep Voice 3, a convolutional network based text-to-speech generative model. The implementation is based on [Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning](https://arxiv.org/abs/1710.07654).
We implement Deepvoice 3 in paddle fluid with dynamic graph, which is convenient for flexible network architectures.
We implement Deep Voice 3 using Paddle Fluid with dynamic graph, which is convenient for building flexible network architectures.
## Dataset
## Dataset
...
@@ -15,9 +15,9 @@ tar xjvf LJSpeech-1.1.tar.bz2
...
@@ -15,9 +15,9 @@ tar xjvf LJSpeech-1.1.tar.bz2
## Model Architecture
## Model Architecture
The model consists of an encoder, a decoder and a converter (and a speaker embedding for multispeaker models). The encoder, together with the decoder forms the seq2seq part of the model, and the converter forms the postnet part.
The model consists of an encoder, a decoder and a converter (and a speaker embedding for multispeaker models). The encoder and the decoder together form the seq2seq part of the model, and the converter forms the postnet part.
## Project Structure
## Project Structure
...
@@ -37,7 +37,7 @@ Train the model using train.py, follow the usage displayed by `python train.py -
...
@@ -37,7 +37,7 @@ Train the model using train.py, follow the usage displayed by `python train.py -
1.`--config` is the configuration file to use. The provided `ljspeech.yaml` can be used directly. And you can change some values in the configuration file and train the model with a different config.
1.`--config` is the configuration file to use. The provided `ljspeech.yaml` can be used directly. And you can change some values in the configuration file and train the model with a different config.
2.`--data` is the path of the LJSpeech dataset, the extracted folder from the downloaded archive (the folder which contains metadata.txt).
2.`--data` is the path of the LJSpeech dataset, the extracted folder from the downloaded archive (the folder which contains metadata.txt).
3.`--resume` is the path of the checkpoint. If it is provided, the model would load the checkpoint before trainig.
3.`--resume` is the path of the checkpoint. If it is provided, the model would load the checkpoint before trainig.
4.`--output` is the directory to save results, all result are saved in this directory. The structure of the output directory is shown below.
4.`--output` is the directory to save results, all results are saved in this directory. The structure of the output directory is shown below.
```text
```text
├── checkpoints # checkpoint
├── checkpoints # checkpoint
...
@@ -69,7 +69,7 @@ optional arguments:
...
@@ -69,7 +69,7 @@ optional arguments:
5.`--device` is the device (gpu id) to use for training. `-1` means CPU.
5.`--device` is the device (gpu id) to use for training. `-1` means CPU.
usage: synthesis.py [-h] [-c CONFIG] [-g DEVICE] checkpoint text output_path
usage: synthesis.py [-h] [-c CONFIG] [-g DEVICE] checkpoint text output_path
Synthsize waveform with a checkpoint.
Synthsize waveform from a checkpoint.
positional arguments:
positional arguments:
checkpoint checkpoint to load.
checkpoint checkpoint to load.
...
@@ -107,7 +107,7 @@ optional arguments:
...
@@ -107,7 +107,7 @@ optional arguments:
4.`output_path` is the directory to save results. The output path contains the generated audio files (`*.wav`) and attention plots (*.png) for each sentence.
4.`output_path` is the directory to save results. The output path contains the generated audio files (`*.wav`) and attention plots (*.png) for each sentence.
5.`--device` is the device (gpu id) to use for training. `-1` means CPU.
5.`--device` is the device (gpu id) to use for training. `-1` means CPU.
