profile
viewpoint

Jackiexiao/MTTS 217

A Demo of Mandarin/Chinese TTS frontend

Jackiexiao/obsidian 147

obsidian中文帮助手册

Jackiexiao/10-minutes-to-SpaceVim 139

10-minutes-to-SpaceVim

Jackiexiao/zhtts 33

A demo of zh/Chinese Text to Speech system run on CPU in real time.

Jackiexiao/foam-mkdocs-template 10

using mkdocs / mkdocs-material / mkdocs-roamlinks-plugin

Jackiexiao/init_ubuntu 4

init_ubuntu_16.04

Jackiexiao/10-minutes-to-foam 2

A quick start guide for foam, a personal knowledge management and sharing system

Jackiexiao/docker-TensorFlowTTS-zh 2

Docker image for Mozilla TTS server

Jackiexiao/mkdocs-roamlinks-plugin 1

An MkDocs plugin that automagically generates relative links between markdown pages

startedmindslab-ai/cotatron

started time in 5 minutes

startedalshedivat/al-folio

started time in 15 minutes

issue commentfatchord/WaveRNN

train_tacotron.py: Random CUBLAS_STATUS_INTERNAL_ERROR

Nice, that's a good solution. And yeah, I found out this morning that it picks up from where it left off very well. How many epochs did you leave it to train for? I'm on 100k so far and will probably let it run until close to a million I guess.

I just followed in this guy's steps and fine-tuned the pre-trained model on my own data. I tried going up to 300k, but I found it starts getting worse after ~260k. I don't think I ever tried training it from scratch.

1 million epochs? Wow, that would take quite a while on my hardware. Can I ask what GPU you're using and how fast your training goes?

serg06

comment created time in 33 minutes

issue commentfatchord/WaveRNN

train_tacotron.py: Random CUBLAS_STATUS_INTERNAL_ERROR

Nice, that's a good solution. And yeah, I found out this morning that it picks up from where it left off very well. How many epochs did you leave it to train for? I'm on 100k so far and will probably let it run until close to a million I guess.

serg06

comment created time in an hour

issue commentfatchord/WaveRNN

train_tacotron.py: Random CUBLAS_STATUS_INTERNAL_ERROR

I'm seeing the same thing. Did you find a fix?

I didn't find a fix but I did find a workaround: Automatically restarting after a crash.

train.bat:

:loop
python train_tacotron.py
echo Crash detected, restarting...
timeout /t 5 /nobreak
goto loop

Also, are you able to pick up training from where you left off?

Yep, it always restarts from the latest step for me.

serg06

comment created time in an hour

startedBogiHsu/WG-WaveNet

started time in 2 hours

fork zermylf/eat_pytorch_in_20_days

Pytorch🍊🍉 is delicious, just eat it! 😋😋

fork in 2 hours

fork zermylf/eat_tensorflow2_in_30_days

Tensorflow2.0 🍎🍊 is delicious, just eat it! 😋😋

fork in 2 hours

issue commentNVIDIA/waveglow

How to train new Waveglow model for diffirent language?

@EuphoriaCelestial, I've installed successfully, how many hours (minimum) of audio need to generate correct spelled audio ?

EuphoriaCelestial

comment created time in 2 hours

issue commentNVIDIA/waveglow

How to train new Waveglow model for diffirent language?

just comment out the line, you can add it back the next time to generate a checkpoint.

EuphoriaCelestial

comment created time in 2 hours

issue commentNVIDIA/waveglow

How to train new Waveglow model for diffirent language?

Unable to run train.py : File "train.py", line 188, in <module> train(num_gpus, args.rank, args.group_name, **train_config) File "train.py", line 90, in train optimizer) File "train.py", line 45, in load_checkpoint optimizer.load_state_dict(checkpoint_dict['optimizer']) File "C:\Python37\lib\site-packages\torch\optim\optimizer.py", line 124, in load_state_dict raise ValueError("loaded state dict contains a parameter group " ValueError: loaded state dict contains a parameter group that doesn't match the size of optimizer's group

Can you help me, please?

EuphoriaCelestial

comment created time in 2 hours

issue commentmozilla/TTS

Getting error "RuntimeError: CUDA error: device-side assert triggered" in epoch 0 on WaveGrad vocoder training

I realise each case can be different but I'm pretty sure it went okay for me using the WaveGrad in TTS with the LR at 1e-4, although I did hit some minor problems where I was continuing training (although looks like those aren't quite what you have here).

For the original error, "CUDA error: device-side assert triggered", I think you may be able to get more detail if you run it again with: CUDA_LAUNCH_BLOCKING="1"

thorstenMueller

comment created time in 2 hours

issue commentmozilla/TTS

No module named 'TTS'

I am trying to run my code on collab but it gives me this error.

Traceback (most recent call last):
  File "TTS/train.py", line 27, in <module>
    from TTS.utils.speakers import load_speaker_mapping, save_speaker_mapping, \
  File "/usr/local/lib/python3.6/dist-packages/TTS-0.0.3+b1935c9-py3.6.egg/TTS/utils/speakers.py", line 4, in <module>
    from TTS.datasets.preprocess import get_preprocessor_by_name
  File "/usr/local/lib/python3.6/dist-packages/TTS-0.0.3+b1935c9-py3.6.egg/TTS/datasets/preprocess.py", line 9, in <module>
    from TTS.tts.utils.generic_utils import split_dataset
ModuleNotFoundError: No module named 'TTS.tts'

how to fix it?

athuljayaraj

comment created time in 3 hours

startedJackiexiao/obsidian

started time in 3 hours

startedAzure/DeepLearningForTimeSeriesForecasting

started time in 3 hours

startedchubin/cheat.sh

started time in 3 hours

issue commentmozilla/TTS

Getting error "RuntimeError: CUDA error: device-side assert triggered" in epoch 0 on WaveGrad vocoder training

That would be the most obvious, but it tried decreasing batchsizes from 96 (default) down to 8 with no effect. I also tried changing lr from 1e-4 (default) to 2e-4 (default value in freds0 wavegrad implementation).

I initially set "stats_path" to the "taco2-scale_stats.npy" from our taco2 model training (is this a good idea in general)? If i leave "stats_path" empty i receive a warning before training is failing with previous written error:

 > TRAINING (2020-11-30 15:32:13) 
/home/thorsten/___prj/tts/models/vocoder/wavegrad/mozilla/lib/python3.6/site-packages/torch/optim/lr_scheduler.py:123: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
thorstenMueller

comment created time in 3 hours

startedJackiexiao/zhtts

started time in 4 hours

issue commentmozilla/TTS

Getting error "RuntimeError: CUDA error: device-side assert triggered" in epoch 0 on WaveGrad vocoder training

it is probably OOM error in a different face.

thorstenMueller

comment created time in 4 hours

issue closedmozilla/TTS

ValueError: empty range for randrange() (0,-8, -8)

Error when I try to train vocoder with custom dataset.

 > Using CUDA:  True
 > Number of GPUs:  1
fatal: not a git repository (or any of the parent directories): .git
 > Git Hash: 0000000
 > Experiment folder: vocoder_model/multiband-melgan-Kasım-26-2020_01+29ÖS-0000000
fatal: not a git repository (or any of the parent directories): .git
 > Loading wavs from: LJSpeech-1.1/wavs/
 > Setting up Audio Processor...
 | > sample_rate:22050
 | > num_mels:80
 | > min_level_db:-100
 | > frame_shift_ms:None
 | > frame_length_ms:None
 | > ref_level_db:0
 | > fft_size:1024
 | > power:None
 | > preemphasis:0.0
 | > griffin_lim_iters:None
 | > signal_norm:True
 | > symmetric_norm:True
 | > mel_fmin:50.0
 | > mel_fmax:7600.0
 | > spec_gain:1.0
 | > stft_pad_mode:reflect
 | > max_norm:4.0
 | > clip_norm:True
 | > do_trim_silence:True
 | > trim_db:60
 | > do_sound_norm:False
 | > stats_path:scale_stats.npy
 | > hop_length:256
 | > win_length:1024
 > Generator Model: multiband_melgan_generator
 > Discriminator Model: melgan_multiscale_discriminator
 > Generator has 2539448 parameters
 > Discriminator has 4354998 parameters

 > EPOCH: 0/10000

 > TRAINING (2020-11-26 13:29:50) 
 ! Run is removed from vocoder_model/multiband-melgan-Kasım-26-2020_01+29ÖS-0000000
Traceback (most recent call last):
  File "TTS/vocoder/train.py", line 648, in <module>
    main(args)
  File "TTS/vocoder/train.py", line 552, in main
    epoch)
  File "TTS/vocoder/train.py", line 101, in train
    for num_iter, data in enumerate(data_loader):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 363, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 971, in _next_data
    return self._process_data(data)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 1014, in _process_data
    data.reraise()
  File "/usr/local/lib/python3.6/dist-packages/torch/_utils.py", line 395, in reraise
    raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 2.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/worker.py", line 185, in _worker_loop
    data = fetcher.fetch(index)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 44, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.6/dist-packages/TTS-0.0.3+b1935c9-py3.6.egg/TTS/vocoder/datasets/gan_dataset.py", line 70, in __getitem__
    item1 = self.load_item(idx)
  File "/usr/local/lib/python3.6/dist-packages/TTS-0.0.3+b1935c9-py3.6.egg/TTS/vocoder/datasets/gan_dataset.py", line 116, in load_item
    mel_start = random.randint(0, max_mel_start)
  File "/usr/lib/python3.6/random.py", line 221, in randint
    return self.randrange(a, b+1)
  File "/usr/lib/python3.6/random.py", line 199, in randrange
    raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,-8, -8)

How to fix it?

closed time in 4 hours

xoxoxo13102020

issue commentmozilla/TTS

ValueError: empty range for randrange() (0,-8, -8)

probably.

I close this issue as you fixed it.

xoxoxo13102020

comment created time in 4 hours

issue openedmozilla/TTS

Getting error "RuntimeError: CUDA error: device-side assert triggered" in epoch 0 on WaveGrad vocoder training

First things first: Thanks @nmstoker for providing great gatherup tool :+1: .

  • I'm trying to train WaveGrad vocoder model for use with our pretrained taco2 thorsten model (ljspeech structure)
  • Using NVidia xavier AGX machine (ubuntu aarch64) for training
  • Copied and adjusted "wavegrad_libritts.json" to match our taco2 audio settings
  • I've uploaded taco2 config (irrelevant for wavegrad training, or?) and vocoder config used for training. wavegrad-thorsten-conf.zip

Details

While in epoch 0 i get following error:

python ./TTS/bin/train_vocoder_wavegrad.py --config_path ./TTS/vocoder/configs/wavegrad_thorsten.json 
 > Using CUDA:  True
 > Number of GPUs:  1
   >  Mixed precision is enabled
 > Git Hash: ac46c3f
 > Experiment folder: /home/thorsten/___prj/tts/models/vocoder/wavegrad/mozilla/wavegrad-model-output/wavegrad-thorsten-November-30-2020_01+03PM-ac46c3f
 > Loading wavs from: /home/thorsten/___prj/tts/datasets/thorsten-de_v02/
 > Setting up Audio Processor...
 | > sample_rate:22050
 | > resample:False
 | > num_mels:80
 | > min_level_db:-100
 | > frame_shift_ms:None
 | > frame_length_ms:None
 | > ref_level_db:20
 | > fft_size:1024
 | > power:None
 | > preemphasis:0.0
 | > griffin_lim_iters:None
 | > signal_norm:True
 | > symmetric_norm:True
 | > mel_fmin:0
 | > mel_fmax:8000.0
 | > spec_gain:20.0
 | > stft_pad_mode:reflect
 | > max_norm:1.0
 | > clip_norm:True
 | > do_trim_silence:True
 | > trim_db:60
 | > do_sound_norm:False
 | > stats_path:/home/thorsten/___prj/tts/models/vocoder/wavegrad/mozilla/taco2-files/taco2-scale_stats.npy
 | > hop_length:256
 | > win_length:1024
 > Generator Model: wavegrad
 > WaveGrad has 15827106 parameters

 > EPOCH: 0/10000

 > TRAINING (2020-11-30 13:04:15) 
/media/nvidia/WD_NVME/PyTorch/JetPack_4.4/GA/pytorch-v1.6.0/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [47,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
 ! Run is removed from /home/thorsten/___prj/tts/models/vocoder/wavegrad/mozilla/wavegrad-model-output/wavegrad-thorsten-November-30-2020_01+03PM-ac46c3f
Traceback (most recent call last):
  File "./TTS/bin/train_vocoder_wavegrad.py", line 504, in <module>
    main(args)
  File "./TTS/bin/train_vocoder_wavegrad.py", line 401, in main
    epoch)
  File "./TTS/bin/train_vocoder_wavegrad.py", line 116, in train
    noise, x_noisy, noise_scale = model.compute_y_n(x)
  File "/home/thorsten/___prj/tts/models/vocoder/wavegrad/mozilla/lib/python3.6/site-packages/TTS-0.0.7+ac46c3f-py3.6-linux-aarch64.egg/TTS/vocoder/models/wavegrad.py", line 110, in compute_y_n
    noise_scale = l_a + torch.rand(y_0.shape[0]).to(y_0) * (l_b - l_a)
RuntimeError: CUDA error: device-side assert triggered
(mozilla) thorsten@nvidia-agx:~/___prj/tts/models/vocoder/wavegrad/mozilla/TTS$ 

Platform OS

  • Linux

Python Environment

  • Python 3.6.9

  • Virtual env: Venv / virtualenv

Package Installation

  • TTS installed from source on GitHub <br> <details> <summary>Click to see package list (package count: 116)</summary><br>

    :package: Package list from Pip

    Package Version
    absl-py 0.11.0
    astroid 2.4.2
    astunparse 1.6.3
    attrdict 2.0.1
    attrs 20.3.0
    audioread 2.1.9
    bokeh 1.4.0
    cachetools 4.1.1
    cardboardlint 1.3.0
    certifi 2020.11.8
    cffi 1.14.4
    chardet 3.0.4
    click 7.1.2
    clldutils 3.5.4
    colorama 0.4.4
    colorlog 4.6.2
    commonmark 0.9.1
    confuse 1.3.0
    csvw 1.8.1
    cycler 0.10.0
    Cython 0.29.21
    dataclasses 0.7
    decorator 4.4.2
    docopt 0.6.2
    filelock 3.0.12
    Flask 1.1.2
    future 0.18.2
    gast 0.3.3
    gatherup 0.0.4
    gdown 3.12.2
    german-transliterate 0.1.3
    google-auth 1.23.0
    google-auth-oauthlib 0.4.2
    google-pasta 0.2.0
    grpcio 1.33.2
    h5py 2.10.0
    idna 2.10
    importlib-metadata 3.1.0
    importlib-resources 3.3.0
    inflect 5.0.2
    isodate 0.6.0
    isort 4.3.21
    itsdangerous 1.1.0
    Jinja2 2.11.2
    joblib 0.17.0
    Keras-Preprocessing 1.1.2
    kiwisolver 1.3.1
    lazy-object-proxy 1.4.3
    librosa 0.7.2
    llvmlite 0.31.0
    Markdown 3.3.3
    MarkupSafe 1.1.1
    matplotlib 3.3.3
    mccabe 0.6.1
    nose 1.3.7
    num2words 0.5.10
    numba 0.48.0
    numpy 1.18.5
    oauthlib 3.1.0
    opt-einsum 3.3.0
    packaging 20.7
    phonemizer 2.2.1
    Pillow 8.0.1
    pip 20.2.4
    pkg-resources 0.0.0
    prompt-toolkit 3.0.8
    protobuf 3.14.0
    pyasn1 0.4.8
    pyasn1-modules 0.2.8
    pycparser 2.20
    Pygments 2.7.2
    pylint 2.5.3
    pyparsing 2.4.7
    pysbd 0.3.3
    PySocks 1.7.1
    python-dateutil 2.8.1
    pyworld 0.2.12
    PyYAML 5.3.1
    questionary 1.8.1
    regex 2020.11.13
    requests 2.25.0
    requests-oauthlib 1.3.0
    resampy 0.2.2
    rfc3986 1.4.0
    rich 8.0.0
    rsa 4.6
    scikit-learn 0.23.2
    scipy 1.4.1
    segments 2.1.3
    setuptools 50.3.2
    six 1.15.0
    SoundFile 0.10.3.post1
    tabulate 0.8.7
    tensorboard 2.4.0
    tensorboard-plugin-wit 1.7.0
    tensorboardX 2.1
    tensorflow 2.3.0+nv20.9
    tensorflow-estimator 2.3.0
    termcolor 1.1.0
    threadpoolctl 2.1.0
    toml 0.10.2
    torch 1.6.0
    tornado 6.1
    tqdm 4.54.0
    TTS 0.0.7+ac46c3f
    typed-ast 1.4.1
    typing-extensions 3.7.4.3
    umap-learn 0.4.6
    Unidecode 0.4.20
    uritemplate 3.0.1
    urllib3 1.26.2
    wcwidth 0.2.5
    Werkzeug 1.0.1
    wheel 0.35.1
    wrapt 1.12.1
    zipp 3.4.0

</details>

- generated at 13:31 on Nov 30 2020 using Gather Up tool :gift:

created time in 4 hours

startedzhaoyang-star/test_opencl_image_object

started time in 5 hours

issue commentmozilla/TTS

PR: Use german_transliterate as German cleaner lib

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. You might also look our discourse page for further help. https://discourse.mozilla.org/c/tts

repodiac

comment created time in 5 hours

startedJackiexiao/zhtts

started time in 5 hours

issue commentmozilla/TTS

ValueError: empty range for randrange() (0,-8, -8)

I think I fixed this issue by changing do_trimming argument as false in config file. Will it affect performance of model?

xoxoxo13102020

comment created time in 5 hours

startedJackiexiao/zhtts

started time in 6 hours

fork qhduan/zhtts

A demo of zh/Chinese Text to Speech system run on CPU in real time.

fork in 6 hours

issue openedNVIDIA/waveglow

Expect the `waveglow_old.pt` model file

Hello, author! Do you have the download address for thewaveglow_old.pt model? I want to use this model for testing.

created time in 6 hours

startedPOSTECH-CVLab/PyTorch-StudioGAN

started time in 6 hours

more