add section Available models and languages

This commit is contained in:
Jong Wook Kim 2022-09-22 05:36:25 +09:00
parent cfd6bdda21
commit 49a3ffc997
2 changed files with 4630 additions and 1 deletions

View File

@ -35,9 +35,30 @@ choco install ffmpeg
```
## Available models and languages
There are five model sizes, four with English-only versions, offering speed and accuracy tradeoffs. Below are the names of the available models and their approximate memory requirements and relative speed.
| Size | Parameters | English-only model | Multilingual model | Required VRAM | Relative speed |
|:------:|:----------:|:------------------:|:------------------:|:-------------:|:--------------:|
| tiny | 39 M | `tiny.en` | `tiny` | ~1 GB | ~32x |
| base | 74 M | `base.en` | `base` | ~1 GB | ~16x |
| small | 244 M | `small.en` | `small` | ~2 GB | ~6x |
| medium | 769 M | `medium.en` | `medium` | ~5 GB | ~2x |
| large | 1550 M | N/A | `large` | ~10 GB | 1x |
For English-only applications, the `.en` models tend to perform better, especially for the `tiny.en` and `base.en` models. We observed that the difference becomes less significant for the `small.en` and `medium.en` models.
Whisper's performance varies widely depending on the language. The figure below shows a WER breakdown by languages of Fleurs dataset, using the `large` model. More WER and BLEU scores corresponding to the other models and datasets can be found in Appendix D in [the paper](https://cdn.openai.com/papers/whisper.pdf).
![WER breakdown by language](language-breakdown.svg)
## Command-line usage
There are five model sizes, four with English-only versions, offering speed and accuracy tradeoffs. The following command will transcribe speech in audio files, using the `medium` model:
The following command will transcribe speech in audio files, using the `medium` model:
whisper audio.flac audio.mp3 audio.wav --model medium

4608
language-breakdown.svg Normal file

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 134 KiB