File size: 2,002 Bytes
3ad4db1 fbd27fc 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 3982913 3ad4db1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
---
language:
- en
tags:
- automatic-speech-recognition
- speech-to-text
- whisper
- peft
- lora
- seq2seq
base_model: openai/whisper-small
pipeline_tag: automatic-speech-recognition
---
This model is a fine-tuned version of openai/whisper-small on the UA-Speech dataset.
## Model description
This model fine-tunes Whisper-small for English transcription on dysarthric speech. Training used LoRA (PEFT) on attention and feed-forward modules, and the adapter was merged into the base model weights for deployment (no PEFT required at inference time). This model is used to show generalization via constrained capacity.
## Intended uses & limitations
This model is intended for automatic speech recognition (ASR) on English speech, with an emphasis on robustness to atypical/dysarthric speech patterns resembling UA-Speech-style data. Performance may degrade on out-of-domain audio, heavy noise, non-English speech, or audio sampled far from 16 kHz. For best results, provide mono 16 kHz audio.
## Audio preprocessing
- Audio loaded with `soundfile.read(file_path)`
- If stereo/multi-channel, converted to mono by averaging channels
- Features extracted with `WhisperProcessor.feature_extractor(..., sampling_rate=16000)`
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
learning_rate: 3e-4
train_batch_size: 16
seed: 42
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 8
mixed_precision_training: Native AMP
## LoRA / PEFT configuration
- task_type: SEQ_2_SEQ_LM
- r: 64
- lora_alpha: 128
- lora_dropout: 0.1
- target_modules: ["q_proj", "v_proj", "fc1", "fc2"]
- modules_to_save: None
## Model config modifications
- model.config.forced_decoder_ids = None
- model.config.suppress_tokens = []
## Training results
WER/CER were computed offline after training.
## Framework versions
Transformers 4.56.2
Pytorch 2.8.0+cu128
Datasets 4.4.1
Tokenizers 0.22.1
|