---
language:
- en
tags:
- automatic-speech-recognition
- speech-to-text
- whisper
- peft
- lora
- seq2seq
base_model: openai/whisper-small
pipeline_tag: automatic-speech-recognition
---

This model is a fine-tuned version of openai/whisper-small on the UA-Speech dataset.

## Model description
This model fine-tunes Whisper-small for English transcription on dysarthric speech. Training used LoRA (PEFT) on attention and feed-forward modules, and the adapter was merged into the base model weights for deployment (no PEFT required at inference time). This model is used to show generalization via constrained capacity.

## Intended uses & limitations
This model is intended for automatic speech recognition (ASR) on English speech, with an emphasis on robustness to atypical/dysarthric speech patterns resembling UA-Speech-style data. Performance may degrade on out-of-domain audio, heavy noise, non-English speech, or audio sampled far from 16 kHz. For best results, provide mono 16 kHz audio.

## Audio preprocessing
- Audio loaded with `soundfile.read(file_path)`
- If stereo/multi-channel, converted to mono by averaging channels
- Features extracted with `WhisperProcessor.feature_extractor(..., sampling_rate=16000)`

## Training procedure

### Training hyperparameters
The following hyperparameters were used during training:

learning_rate: 3e-4  
train_batch_size: 16  
seed: 42  
optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08  
lr_scheduler_type: linear  
num_epochs: 8  
mixed_precision_training: Native AMP  

## LoRA / PEFT configuration
- task_type: SEQ_2_SEQ_LM
- r: 64
- lora_alpha: 128
- lora_dropout: 0.1
- target_modules: ["q_proj", "v_proj", "fc1", "fc2"]
- modules_to_save: None

## Model config modifications
- model.config.forced_decoder_ids = None
- model.config.suppress_tokens = []

## Training results
WER/CER were computed offline after training.

## Framework versions
Transformers 4.56.2  
Pytorch 2.8.0+cu128  
Datasets 4.4.1  
Tokenizers 0.22.1