---
library_name: transformers
tags: []
---

# Model Card for `Walid-Ahmed/finetuned-falcon-medtext`

This model is a fine-tuned version of the Falcon-7B model, trained on a medical question-answering dataset to generate medically relevant text. It uses causal language modeling and was fine-tuned to better understand structured queries and answers in the healthcare domain.

---

## Model Details

### Model Description

This model adapts the Falcon-7B architecture for domain-specific use in medical dialogue and question answering. It was trained using the Hugging Face `Trainer` with causal language modeling on a dataset that combines patient prompts and medically appropriate responses.

- **Developed by:** Walid Ahmed  
- **Model type:** Causal Language Model  
- **Language(s):** English  
- **License:** Apache 2.0  
- **Finetuned from:** [tiiuae/falcon-7b](https://huggingface.co/tiiuae/falcon-7b)

### Model Sources

- **Repository:** https://huggingface.co/Walid-Ahmed/finetuned-falcon-medtext

---

## Uses

### Direct Use

This model can be used as a text-generation tool for medical Q&A or dialogue simulation, particularly for research, academic, and educational purposes.

#### Example Prompt:

```
A 25-year-old female presents with swelling, pain, and inability to bear weight on her left ankle following a fall during a basketball game where she landed awkwardly on her foot. The pain is on the outer side of her ankle. What is the likely diagnosis and next steps?
```

### Expected Output

The model will generate a medically plausible diagnosis (e.g., lateral ankle sprain or fracture) and suggest typical next steps such as physical examination, Ottawa ankle rules assessment, or imaging (e.g., ankle X-ray) based on clinical best practices.

### Downstream Use

- Educational tools in clinical reasoning  
- Prompt-based evaluation research  
- Foundation for further fine-tuning in healthcare

### Out-of-Scope Use

- Real-world diagnosis or treatment  
- Use in clinical settings  
- Any patient-facing applications without human supervision

---

## Bias, Risks, and Limitations

- May hallucinate or generate incorrect medical information  
- Model is not verified by medical professionals  
- Trained data may reflect bias or outdated practices

### Recommendations

- Strictly for educational and research use  
- Avoid real-world clinical reliance without oversight

---

## How to Get Started with the Model

```python
from transformers import pipeline

pipe = pipeline("text-generation", model="Walid-Ahmed/finetuned-falcon-medtext")

prompt = (
    "A 25-year-old female presents with swelling, pain, and inability to bear weight "
    "on her left ankle following a fall during a basketball game where she landed awkwardly "
    "on her foot. The pain is on the outer side of her ankle. What is the likely diagnosis "
    "and next steps?"
)

output = pipe(prompt, max_new_tokens=100, do_sample=True, temperature=0.7)
print(output[0]["generated_text"])
```

**Example Output:**
```
The likely diagnosis is a lateral ankle sprain or a possible fracture of the fibula, such as a distal fibular fracture. The next steps include a physical examination using the Ottawa ankle rules and ordering an X-ray of the ankle to rule out a fracture. Initial treatment should involve R.I.C.E. (Rest, Ice, Compression, Elevation) and pain control.
```

---

## Training Details

### Training Data

This model was fine-tuned on the [BI55/MedText](https://huggingface.co/datasets/BI55/MedText) dataset.

The dataset contains over 1,000 high-quality clinical scenarios, representing textbook-level patient presentations across:

- The 100 most common diseases
- The 30 most common hospital-presenting injuries
- Scenarios ranging from mild to severe

The dataset also includes:
- Scenarios where the AI should **not** answer due to ambiguity or lack of data
- Ethical messaging reminding users that AI **cannot replace a human doctor**
- Completions that encourage seeking **professional medical advice**

#### Sample Prompt & Completion

**Prompt:**

```
A 50-year-old male presents with a history of recurrent kidney stones and osteopenia. He has been taking high-dose vitamin D supplements due to a previous diagnosis of vitamin D deficiency. Laboratory results reveal hypercalcemia and hypercalciuria. What is the likely diagnosis, and what is the treatment?
```

**Completion:**

```
This patient's history of recurrent kidney stones, osteopenia, and high-dose vitamin D supplementation, along with laboratory findings of hypercalcemia and hypercalciuria, suggest the possibility of vitamin D toxicity. Excessive intake of vitamin D can cause increased absorption of calcium from the gut, leading to hypercalcemia and hypercalciuria, which can result in kidney stones and bone loss. Treatment would involve stopping the vitamin D supplementation and potentially providing intravenous fluids and loop diuretics to promote the excretion of calcium.
```

---

### Training Procedure

#### Preprocessing

- Tokenized using Falcon-7B tokenizer  
- Formatted as single-sequence CLM tasks  
- Max input length: 1024 tokens

#### Training Hyperparameters

```python
from transformers import TrainingArguments

training_args = TrainingArguments(
    output_dir="./finetuned_falcon",
    eval_strategy="epoch",
    learning_rate=2e-5,
    weight_decay=0.01,
    fp16=True,
    per_device_train_batch_size=1,
    per_device_eval_batch_size=1,
    gradient_accumulation_steps=1,
    logging_steps=1,
    num_train_epochs=1,
    optim="paged_adamw_8bit",
    report_to="none"
)
```

#### Speeds, Sizes, Times

- Training duration: ~[fill in]  
- Model size: ~13.4 GB (fp16)  
- Hardware used: e.g., 1× A100 80GB

---

## Evaluation

### Testing

Manual inspection of unseen medical prompts.  
Focus: Fluency, correctness, and relevance.

### Metrics

No automated evaluation metrics reported.  
Qualitative review confirms accurate basic clinical reasoning.

---

## Environmental Impact

- **Hardware Type:** NVIDIA A100  
- **Hours used:** [estimated]  
- **Compute Region:** [Optional]  
- **Carbon Emitted:** Estimate using [ML CO2 calculator](https://mlco2.github.io/impact#compute)

---

## Technical Specifications

- Architecture: Falcon-7B  
- Objective: Causal Language Modeling (CLM)  
- Precision: fp16  
- Training Framework: Hugging Face Transformers  
- Optimizer: 8-bit AdamW (paged)

---

## Citation

**BibTeX:**
```bibtex
@misc{falcon-medtext,
  author = {Walid Ahmed},
  title = {Fine-Tuned Falcon-7B for Medical QA},
  year = {2025},
  url = {https://huggingface.co/Walid-Ahmed/finetuned-falcon-medtext}
}
```

**APA:**

Ahmed, W. (2025). *Fine-Tuned Falcon-7B for Medical QA*. Hugging Face. https://huggingface.co/Walid-Ahmed/finetuned-falcon-medtext

---

## Contact

**Author:** Walid Ahmed  
**Email:** walidmaly@yahoo.com  
**Date:** April 17, 2025

---