Update README.md

README.md

@@ -1,199 +1,280 @@
 ---
-library_name: transformers
-tags: []
+language: en
+license: apache-2.0
+tags:
+- populism
+- political-discourse
+- text-classification
+- qwen3
+- lora
+- fine-tuned
+datasets:
+- coastalcph/populism-trump-2016
+metrics:
+- accuracy
+- f1
+- precision
+- recall
+base_model: Qwen/Qwen3-4B
+pipeline_tag: text-classification
 ---
 
-# Model Card for Model ID
+# Qwen3-4B Fine-tuned for Populism Detection
 
-<!-- Provide a quick summary of what the model is/does. -->
+## Model Description
 
+This model is a fine-tuned version of [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) trained to identify fine-grained forms of populism in political discourse. The model performs 4-way classification to detect:
 
+- **(a) No populism** - Sentences without populist rhetoric
+- **(b) Anti-elitism** - Negative invocations of "elites"
+- **(c) People-centrism** - Positive invocations of the "people"
+- **(d) Both** - Sentences combining anti-elitism and people-centrism
 
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+The model was fine-tuned using **LoRA (Low-Rank Adaptation)** for parameter-efficient training on political speeches from Donald Trump's 2016 presidential campaign.
 
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
+## Model Details
 
-[More Information Needed]
+### Model Type
+- **Base Model:** Qwen/Qwen3-4B (4B parameters)
+- **Fine-tuning Method:** LoRA (Low-Rank Adaptation)
+- **Task:** Multi-class Text Classification (4 categories)
+- **Language:** English
+- **Domain:** Political discourse
+
+### Training Details
+
+**LoRA Configuration:**
+- **Rank (r):** 16
+- **Alpha:** 32
+- **Dropout:** 0.05
+- **Target Modules:** `q_proj`, `k_proj`, `v_proj`, `o_proj`, `gate_proj`, `up_proj`, `down_proj`
+- **Bias:** None
+- **Task Type:** CAUSAL_LM
+
+**Training Hyperparameters:**
+- **Epochs:** 3
+- **Batch Size:** 8 (per device)
+- **Gradient Accumulation Steps:** 4
+- **Learning Rate:** 2e-4
+- **Learning Rate Scheduler:** Cosine with 0.1 warmup ratio
+- **Optimizer:** paged_adamw_32bit
+- **Weight Decay:** 0.001
+- **Max Gradient Norm:** 1.0
+- **Max Sequence Length:** 300 tokens
+- **Quantization:** 4-bit (NF4) with double quantization
+- **Mixed Precision:** FP16/BF16
+- **Total Training Steps:** 1,476
+
+**Training Loss Progression:**
+- Step 1: 4.9333
+- Step 100: 0.4583
+- Step 500: 0.0098
+- Step 1000: 0.003
+- Step 1476: ~0.001
+
+## Training Data
+
+**Dataset:** [coastalcph/populism-trump-2016](https://huggingface.co/datasets/coastalcph/populism-trump-2016)
+
+**Data Source:** Sentence-level annotations from Donald Trump's 2016 presidential campaign speeches.
+
+**Label Distribution (Original):**
+- **(a) No populism:** 92%
+- **(b) Anti-elitism:** 4%
+- **(c) People-centrism:** 2%
+- **(d) Both:** 2%
+
+**Data Preprocessing:**
+- Sentences truncated to maximum 48 words
+- **5x upsampling** applied to minority classes (b, c, d) to address class imbalance
+- Tokenized with max_length=300
+- Shuffled with seed=42
+
+**Populism Definition:**
+Following political science literature, populism is defined as an anti-elite discourse in the name of "the people", characterized by:
+1. **Anti-elitism:** Negative references to elite groups
+2. **People-centrism:** Positive references to ordinary citizens or "the people"
+
+## How to Use
+
+### Option 1: Load Merged Model (Recommended)
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
+
+# Load model and tokenizer
+model_name = "armaniii/Qwen3-4B-populism"
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+model = AutoModelForCausalLM.from_pretrained(
+    model_name,
+    device_map="auto",
+    torch_dtype="auto"
+)
+
+# Create pipeline
+pipe = pipeline(
+    "text-generation",
+    model=model,
+    tokenizer=tokenizer
+)
+```
+
+### Inference Example
+
+```python
+# System prompt
+SYSTEM_PROMPT = """You are a helpful AI assistant with expertise in identifying populism in public discourse.
+
+Populism can be defined as an anti-elite discourse in the name of the "people". In other words, populism emphasizes the idea of the common "people" and often positions this group in opposition to a perceived elite group.
+
+There are two core elements in identifying populism: (i) anti-elitism, i.e., negative invocations of "elites", and (ii) people-centrism, i.e., positive invocations of the "people".
+"""
+
+# Instruction template
+INSTRUCTION = """You must classify each sentence in one of the following categories:
+
+(a) No populism.
+(b) Anti-elitism, i.e., negative invocations of "elites".
+(c) People-centrism, i.e., positive invocations of the "People".
+(d) Both people-centrism and anti-elitism populism.
+"""
+
+# Example sentence
+sentence = "We need to stand up against the corrupt establishment that has betrayed the American people."
+
+# Build prompt
+conversation = [
+    {"role": "system", "content": SYSTEM_PROMPT},
+    {"role": "user", "content": INSTRUCTION + f'\n\nWhich is the most relevant category for the sentence: "{sentence}"?'}
+]
+
+# Apply chat template
+prompt = tokenizer.apply_chat_template(
+    conversation=conversation,
+    tokenize=False,
+    add_generation_prompt=True
+)
+
+# Generate response
+response = pipe(
+    prompt,
+    do_sample=False,
+    max_new_tokens=256,
+    eos_token_id=tokenizer.eos_token_id,
+    pad_token_id=tokenizer.eos_token_id,
+)
 
-## Environmental Impact
+print(response[0]['generated_text'])
+# Expected output: "I would categorize this sentence as (d)..."
+```
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+### Expected Output Format
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+The model responds in the following format:
+```
+I would categorize this sentence as (X)
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+[Explanation of the classification decision]
+```
+
+Where `X` is one of: `a`, `b`, `c`, or `d`.
+
+## Performance
+
+Detailed performance metrics and comparisons are available in the original paper (see Citation section below).
 
-## Technical Specifications [optional]
+**Key Findings:**
+- Fine-tuned models significantly outperform zero-shot instruction-tuned LLMs
+- The model shows strong performance on in-domain political discourse
+- Cross-context evaluation demonstrates reasonable generalization to European political speeches
+- LoRA fine-tuning provides efficient adaptation with minimal trainable parameters
 
-### Model Architecture and Objective
+## Limitations and Bias
 
-[More Information Needed]
+**Training Data Limitations:**
+- Trained primarily on Donald Trump's 2016 campaign speeches
+- May not generalize equally well to other political contexts, time periods, or speakers
+- Performance may vary on non-US political discourse
 
-### Compute Infrastructure
+**Class Imbalance:**
+- Original data is highly imbalanced (92% "No populism")
+- Upsampling (5x) applied to minority classes during training
+- Model may still show bias toward the majority class
+
+**Domain Specificity:**
+- Optimized for sentence-level classification
+- Performance on longer texts may require sentence segmentation
+- Best suited for political discourse and campaign rhetoric
+
+**Ethical Considerations:**
+- This model provides automated analysis and should not be the sole basis for political judgments
+- Results should be interpreted by domain experts
+- May reflect biases present in the training data
 
-[More Information Needed]
+## Intended Use
 
-#### Hardware
+**Primary Use Cases:**
+- Research in political science and computational social science
+- Analysis of populist rhetoric in political campaigns
+- Educational purposes in understanding populist discourse
+- Automated annotation of political speech datasets
 
-[More Information Needed]
+**Out-of-Scope Use:**
+- Making definitive political judgments about individuals or parties
+- Real-time moderation or censorship of political speech
+- Use without human oversight in sensitive political contexts
 
-#### Software
+## Citation
 
-[More Information Needed]
+If you use this model, please cite the original paper:
 
-## Citation [optional]
+```bibtex
+@misc{chalkidis2025populism,
+    title={Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns},
+    author={Chalkidis, Ilias and Brandl, Stephanie and Aslanidis, Paris},
+    year={2025},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL},
+    url={https://arxiv.org/abs/2507.19303},
+    doi={10.48550/arXiv.2507.19303}
+}
+```
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+**Paper:** [Identifying Fine-grained Forms of Populism in Political Discourse](https://arxiv.org/abs/2507.19303)
 
-**BibTeX:**
+**Dataset:** [coastalcph/populism-trump-2016](https://huggingface.co/datasets/coastalcph/populism-trump-2016)
 
-[More Information Needed]
+## Acknowledgments
 
-**APA:**
+This model was developed based on the research by:
+- **Ilias Chalkidis** (University of Copenhagen)
+- **Stephanie Brandl** (University of Copenhagen)
+- **Paris Aslanidis** (Aristotle University of Thessaloniki)
 
-[More Information Needed]
+The fine-tuning was performed using:
+- **Base Model:** [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) by Alibaba Cloud
+- **Dataset:** [coastalcph/populism-trump-2016](https://huggingface.co/datasets/coastalcph/populism-trump-2016)
+- **PEFT Library:** Hugging Face PEFT for LoRA implementation
+- **Training Framework:** Hugging Face Transformers and TRL
 
-## Glossary [optional]
+## Model Card Authors
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+[Your Name/Organization]
 
-[More Information Needed]
+## Model Card Contact
 
-## More Information [optional]
+[Your Contact Information]
 
-[More Information Needed]
+## License
 
-## Model Card Authors [optional]
+This model is released under the Apache 2.0 License, consistent with the Qwen3-4B base model.
 
-[More Information Needed]
+**Responsible Use:** This model is intended for research and educational purposes. Users should be aware of the limitations and biases described above and use the model responsibly with appropriate human oversight.
 
-## Model Card Contact
+---
 
-[More Information Needed]
+**Related Resources:**
+- 📄 [Paper](https://arxiv.org/abs/2507.19303)
+- 🤗 [Dataset](https://huggingface.co/datasets/coastalcph/populism-trump-2016)
+- 💻 [GitHub Repository](https://github.com/armaniii)
+- 🎯 [Demo Space](https://huggingface.co/spaces/armaniii/populism-detector)