Model card for eu-delegation-constraints-distilbert

This model is a fine-tuned version of distilbert-base-uncased trained on the
giovannip/eu-delegation-constraints-annotations dataset.
It detects delegating, constraining, and soft obligation provisions in sentences from EU secondary legislation (directives and regulations, 1958–2019).

The model is part of the benchmark suite introduced in

Franchino, F., Migliorati, M., Pagano, G., & Vignoli, V. (2025).
Identifying Delegation and Constraints in Legislative Texts: A Computational Method Applied to the European Union.
European Union Politics.

---

How to use the model

Simple text-classification pipeline

from transformers import pipeline

# Load the fine-tuned model
classifier = pipeline(
    "text-classification",
    model="giovannip/eu-delegation-constraints-distilbert",
    tokenizer="giovannip/eu-delegation-constraints-distilbert",
    return_all_scores=True,
    function_to_apply="sigmoid"  
)

# Example: 
sentence = "The Commission shall adopt implementing acts to lay down the detailed arrangements for achieving the objectives referred to in paragraph 1."

preds = classifier(sentence)[0]

# Display results (sorted by score)
for p in sorted(preds, key=lambda x: x['score'], reverse=True):
    print(f"{p['label']:10s} | {p['score']:.3f}")

Explicit code

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "giovannip/eu-delegation-constraints-distilbert"

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Ensure multi-label mode
model.config.problem_type = "multi_label_classification"

# Example
text = "Member States shall adopt measures necessary to ensure compliance with this Directive."

# Tokenize and run inference
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
with torch.no_grad():
    logits = model(**inputs).logits
probs = torch.sigmoid(logits).squeeze().tolist()

labels = list(model.config.id2label.values())
results = sorted(zip(labels, probs), key=lambda x: x[1], reverse=True)

print("\nIndependent multi-label predictions:")
for label, score in results:
    print(f"{label:10s} | {score:.3f}")

---

Model description

• Architecture: distilBERT, uncased.
• Task: Multi-label sentence classification
• Labels (14): delegation, constraint, and soft-obligation categories for four actors (Member States, NCAs, Commission, Agencies) plus agenda-setting powers
• Objective: Binary cross-entropy with logits (BCEWithLogitsLoss)
• Training: Fine-tuned for 5 epochs on the full annotated corpus after 5-fold cross-validation benchmarking

---

Cross-validation performance (mean across 5 folds, 13 labels)

Metric	Mean (%)
Precision	88.4
Recall	86.9
F1-score	87.2
Matthews Corr. Coef. (MCC)	86.5

---

Intended uses

• Research on delegation and constraints in EU law
• Legal-text classification models

---

Training and evaluation setup

• Training data: 3 000 expert-annotated sentences
• Evaluation: 5-fold cross-validation
• Optimizer: AdamW (lr=2e-5)
• Batch size: 8
• Epochs: 5
• Seed: 42

---

Framework versions

• Transformers 4.46.2
• PyTorch 2.8.0 + CUDA 12.6
• Datasets 4.4.1
• Tokenizers 0.20.3

---

Citation

APA

Franchino, F., Migliorati, M., Pagano, G., & Vignoli, V. (2025).
Identifying Delegation and Constraints in Legislative Texts: A Computational Method Applied to the European Union.
European Union Politics.

BibTeX

@article{franchino2025delegation,
  title={Identifying Delegation and Constraints in Legislative Texts: A Computational Method Applied to the European Union},
  author={Franchino, Fabio and Migliorati, Marta and Pagano, Giovanni and Vignoli, Valerio},
  journal={European Union Politics},
  year={2025}
}