| | --- |
| | library_name: transformers |
| | tags: |
| | - text-classification |
| | - spam-detection |
| | - sms |
| | license: apache-2.0 |
| | --- |
| | |
| | # 🛡️ Model Card for `alusci/distilbert-smsafe` |
| |
|
| | A lightweight DistilBERT model fine-tuned for spam detection in SMS messages. The model classifies input messages as either **spam** or **ham** (not spam), using a custom dataset of real-world OTP (One-Time Password) and spam SMS messages. |
| |
|
| | --- |
| |
|
| | ## Model Details |
| |
|
| | ### Model Description |
| |
|
| | - **Developed by:** [alusci](https://huggingface.co/alusci) |
| | - **Model type:** Transformer-based binary classifier |
| | - **Language(s):** English |
| | - **License:** Apache 2.0 |
| | - **Finetuned from model:** `distilbert-base-uncased` |
| |
|
| | ### Model Sources |
| |
|
| | - **Repository:** [https://huggingface.co/alusci/distilbert-smsafe](https://huggingface.co/alusci/distilbert-smsafe) |
| |
|
| | --- |
| |
|
| | ## 🛠️ Uses |
| |
|
| | ### Direct Use |
| |
|
| | - Detect whether an SMS message is spam or ham (OTP or not). |
| | - Useful in prototypes, educational settings, or lightweight filtering applications. |
| |
|
| | ```python |
| | from transformers import pipeline |
| | |
| | classifier = pipeline("text-classification", model="alusci/distilbert-smsafe") |
| | result = classifier("Your verification code is 123456. Please do not share it with anyone.") |
| | |
| | # Optional: map the label to human-readable terms |
| | label_map = {"LABEL_0": "ham", "LABEL_1": "spam"} |
| | print(f"Label: {label_map[result[0]['label']]} - Score: {result[0]['score']:.2f}") |
| | ``` |
| |
|
| | ### Out-of-Scope Use |
| |
|
| | - Not intended for email spam detection or multilingual message filtering. |
| | - Not suitable for production environments without further testing and evaluation. |
| |
|
| | --- |
| |
|
| | ## 🧪 Bias, Risks, and Limitations |
| |
|
| | - The model may reflect dataset biases (e.g., message structure, language patterns). |
| | - It may misclassify legitimate OTPs or non-standard spam content. |
| | - Risk of false positives in edge cases. |
| |
|
| | ### Recommendations |
| |
|
| | - Evaluate on your own SMS dataset before deployment. |
| | - Consider combining with rule-based or heuristic systems in production. |
| |
|
| | --- |
| |
|
| | ## 📚 Training Details |
| |
|
| | ### Training Data |
| |
|
| | - Dataset used: [`alusci/sms-otp-spam-dataset`](https://huggingface.co/datasets/alusci/sms-otp-spam-dataset) |
| | - Binary labels for spam and non-spam OTP messages |
| |
|
| | ### Training Procedure |
| |
|
| | - **Epochs:** 5 |
| | - **Batch Size:** 16 (assumed) |
| | - **Loss Function:** CrossEntropyLoss |
| | - **Optimizer:** AdamW |
| | - **Tokenizer:** `distilbert-base-uncased` |
| |
|
| | --- |
| |
|
| | ## 📈 Evaluation |
| |
|
| | ### Metrics |
| |
|
| | - Accuracy, Precision, Recall, F1-score on held-out validation set |
| | - Binary classification labels: |
| | - `LABEL_0` → ham |
| | - `LABEL_1` → spam |
| |
|
| | ### Results |
| |
|
| | **Evaluation metrics after 5 epochs:** |
| |
|
| | - **Loss:** 0.2962 |
| | - **Accuracy:** 91.35% |
| | - **Precision:** 90.26% |
| | - **Recall:** 100.00% |
| | - **F1-score:** 94.88% |
| |
|
| | **Performance:** |
| |
|
| | - **Evaluation runtime:** 4.37 seconds |
| | - **Samples/sec:** 457.27 |
| | - **Steps/sec:** 9.15 |
| |
|
| | --- |
| |
|
| | ## 🌱 Environmental Impact |
| |
|
| | - **Hardware Type:** Apple Silicon MPS GPU (Mac) |
| | - **Hours used:** <1 hour (small dataset) |
| | - **Cloud Provider:** None (trained locally) |
| | - **Carbon Emitted:** Minimal due to local and efficient hardware |
| |
|
| | --- |
| |
|
| | ## 🔧 Technical Specifications |
| |
|
| | ### Model Architecture and Objective |
| |
|
| | - **Base:** DistilBERT |
| | - **Objective:** Binary classification head on pooled output |
| | - **Parameters:** ~66M (same as distilbert) |
| |
|
| | --- |
| |
|
| | ## 📬 Model Card Contact |
| |
|
| | For questions or feedback, please contact via [Hugging Face profile](https://huggingface.co/alusci). |
