File size: 3,470 Bytes
f5e2104 dd66b05 f5e2104 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 |
---
tags:
- autogluon
- multimodal
- image-classification
- binary-classification
- ensemble-learning
- education
- homework
datasets:
- ecopus/sign_identification
license: mit
---
# HW1 Sign Identification with AutoGluon
## Model Description
This repository contains an **AutoML image classification model trained with AutoGluon Multimodal**
to identify two categories of sign images.
The model was trained as part of **Homework 1** in CMU 24-679 (Designing and Deploying AI/ML).
- **Framework**: [AutoGluon Multimodal](https://auto.gluon.ai/stable/tutorials/multimodal/index.html)
- **Backbone**: TimmAutoModelForImagePrediction (~194M parameters)
- **Task**: Binary image classification (`label`)
- **Classes**: `0`, `1`
---
## Training Setup
- **Dataset**: [ecopus/sign_identification](https://huggingface.co/datasets/ecopus/sign_identification)
- `augmented` split (385 samples) → used for training/validation (80/20).
- `original` split (35 samples) → reserved for final testing.
- **Time budget**: 300 seconds (≈7 minutes).
- **Hardware**: Colab GPU (CUDA 12.6, mixed precision).
- **Presets**: `best_quality` (ensembling + hyperparameter tuning).
---
## Results
- **Validation ROC-AUC**: 0.998
- **Test Accuracy**: 97.1%
- **Weighted F1**: 97.1%
### Classification Report (Test Set)
precision recall f1-score support
0 0.95 1.00 0.97 19
1 1.00 0.94 0.97 16
accuracy 0.97 35
macro avg 0.97 0.97 0.97 35
weighted avg 0.97 0.97 0.97 35
---
## How to Use
### Install requirements
```bash
pip install autogluon.multimodal huggingface_hub cloudpickle
import cloudpickle
from huggingface_hub import hf_hub_download
pkl_path = hf_hub_download(
repo_id="cassieli226/sign-identification-automl",
filename="autogluon_predictor.pkl",
repo_type="model"
)
with open(pkl_path, "rb") as f:
predictor = cloudpickle.load(f)
# Predict on new data (expects DataFrame with 'image' column containing file paths)
import pandas as pd
X_test = pd.DataFrame({"image": ["path/to/your/image.png"]})
print(predictor.predict(X_test))
import pathlib, shutil, zipfile
from huggingface_hub import hf_hub_download
import autogluon.multimodal as ag
import pandas as pd
zip_path = hf_hub_download(
repo_id="cassieli226/sign-identification-automl",
filename="autogluon_predictor_dir.zip",
repo_type="model"
)
extract_dir = pathlib.Path("predictor_dir")
if extract_dir.exists():
shutil.rmtree(extract_dir)
with zipfile.ZipFile(zip_path, "r") as zf:
zf.extractall(str(extract_dir))
predictor = ag.MultiModalPredictor.load(str(extract_dir))
print(predictor.predict(pd.DataFrame({"image": ["path/to/your/image.png"]})))
---
#Intended Use
- Coursework demonstration of AutoML for neural networks on images.
- Educational example for using augmented vs. original splits for training and evaluation.
#Limitations
- Trained on a small student-collected dataset (≈420 images).
- Accuracy may not generalize to unseen real-world data.
- Model assumes binary labels only (0, 1).
#Ethical Notes
- Dataset is non-sensitive, contains no personal information.
- Augmentation was applied responsibly to avoid unrealistic samples.
# References
- Dataset: ecopus/sign_identification
- Framework: AutoGluon
- OpenAI’s ChatGPT (2025) was used for code generation, structuring, and debugging.
|