---
tags:
- autogluon
- multimodal
- image-classification
- binary-classification
- ensemble-learning
- education
- homework
datasets:
- ecopus/sign_identification
license: mit
---

# HW1 Sign Identification with AutoGluon

## Model Description
This repository contains an **AutoML image classification model trained with AutoGluon Multimodal**  
to identify two categories of sign images.  
The model was trained as part of **Homework 1** in CMU 24-679 (Designing and Deploying AI/ML).  

- **Framework**: [AutoGluon Multimodal](https://auto.gluon.ai/stable/tutorials/multimodal/index.html)  
- **Backbone**: TimmAutoModelForImagePrediction (~194M parameters)  
- **Task**: Binary image classification (`label`)  
- **Classes**: `0`, `1`  

---

## Training Setup
- **Dataset**: [ecopus/sign_identification](https://huggingface.co/datasets/ecopus/sign_identification)  
  - `augmented` split (385 samples) → used for training/validation (80/20).  
  - `original` split (35 samples) → reserved for final testing.  
- **Time budget**: 300 seconds (≈7 minutes).  
- **Hardware**: Colab GPU (CUDA 12.6, mixed precision).  
- **Presets**: `best_quality` (ensembling + hyperparameter tuning).  

---

## Results
- **Validation ROC-AUC**: 0.998  
- **Test Accuracy**: 97.1%  
- **Weighted F1**: 97.1%  

### Classification Report (Test Set)


          precision    recall  f1-score   support
       0       0.95      1.00      0.97        19
       1       1.00      0.94      0.97        16
accuracy                           0.97        35


macro avg 0.97 0.97 0.97 35
weighted avg 0.97 0.97 0.97 35

---

## How to Use

### Install requirements
```bash
pip install autogluon.multimodal huggingface_hub cloudpickle

import cloudpickle
from huggingface_hub import hf_hub_download

pkl_path = hf_hub_download(
    repo_id="cassieli226/sign-identification-automl",
    filename="autogluon_predictor.pkl",
    repo_type="model"
)

with open(pkl_path, "rb") as f:
    predictor = cloudpickle.load(f)

# Predict on new data (expects DataFrame with 'image' column containing file paths)
import pandas as pd
X_test = pd.DataFrame({"image": ["path/to/your/image.png"]})
print(predictor.predict(X_test))


import pathlib, shutil, zipfile
from huggingface_hub import hf_hub_download
import autogluon.multimodal as ag
import pandas as pd

zip_path = hf_hub_download(
    repo_id="cassieli226/sign-identification-automl",
    filename="autogluon_predictor_dir.zip",
    repo_type="model"
)

extract_dir = pathlib.Path("predictor_dir")
if extract_dir.exists():
    shutil.rmtree(extract_dir)
with zipfile.ZipFile(zip_path, "r") as zf:
    zf.extractall(str(extract_dir))

predictor = ag.MultiModalPredictor.load(str(extract_dir))
print(predictor.predict(pd.DataFrame({"image": ["path/to/your/image.png"]})))

---

#Intended Use
 - Coursework demonstration of AutoML for neural networks on images.
 - Educational example for using augmented vs. original splits for training and evaluation.
#Limitations
 - Trained on a small student-collected dataset (≈420 images).
 - Accuracy may not generalize to unseen real-world data.
 - Model assumes binary labels only (0, 1).
#Ethical Notes
 - Dataset is non-sensitive, contains no personal information.
 - Augmentation was applied responsibly to avoid unrealistic samples.
# References
 - Dataset: ecopus/sign_identification
 - Framework: AutoGluon
 - OpenAI’s ChatGPT (2025) was used for code generation, structuring, and debugging.