learn-abc/banking-multilingual-intent-classifier

---

Banking Multilingual Intent Classifier

• Model Name: Banking Multilingual Intent Classifier
• Base Model: google/muril-base-cased
• Task: Multilingual Intent Classification
• Intents: 14
• Languages: English, Bangla (Bengali script), Banglish (Romanized Bengali), Code-Mixed

---

1. Model Overview

This model is a multilingual intent classifier designed for production-grade banking chatbot systems. It supports English, Bangla (Bengali script), and Banglish, including limited code-mixed input.

The model classifies user queries into 14 banking-specific intents with strong fallback detection for out-of-domain queries.

---

2. Intended Use

Primary Use Case

• Banking virtual assistants
• Customer support chatbots
• Voice-to-intent pipelines
• Multilingual conversational banking systems

Supported Capabilities

• Transaction queries
• Balance inquiries
• Card management
• Lost/stolen reporting
• Fee clarification
• ATM issues
• Account updates
• General banking information
• Robust fallback detection for non-banking queries

---

3. Dataset Summary

Total Samples

110,364 original samples

• 500 additional code-mixed samples
• training augmentation

Final training size:

• Train: 99,273
• Test: 22,173

Language Distribution

Language	Count
English	36,788
Bangla	36,788
Banglish	36,788
Code-Mixed	~0.45%

Balanced across main three languages.

---

4. Intent Classes

Total Intents: 14

ACCOUNT_INFO
ATM_SUPPORT
CARD_ISSUE
CARD_MANAGEMENT
CARD_REPLACEMENT
CHECK_BALANCE
EDIT_PERSONAL_DETAILS
FAILED_TRANSFER
FALLBACK
FEES
GREETING
LOST_OR_STOLEN_CARD
MINI_STATEMENT
TRANSFER

---

5. Data Characteristics

• Stratified 80/20 split

• Balanced language distribution

• Weighted loss for class imbalance

• Lowercase augmentation applied

• Hard negative examples included for:

  • General knowledge
  • Math queries
  • Stock/crypto
  • Biography queries
  • Metaphorical financial language
  • Government and legal topics

FALLBACK class strengthened for production safety.

---

6. Evaluation Results

Overall Performance (Test Set: 22,173 samples)

• Accuracy: 98.36%
• F1 Micro: 98.36%
• F1 Macro: 98.21%

Accuracy by Intent

Intent	Accuracy
ACCOUNT_INFO	99.27%
ATM_SUPPORT	99.08%
CARD_ISSUE	99.15%
CARD_MANAGEMENT	98.70%
CARD_REPLACEMENT	99.55%
CHECK_BALANCE	97.77%
EDIT_PERSONAL_DETAILS	99.66%
FAILED_TRANSFER	98.62%
FALLBACK	97.04%
FEES	99.58%
GREETING	95.02%
LOST_OR_STOLEN_CARD	98.43%
MINI_STATEMENT	98.56%
TRANSFER	99.25%

---

7. Example Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_name = "learn-abc/banking-multilingual-intent-classifier"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
model.eval()

# Prediction function
def predict_intent(text):
    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)
    inputs = {k: v.to(device) for k, v in inputs.items()}
    
    with torch.no_grad():
        outputs = model(**inputs)
        prediction = torch.argmax(outputs.logits, dim=-1).item()
        confidence = torch.softmax(outputs.logits, dim=-1)[0][prediction].item()
    
    predicted_intent = model.config.id2label[prediction]
    
    return {
        "intent": predicted_intent,
        "confidence": confidence
    }

# Example usage - English
result = predict_intent("what is my balance")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: CHECK_BALANCE, Confidence: 0.99

# Example usage - Bangla
result = predict_intent("আমার ব্যালেন্স কত")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: CHECK_BALANCE, Confidence: 0.98

# Example usage - Banglish (Romanized)
result = predict_intent("amar balance koto ache")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: CHECK_BALANCE, Confidence: 0.97

# Example usage - Code-mixed
result = predict_intent("আমার last 10 transaction দেখাও")
print(f"Intent: {result['intent']}, Confidence: {result['confidence']:.2f}")
# Output: Intent: MINI_STATEMENT, Confidence: 0.98

---

8. Strengths

• Strong multilingual generalization
• High performance on transactional intents
• Robust fallback detection for out-of-domain queries
• Resistant to keyword leakage
• Stable performance across English, Bangla, Banglish
• Class imbalance handled using weighted loss
• Production-safe fallback tuning

---

9. Known Limitations

• Very short ambiguous inputs may drift between:

  • GREETING
  • FALLBACK

• Highly ambiguous informational queries may overlap between:

  • MINI_STATEMENT
  • ACCOUNT_INFO

• Code-mixed coverage is limited compared to core languages

• Model not optimized for long multi-turn conversational memory

---

10. Safety & Risk Considerations

• Model prioritizes safe fallback over risky misclassification.
• Non-banking queries are correctly routed to FALLBACK.
• Reduces risk of executing unintended financial actions.

Recommended Production Safeguards:

• Confidence threshold filtering
• Human fallback escalation for low-confidence cases
• Logging for monitoring drift

---

11. Inference Performance

• Evaluation throughput: ~950 samples/sec
• GPU inference optimized
• Suitable for real-time chatbot systems

---

12. Version

• Version: 6.0
• Status: Production-ready with monitoring
• Last Evaluated: Epoch 5

---

13. Suggested Deployment Architecture

Recommended stack:

User Input

• → Language detection (optional)
• → Intent classifier (this model)
• → Confidence threshold
• → Business logic router
• → Response generator

---

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contact Me

For any inquiries or support, please reach out to:

• Author: Abhishek Singh
• LinkedIn: My LinkedIn Profile
• Portfolio: Abhishek Singh Portfolio

---