LexCAT: Lexicon-Enhanced Code-Switched Attention Transformer for Tagalog–English Sentiment Analysis

Author: Glenn Marcus D. Cinco
Institution: Mapúa University
Thesis Title: LexCAT: A Lexicon-Based Approach for Code-Switching Analysis with Transformers Using XLM-RoBERTa and LexiLiksik
Degree: BS & MS in Computer Science
Date: July 2025

---

🎓 Abstract

LexCAT is a fine-tuned XLM-RoBERTa model enhanced with LexiLiksik, a cross-lingual, context-aware Tagalog–English sentiment lexicon developed to handle intra-sentential sentiment shifts in code-switched Taglish text. LexCAT achieves 84.31% validation accuracy on the FiReCS dataset, significantly outperforming monolingual baselines (FilCon: 73.12%, SentiWordNet: 69.52%).

This model is the final output of a thesis that systematically addresses gaps in code-switched sentiment analysis through lexicon development, attention biasing, and real-world fine-tuning.

---

🔧 Model Architecture

• Base Model: xlm-roberta-base
• Enhancement: Integrated LexiLiksik via:
  • Soft constraints during fine-tuning
  • Attention weight adjustment for sentiment-relevant tokens
  • Metadata-aware pooling (POS, code-switching type)
• Fine-tuning Dataset: FiReCS (10,487 Filipino–English code-switched reviews)
• Evaluation Dataset: SentiTaglish Products and Services + FiReCS Test Set

---

📊 Performance

Metric	Value
Accuracy	84.31%
F1-Score	0.8566
Precision	0.8353
Recall	0.8574
Cohen’s κ	0.83 (from FiReCS annotators)

✅ Key Strength: Correctly classifies contrastive phrases like “Maganda pero expensive” as negative.

---

📥 How to Use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "your-hf-username/LexCAT-LexiLiksik-Final"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "sobrang lambot ng burger pero expensive tlga"
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)
outputs = model(**inputs)
predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
predicted_class = torch.argmax(predictions, dim=1).item()

# Class: 0 = Negative, 1 = Neutral, 2 = Positive
sentiment = ["Negative", "Neutral", "Positive"][predicted_class]
print(f"Predicted Sentiment: {sentiment}")

---

📚 Citation

@mastersthesis{cinco2025lexcat,
  author = {Cinco, Glenn Marcus D.},
  title = {LexCAT: A Lexicon-Based Approach for Code-Switching Analysis with Transformers Using XLM-RoBERTa and LexiLiksik},
  school = {Mapúa University},
  year = {2025},
  month = {July}
}