chrissoria/catllm-json-formatter

CatLLM JSON Formatter

A fine-tuned Qwen2.5-0.5B-Instruct model that converts messy LLM classification output into valid cat-llm JSON format.

Task

Given a list of numbered categories and raw (possibly malformed) classification output from another LLM, this model produces clean JSON:

{"1": "0", "2": "1", "3": "0", ...}

Usage

This model is used automatically by cat-llm when json_formatter=True:

import catllm as cat

results = cat.classify(
    input_data=df["responses"],
    categories=["Positive", "Negative", "Neutral"],
    api_key="your-key",
    json_formatter=True,  # enables the formatter fallback
)

Install the formatter dependencies: pip install cat-llm[formatter]

Training

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Method: LoRA (r=16, alpha=32) merged into base weights
• Training data: 4,000 synthetic examples covering 26+ messy output formats
• Epochs: 3
• Metrics: 100% parse success, 98% exact match on held-out test set

Prompt Format

The model uses the Qwen chat template with:

System: JSON formatter instructions (built into cat-llm)

User:

Categories:
1. Category A
2. Category B
...

Raw classification output:
{messy output here}

Assistant: {"1":"0","2":"1",...}