wangjazz/Hunyuan-MT1.5-7B-MNN

HY-MT1.5-7B-MNN

This is a 4-bit quantized MNN version of Tencent's HY-MT1.5-7B translation model, optimized for Apple Silicon (macOS/iOS) edge inference.

Model Description

HY-MT1.5-7B is an upgraded version of Hunyuan-MT-7B (WMT25 Champion), featuring:

• 36 Language Support: Extended language coverage
• Terminology Intervention: Custom terminology control during translation
• Context-Aware Translation: Improved accuracy with context understanding
• Format Preservation: Maintains original text formatting

Quantization Details

Property	Value
Original Model	tencent/HY-MT1.5-7B
Original Size	~16 GB
Quantized Size	4.47 GB
Compression Ratio	72%
Quantization Type	4-bit (q4_k_m)
Block Size	64

Hardware Acceleration

Optimized for Apple Silicon with:

• ✅ INT8 Dot Product (i8sdot)
• ✅ FP16 Operations
• ✅ INT8 Matrix Multiply (i8mm)
• ✅ Scalable Matrix Extension 2 (sme2)
• ✅ Metal GPU Acceleration

Files

├── llm.mnn              # Model structure (576 KB)
├── llm.mnn.weight       # Quantized weights (4.47 GB)
├── tokenizer.txt        # Tokenizer vocabulary
├── llm_config.json      # MNN runtime config
├── config.json          # Model config
├── model_info.json      # Model metadata
└── export_args.json     # Conversion parameters

Usage

With MNN LLM Demo

# Clone MNN and build llm_demo
git clone https://github.com/alibaba/MNN.git
cd MNN && mkdir build && cd build
cmake .. -DMNN_BUILD_LLM=ON -DMNN_LOW_MEMORY=ON
make -j8 llm_demo

# Run inference
cd /path/to/HY-MT1.5-7B-MNN
./llm_demo ./

Example

User: Translate into English: 今天天氣很好
A: The weather is very nice today.

Prompt Templates

# Basic translation
Translate into {language}:
{text}

# With terminology
Translate into {language}, using terms: {terms}
{text}

# With context
Context: {context}
Translate into {language}:
{text}

Performance

Metric	Value
Model Load Time	~3.5s
Inference Speed	20-30 tokens/s
Target Device	Apple Silicon Mac

Conversion Info

• Tool: MNN llmexport.py
• MNN Version: 3.0.0
• Conversion Date: 2025-12-31
• Source Format: HuggingFace safetensors

Related Models

• HY-MT1.5-1.8B-MNN - Smaller version for iOS devices
• Hunyuan-MT-7B-MNN - Original WMT25 version

License

This model inherits the license from the original HY-MT1.5-7B model.

Acknowledgments

• Tencent Hunyuan Team for the original model
• Alibaba MNN Team for the inference framework