README.md

metadata

title: SAM3D
emoji: 😻
colorFrom: pink
colorTo: blue
sdk: gradio
sdk_version: 6.0.1
app_file: app.py
pinned: false
license: mit
short_description: Creating and editing 3d models using text prompts
tags:
  - text-to-3d
  - mcp
  - gemini
  - sam-3d
  - building-mcp-track-multimodal

🎨 Text to 3D – MCP Server

Generate and edit 3D models from text prompts using Gemini + Meta SAM-3D.

🎬 Demo: [YouTube Link]
📱 Post: [X/LinkedIn Link]

💡 Use Cases

This tool enables rapid 3D asset creation through natural language:

• Game Developers – Quickly prototype 3D assets by describing them ("a medieval sword", "a sci-fi helmet")
• Designers & Artists – Generate concept models and iterate with text edits ("make it larger", "change color to gold")
• E-commerce – Create product mockups from descriptions for visualization
• Education – Generate 3D models for teaching (anatomy, architecture, objects)
• AI Agents – Claude/Cursor can autonomously create 3D assets as part of larger workflows via MCP

🚀 Future Scope

• Multi-object scene generation
• Style transfer for 3D models
• Animation support
• Direct 3D file editing without regeneration
• Integration with game engines (Unity/Unreal)

🏗️ Architecture

┌─────────────────┐      MCP Protocol      ┌─────────────────────┐
│  Claude Desktop │ ◄────────────────────► │   Gradio MCP Server │
│  Cursor / Cline │                        │   (HF Space)        │
└─────────────────┘                        └──────────┬──────────┘
                                                      │
                        ┌─────────────────────────────┼─────────────────────────────┐
                        ▼                             ▼                             ▼
                 ┌─────────────┐              ┌─────────────┐              ┌─────────────┐
                 │   Gemini    │              │   Gemini    │              │   SAM-3D    │
                 │  Generate   │──────────────│    Edit     │──────────────│   (Modal)   │
                 │   Image     │              │  / Remove   │              │  3D Recon   │
                 └─────────────┘              └─────────────┘              └─────────────┘

Flow: Text → Image → Remove BG → Grayscale Mask → SAM-3D → 3D Model (GLB/PLY)

🔌 MCP Tools

Tool	Description
generate_3d	Text prompt → 3D model
edit_3d	Edit existing model with text

Connect from Claude Desktop

{
  "mcpServers": {
    "text-to-3d": {
      "command": "npx",
      "args": ["mcp-remote", "https://MCP-1st-Birthday-text-to-3d-SAM3D.hf.space/gradio_api/mcp/sse"]
    }
  }
}

🛠️ Tech Stack

Component	Purpose
Gemini 2.5 Flash	Image generation
Gemini 3 Pro	Background removal & editing
SAM-3D (Meta)	3D reconstruction
Modal	Serverless GPU
Gradio	UI + MCP server

📦 Outputs

• GLB – 3D mesh
• PLY – Gaussian splat

---

Built for MCP 1st Birthday Hackathon 🎂