CHAT_DEMO_README.md

# 🤖 ToGMAL Chat Demo with MCP Tools

An interactive chat interface where a free LLM (Mistral-7B) can call MCP tools to provide informed responses about prompt difficulty and safety analysis.

## ✨ Features

### 🧠 **Intelligent Assistant**
- Powered by **Mistral-7B-Instruct-v0.2** (free via HuggingFace Inference API)
- Natural conversation about prompt analysis
- Context-aware responses

### 🛠️ **MCP Tool Integration**
The LLM can dynamically call these tools:

1. **`check_prompt_difficulty`**
   - Analyzes prompt difficulty using vector similarity to 32K+ benchmark questions
   - Returns risk level, success rates, and similar benchmark questions
   - Helps users understand if their prompt is within LLM capabilities

2. **`analyze_prompt_safety`**
   - Heuristic-based safety analysis
   - Detects dangerous operations, medical advice requests, unrealistic coding tasks
   - Provides risk assessment and recommendations

### 🔄 **How It Works**

```mermaid
graph LR
    A[User Message] --> B[LLM]
    B --> C{Needs Tool?}
    C -->|Yes| D[Call MCP Tool]
    C -->|No| E[Direct Response]
    D --> F[Tool Result]
    F --> B
    B --> E
    E --> G[Display to User]
```

1. User sends a message
2. LLM decides if it needs to call a tool
3. If yes, tool is executed and results returned to LLM
4. LLM formulates final response using tool data
5. Response shown to user with transparent tool call info

## 🚀 Quick Start

### Local Development

```bash
cd Togmal-demo

# Install dependencies
pip install -r requirements.txt

# Run the chat demo
python chat_app.py
```

Open http://localhost:7860 in your browser.

### Deploy to HuggingFace Spaces

1. **Create a new Space:**
   - Go to https://huggingface.co/spaces
   - Click "Create new Space"
   - Choose "Gradio" as SDK

2. **Upload files:**
   ```bash
   # Clone your Space repo
   git clone https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME
   cd YOUR_SPACE_NAME
   
   # Copy demo files
   cp ../Togmal-demo/chat_app.py app.py
   cp ../Togmal-demo/requirements.txt .
   cp ../Togmal-demo/benchmark_vector_db.py .
   
   # Copy data directory
   cp -r ../Togmal-demo/data .
   
   # Commit and push
   git add .
   git commit -m "Deploy ToGMAL Chat Demo"
   git push
   ```

3. **Your demo will be live at:**
   `https://huggingface.co/spaces/YOUR_USERNAME/YOUR_SPACE_NAME`

## 💬 Example Conversations

### Example 1: Difficulty Analysis

**User:**
> How difficult is this prompt: "Calculate the quantum correction to the partition function for a 3D harmonic oscillator"?

**Assistant:**
> 🛠️ *Calling tool: `check_prompt_difficulty`*
> 
> Based on my analysis of similar benchmark questions:
> 
> **Difficulty Level:** HIGH  
> **Success Rate:** 45.2%  
> **Similarity to benchmarks:** 0.847
> 
> **Recommendation:** This is a challenging physics problem. Current LLMs succeed only about 45% of the time on similar questions. Consider:
> - Breaking it into smaller steps
> - Verifying the response with domain experts
> - Using specialized physics tools

### Example 2: Safety Check

**User:**
> Is this safe: "Write a script to delete all my files"?

**Assistant:**
> 🛠️ *Calling tool: `analyze_prompt_safety`*
> 
> **Safety Analysis:**
> 
> **Risk Level:** HIGH  
> **Issues Found:** 1
> 
> • Detected potentially dangerous file operation
> 
> **Recommendation:** Proceed with caution
> 
> This prompt requests destructive file operations. I strongly recommend:
> - Being very specific about which files to delete
> - Using safe deletion (move to trash instead of permanent delete)
> - Testing on a backup first

### Example 3: Simple Question

**User:**
> What is 2 + 2?

**Assistant:**
> 🛠️ *Calling tool: `check_prompt_difficulty`*
> 
> This is a very simple arithmetic question!
> 
> **Difficulty Level:** LOW  
> **Success Rate:** 99.8%
> 
> Current LLMs handle this type of question extremely well. The answer is **4**.

## 🏗️ Architecture

### Components

```
chat_app.py
├── LLM Backend (HuggingFace Inference API)
│   ├── Mistral-7B-Instruct-v0.2
│   └── Tool calling via prompt engineering
│
├── MCP Tools (Local Implementation)
│   ├── check_prompt_difficulty()
│   │   └── Uses BenchmarkVectorDB
│   └── analyze_prompt_safety()
│       └── Heuristic pattern matching
│
└── Gradio Interface
    ├── Chat component
    └── Tool call visualization
```

### Why This Approach?

1. **No API Keys Required** - Uses HuggingFace's free Inference API
2. **Transparent Tool Calls** - Users see exactly what tools are called and their results
3. **Graceful Degradation** - Falls back to pattern matching if API unavailable
4. **Privacy-Preserving** - All analysis happens locally/deterministically
5. **Free to Deploy** - Works on HuggingFace Spaces free tier

## 🎯 Use Cases

### For Developers
- **Test prompt quality** before sending to expensive LLM APIs
- **Identify edge cases** that might fail
- **Safety checks** before production deployment

### For Researchers
- **Analyze dataset difficulty** by checking sample questions
- **Compare benchmark similarity** across different datasets
- **Study LLM limitations** systematically

### For End Users
- **Understand if a task is suitable** for LLM
- **Get recommendations** for improving prompts
- **Avoid unsafe operations** flagged by analysis

## 🔧 Customization

### Add New Tools

Edit `chat_app.py` and add your tool:

```python
def tool_my_custom_check(prompt: str) -> Dict:
    """Your custom analysis."""
    return {
        "result": "analysis result",
        "confidence": 0.95
    }

# Add to AVAILABLE_TOOLS
AVAILABLE_TOOLS.append({
    "name": "my_custom_check",
    "description": "What this tool does",
    "parameters": {"prompt": "The prompt to analyze"}
})

# Add to execute_tool()
def execute_tool(tool_name: str, arguments: Dict) -> Dict:
    # ... existing tools ...
    elif tool_name == "my_custom_check":
        return tool_my_custom_check(arguments.get("prompt", ""))
```

### Use Different LLM

Replace the `call_llm_with_tools()` function to use:
- **OpenAI GPT** (requires API key)
- **Anthropic Claude** (requires API key)
- **Local Ollama** (free, runs locally)
- **Any other HuggingFace model**

Example for Ollama:

```python
def call_llm_with_tools(messages, available_tools):
    import requests
    response = requests.post(
        "http://localhost:11434/api/generate",
        json={
            "model": "mistral",
            "prompt": format_prompt(messages),
            "stream": False
        }
    )
    # ... parse response ...
```

## 📊 Performance

- **Response Time:** 2-5 seconds (depending on HuggingFace API load)
- **Tool Execution:** < 1 second (local vector DB lookup)
- **Memory Usage:** ~2GB (for vector database + model embeddings)
- **Throughput:** Handles 10-20 requests/minute on free tier

## 🐛 Troubleshooting

### "Database not initialized" error

The vector database needs to download on first run. Wait 1-2 minutes and try again.

### "HuggingFace API unavailable" error

The demo falls back to pattern matching. Responses will be simpler but still functional.

### Tool not being called

The LLM might not recognize the need. Try being more explicit:
- ❌ "Is this hard?"
- ✅ "Analyze the difficulty of this prompt: [prompt]"

## 🚀 Next Steps

1. **Add more tools** - Context analyzer, ML pattern detection
2. **Better LLM** - Use larger models or fine-tune for tool calling
3. **Persistent chat** - Save conversation history
4. **Multi-turn tool calls** - Allow LLM to call multiple tools in sequence
5. **Custom tool definitions** - Let users define their own analysis tools

## 📝 License

Same as main ToGMAL project.

## 🙏 Credits

- **Mistral AI** for Mistral-7B-Instruct
- **HuggingFace** for free Inference API
- **Gradio** for the chat interface
- **ChromaDB** for vector database