Deploy Neuro-Flyt 3D Training

.gitignore

@@ -0,0 +1,58 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+
+# Virtual Environment
+venv/
+ENV/
+.venv
+# Note: env/ is our project folder, not a virtual environment
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# PyTorch
+*.pth
+*.pt
+
+# Model checkpoints and logs
+models/*.pkl
+models/*.h5
+models/*.ckpt
+logs/
+runs/
+wandb/
+
+# Environment variables
+.env
+.env.local
+

Dockerfile

@@ -0,0 +1,22 @@
+FROM pytorch/pytorch:2.0.1-cuda11.7-cudnn8-runtime
+
+# Install system dependencies for PyBullet/OpenGL
+RUN apt-get update && apt-get install -y \
+    libgl1-mesa-glx \
+    libglib2.0-0 \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+# Copy requirements and install
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
+RUN pip install huggingface_hub
+
+# Copy project files
+COPY . .
+
+# Default command (can be overridden in Space settings)
+# Expects HF_TOKEN and REPO_ID env vars to be set in the Space
+CMD ["python", "train_hf.py", "--repo_id", "ylop/neuro-flyt-3d", "--steps", "500000"]

README.md

@@ -1,10 +1,34 @@
----
-title: Neuro Flyt Training
-emoji: 👁
-colorFrom: blue
-colorTo: pink
-sdk: docker
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Project Neuro-Flyt 3D
+
+**Goal:** Build a verifiable 3D Drone Control verification demo using Liquid Neural Networks.
+
+## Installation
+
+1.  **Clone the repository** (if you haven't already).
+2.  **Install dependencies:**
+    ```bash
+    pip install -r requirements.txt
+    ```
+    *Note: You may need to install `opensimplex` and `ncps` manually if they are not in `requirements.txt` yet.*
+    ```bash
+    pip install opensimplex ncps
+    ```
+
+## Usage
+
+### Run the Demo
+To launch the 3D visualization with the "Antigravity" hurricane effect:
+```bash
+./run_demo.sh
+```
+
+### Verify Physics
+To verify that the wind field is generating non-zero forces:
+```bash
+python test_physics.py
+```
+
+## Project Structure
+-   `env/drone_3d.py`: Custom PyFlyt environment with 3D Perlin noise wind field.
+-   `models/liquid_ppo.py`: PPO agent with LTC (Liquid Time-Constant) feature extractor.
+-   `demo_3d.py`: Main visualization script.

README_HF.md

@@ -0,0 +1,79 @@
+# Deploying Neuro-Flyt 3D to Hugging Face Spaces
+
+This guide explains how to use your organization's GPUs on Hugging Face to train the Neuro-Flyt 3D model.
+
+## Prerequisites
+1.  A Hugging Face Account.
+2.  An Organization with GPU billing enabled (or a personal account with GPU access).
+3.  A Write Access Token (Settings -> Access Tokens).
+
+## Steps
+
+### 1. Create a New Space
+1.  Go to [huggingface.co/new-space](https://huggingface.co/new-space).
+2.  **Owner:** Select your Organization.
+3.  **Space Name:** `neuro-flyt-training` (or similar).
+4.  **SDK:** Select **Docker**.
+5.  **Space Hardware:** Select a GPU instance (e.g., **T4 small** or **A10G**).
+
+### 2. Configure Secrets
+In the Space settings, go to **Settings -> Variables and secrets**.
+Add the following **Secret**:
+-   `HF_TOKEN`: Your Write Access Token (starts with `hf_...`).
+
+### 3. Deploy Code
+You can deploy by pushing the code to the Space's Git repository.
+
+```bash
+# 1. Install git-lfs if needed
+git lfs install
+
+# 2. Clone your Space (replace with your actual repo URL)
+git clone https://huggingface.co/spaces/YOUR_ORG/neuro-flyt-training
+cd neuro-flyt-training
+
+# 3. Copy project files
+cp -r /path/to/Drone-go-brrrrr/* .
+
+# 4. Push to Space
+git add .
+git commit -m "Deploy training job"
+git push
+```
+
+### 4. Monitor Training
+-   Go to the **App** tab in your Space.
+-   You will see the training logs in real-time.
+-   The training will run for 500,000 steps.
+
+### 5. Access Trained Model
+-   Once finished, the script will automatically push the trained model (`liquid_ppo_drone_final.zip`) to your Model Repository (defined in `train_hf.py` or via arguments).
+-   You can then download this model and use it locally with `demo_3d.py`.
+
+## Customization
+-   **Repo ID:** Edit `Dockerfile` or `train_hf.py` to change the target Model Repository ID (`--repo_id`).
+-   **Steps:** Change `--steps` in `Dockerfile` to adjust training duration.
+
+## Hardware & Training Recommendations
+
+### Which GPU?
+*   **A100 Large (80GB):** **The Ultimate Choice.** If you want to train for 5M+ episodes in the shortest time possible, pick this. We have optimized the code to use **16 Parallel Environments** and **Large Batch Sizes (4096)** to fully saturate the A100.
+*   **A10G Large (24GB):** **Excellent Value.** Very fast and capable. It will handle the parallel training easily and is much cheaper than the A100.
+*   **T4 (16GB):** **Budget Option.** It will work, but you won't see the massive speedup from the parallelization as clearly as with the Ampere cards (A10/A100).
+
+### Efficiency Optimization (Implemented)
+To ensure the GPU doesn't sit idle, we have updated `train_hf.py` to:
+1.  **Parallel Physics:** Run **16 Drones** simultaneously on the CPU.
+2.  **Large Batches:** Process **4096 samples** at once on the GPU.
+3.  **Result:** Training is ~10-15x faster than the standard script.
+
+### How Many Episodes?
+The environment `max_steps` is 1000.
+*   **Minimum (Proof of Concept):** **500,000 Steps** (500 Episodes). The drone will learn to hover and roughly follow the target.
+*   **Recommended (Robust):** **1,000,000 - 2,000,000 Steps** (1000 - 2000 Episodes). This allows the Liquid Network to fully adapt to the random wind turbulence and master the physics.
+*   **High Performance:** **5,000,000+ Steps**. For "perfect" flight control.
+
+### Efficiency Tip
+Reinforcement Learning is often CPU-bound (physics simulation). To train efficiently:
+1.  Use a Space with **many CPU vCores** (8+) to run environments in parallel.
+2.  Use the **A10G** GPU to handle the heavy math of the Liquid Time-Constant (LTC) cells.

debug_imports.py

@@ -0,0 +1,31 @@
+import sys
+print(f"Python version: {sys.version}")
+try:
+    import numpy
+    print("Numpy imported")
+except ImportError as e:
+    print(f"Numpy failed: {e}")
+
+try:
+    import gymnasium
+    print("Gymnasium imported")
+except ImportError as e:
+    print(f"Gymnasium failed: {e}")
+
+try:
+    import PyFlyt
+    print("PyFlyt imported")
+except ImportError as e:
+    print(f"PyFlyt failed: {e}")
+
+try:
+    import opensimplex
+    print("Opensimplex imported")
+except ImportError as e:
+    print(f"Opensimplex failed: {e}")
+
+try:
+    import ncps
+    print("Ncps imported")
+except ImportError as e:
+    print(f"Ncps failed: {e}")

demo/__init__.py

@@ -0,0 +1,2 @@
+"""Demo and visualization scripts."""
+

demo/visualize_drone.py

@@ -0,0 +1,425 @@
+"""
+Visual demonstration of the drone environment using Pygame.
+
+This script loads a trained model and visualizes the drone navigating
+through the environment with wind forces.
+"""
+
+import os
+import sys
+import pygame
+import numpy as np
+from typing import Optional
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from env.drone_env import DroneWindEnv
+from stable_baselines3 import PPO
+
+
+# Pygame constants
+WINDOW_WIDTH = 800
+WINDOW_HEIGHT = 600
+FPS = 30
+
+# Color definitions
+BLACK = (0, 0, 0)
+WHITE = (255, 255, 255)
+RED = (255, 0, 0)
+GREEN = (0, 255, 0)
+BLUE = (0, 0, 255)
+YELLOW = (255, 255, 0)
+CYAN = (0, 255, 255)
+MAGENTA = (255, 0, 255)
+GRAY = (128, 128, 128)
+DARK_GRAY = (64, 64, 64)
+ORANGE = (255, 165, 0)
+
+
+class DroneVisualizer:
+    """Pygame-based visualizer for the drone environment."""
+    
+    def __init__(self, env: DroneWindEnv, model: Optional[PPO] = None):
+        """
+        Initialize the visualizer.
+        
+        Args:
+            env: DroneWindEnv instance
+            model: Optional trained PPO model (if None, uses random actions)
+        """
+        self.env = env
+        self.model = model
+        
+        # Initialize Pygame
+        pygame.init()
+        self.screen = pygame.display.set_mode((WINDOW_WIDTH, WINDOW_HEIGHT))
+        pygame.display.set_caption("Drone RL - Visual Demonstration")
+        self.clock = pygame.time.Clock()
+        self.font = pygame.font.Font(None, 24)
+        self.small_font = pygame.font.Font(None, 18)
+        
+        # World to screen scaling
+        # Environment is [0, 1] x [0, 1], we'll use most of the screen
+        self.world_margin = 50
+        self.world_width = WINDOW_WIDTH - 2 * self.world_margin
+        self.world_height = WINDOW_HEIGHT - 2 * self.world_margin
+        
+    def world_to_screen(self, x: float, y: float) -> tuple[int, int]:
+        """Convert world coordinates [0,1] to screen coordinates."""
+        screen_x = int(self.world_margin + x * self.world_width)
+        # Flip y-axis (world y=0 is bottom, screen y=0 is top)
+        screen_y = int(WINDOW_HEIGHT - self.world_margin - y * self.world_height)
+        return screen_x, screen_y
+    
+    def draw_drone(self, x: float, y: float, vx: float, vy: float):
+        """Draw the drone as a circle with velocity vector."""
+        screen_x, screen_y = self.world_to_screen(x, y)
+        
+        # Draw drone body (circle)
+        drone_radius = 15
+        pygame.draw.circle(self.screen, CYAN, (screen_x, screen_y), drone_radius)
+        pygame.draw.circle(self.screen, BLUE, (screen_x, screen_y), drone_radius, 2)
+        
+        # Draw velocity vector
+        if abs(vx) > 0.01 or abs(vy) > 0.01:
+            # Scale velocity for visualization
+            scale = 30
+            end_x = screen_x + int(vx * scale)
+            end_y = screen_y - int(vy * scale)  # Flip y for screen
+            pygame.draw.line(self.screen, YELLOW, (screen_x, screen_y), (end_x, end_y), 3)
+            # Draw arrowhead
+            if abs(vx) > 0.01 or abs(vy) > 0.01:
+                angle = np.arctan2(-vy, vx)  # Negative vy because screen y is flipped
+                arrow_size = 8
+                arrow_x1 = end_x - arrow_size * np.cos(angle - np.pi / 6)
+                arrow_y1 = end_y - arrow_size * np.sin(angle - np.pi / 6)
+                arrow_x2 = end_x - arrow_size * np.cos(angle + np.pi / 6)
+                arrow_y2 = end_y - arrow_size * np.sin(angle + np.pi / 6)
+                pygame.draw.line(self.screen, YELLOW, (end_x, end_y), (int(arrow_x1), int(arrow_y1)), 2)
+                pygame.draw.line(self.screen, YELLOW, (end_x, end_y), (int(arrow_x2), int(arrow_y2)), 2)
+    
+    def draw_wind(self, wind_x: float, wind_y: float):
+        """Draw wind arrows indicating direction."""
+        # Draw fewer, clearer wind arrows
+        grid_size = 6
+        for i in range(grid_size):
+            for j in range(grid_size):
+                x = (i + 0.5) / grid_size
+                y = (j + 0.5) / grid_size
+                screen_x, screen_y = self.world_to_screen(x, y)
+                
+                # Draw wind arrow
+                if abs(wind_x) > 0.01 or abs(wind_y) > 0.01:
+                    scale = 25
+                    end_x = screen_x + int(wind_x * scale)
+                    end_y = screen_y - int(wind_y * scale)  # Flip y
+                    
+                    # Color based on wind strength
+                    wind_strength = abs(wind_x) + abs(wind_y)
+                    if wind_strength < 1.0:
+                        color = GREEN
+                    elif wind_strength < 1.5:
+                        color = YELLOW
+                    else:
+                        color = ORANGE
+                    
+                    # Draw arrow line
+                    pygame.draw.line(self.screen, color, (screen_x, screen_y), (end_x, end_y), 3)
+                    
+                    # Draw arrowhead
+                    if abs(wind_x) > 0.01 or abs(wind_y) > 0.01:
+                        angle = np.arctan2(-wind_y, wind_x)  # Negative y because screen y is flipped
+                        arrow_size = 10
+                        arrow_x1 = end_x - arrow_size * np.cos(angle - np.pi / 6)
+                        arrow_y1 = end_y - arrow_size * np.sin(angle - np.pi / 6)
+                        arrow_x2 = end_x - arrow_size * np.cos(angle + np.pi / 6)
+                        arrow_y2 = end_y - arrow_size * np.sin(angle + np.pi / 6)
+                        pygame.draw.polygon(self.screen, color, [
+                            (end_x, end_y),
+                            (int(arrow_x1), int(arrow_y1)),
+                            (int(arrow_x2), int(arrow_y2))
+                        ])
+    
+    def draw_boundaries(self):
+        """Draw the world boundaries."""
+        # Top boundary
+        top_left = self.world_to_screen(0, 1)
+        top_right = self.world_to_screen(1, 1)
+        pygame.draw.line(self.screen, RED, top_left, top_right, 3)
+        
+        # Bottom boundary
+        bot_left = self.world_to_screen(0, 0)
+        bot_right = self.world_to_screen(1, 0)
+        pygame.draw.line(self.screen, RED, bot_left, bot_right, 3)
+        
+        # Left boundary
+        pygame.draw.line(self.screen, RED, top_left, bot_left, 3)
+        
+        # Right boundary
+        pygame.draw.line(self.screen, RED, top_right, bot_right, 3)
+    
+    def draw_target_zone(self, target_spawned: bool = True):
+        """Draw the target zone (box) that the drone needs to reach."""
+        from env.drone_env import TARGET_X_MIN, TARGET_X_MAX, TARGET_Y_MIN, TARGET_Y_MAX, TARGET_SPAWN_DELAY
+        
+        # Only draw if target has spawned
+        if not target_spawned:
+            return
+        
+        # Get screen coordinates for target zone corners
+        top_left = self.world_to_screen(TARGET_X_MIN, TARGET_Y_MAX)
+        top_right = self.world_to_screen(TARGET_X_MAX, TARGET_Y_MAX)
+        bot_left = self.world_to_screen(TARGET_X_MIN, TARGET_Y_MIN)
+        bot_right = self.world_to_screen(TARGET_X_MAX, TARGET_Y_MIN)
+        
+        # Draw target zone as a semi-transparent box
+        # Create a surface for transparency
+        target_surface = pygame.Surface((WINDOW_WIDTH, WINDOW_HEIGHT))
+        target_surface.set_alpha(100)  # Semi-transparent
+        
+        # Draw filled rectangle
+        rect = pygame.Rect(
+            top_left[0], top_left[1],
+            top_right[0] - top_left[0],
+            bot_left[1] - top_left[1]
+        )
+        pygame.draw.rect(target_surface, MAGENTA, rect)
+        self.screen.blit(target_surface, (0, 0))
+        
+        # Draw border
+        pygame.draw.line(self.screen, MAGENTA, top_left, top_right, 3)
+        pygame.draw.line(self.screen, MAGENTA, top_right, bot_right, 3)
+        pygame.draw.line(self.screen, MAGENTA, bot_right, bot_left, 3)
+        pygame.draw.line(self.screen, MAGENTA, bot_left, top_left, 3)
+        
+        # Draw label
+        label_x = (top_left[0] + top_right[0]) // 2
+        label_y = (top_left[1] + bot_left[1]) // 2
+        text = self.small_font.render("TARGET", True, WHITE)
+        text_rect = text.get_rect(center=(label_x, label_y))
+        self.screen.blit(text, text_rect)
+    
+    def draw_info(self, step: int, reward: float, action: Optional[int] = None, in_target: bool = False):
+        """Draw information text."""
+        y_offset = 10
+        
+        # Step count
+        text = self.font.render(f"Step: {step}", True, WHITE)
+        self.screen.blit(text, (10, y_offset))
+        y_offset += 30
+        
+        # Reward
+        text = self.font.render(f"Reward: {reward:.2f}", True, WHITE)
+        self.screen.blit(text, (10, y_offset))
+        y_offset += 30
+        
+        # In target zone status
+        target_color = GREEN if in_target else GRAY
+        target_text = "IN TARGET ZONE!" if in_target else "Not in target"
+        text = self.font.render(target_text, True, target_color)
+        self.screen.blit(text, (10, y_offset))
+        y_offset += 30
+        
+        # Position
+        text = self.small_font.render(f"Position: ({self.env.x:.2f}, {self.env.y:.2f})", True, WHITE)
+        self.screen.blit(text, (10, y_offset))
+        y_offset += 25
+        
+        # Velocity
+        text = self.small_font.render(f"Velocity: ({self.env.vx:.2f}, {self.env.vy:.2f})", True, WHITE)
+        self.screen.blit(text, (10, y_offset))
+        y_offset += 25
+        
+        # Wind
+        text = self.small_font.render(f"Wind: ({self.env.wind_x:.2f}, {self.env.wind_y:.2f})", True, GREEN)
+        self.screen.blit(text, (10, y_offset))
+        y_offset += 25
+        
+        # Action
+        if action is not None:
+            action_names = ["No thrust", "Up", "Down", "Left", "Right"]
+            text = self.small_font.render(f"Action: {action_names[action]}", True, YELLOW)
+            self.screen.blit(text, (10, y_offset))
+            y_offset += 25
+        
+        # Model info
+        if self.model is not None:
+            text = self.small_font.render("Mode: AI Agent (Liquid NN)", True, CYAN)
+        else:
+            text = self.small_font.render("Mode: Random Actions", True, GRAY)
+        self.screen.blit(text, (10, y_offset))
+    
+    def run(self, max_steps: int = 500, speed: float = 1.0):
+        """
+        Run the visualization.
+        
+        Args:
+            max_steps: Maximum number of steps to run
+            speed: Speed multiplier (1.0 = normal, higher = faster)
+        """
+        obs, info = self.env.reset()
+        done = False
+        truncated = False
+        step_count = 0
+        action = None
+        
+        running = True
+        paused = False
+        
+        while running and step_count < max_steps:
+            # Handle events
+            for event in pygame.event.get():
+                if event.type == pygame.QUIT:
+                    running = False
+                elif event.type == pygame.KEYDOWN:
+                    if event.key == pygame.K_SPACE:
+                        paused = not paused
+                    elif event.key == pygame.K_r:
+                        # Reset
+                        obs, info = self.env.reset()
+                        done = False
+                        truncated = False
+                        step_count = 0
+                    elif event.key == pygame.K_ESCAPE:
+                        running = False
+            
+            if not paused and not done and not truncated:
+                # Get action
+                if self.model is not None:
+                    action, _ = self.model.predict(obs, deterministic=True)
+                else:
+                    action = self.env.action_space.sample()
+                
+                # Step environment
+                obs, reward, done, truncated, info = self.env.step(action)
+                step_count += 1
+                in_target = info.get("in_target", False)
+                target_spawned = info.get("target_spawned", False)
+            
+            # Draw everything
+            self.screen.fill(BLACK)
+            
+            # Draw boundaries
+            self.draw_boundaries()
+            
+            # Draw target zone (only if spawned)
+            target_spawned_current = info.get("target_spawned", self.env.step_count >= 50) if not paused else False
+            self.draw_target_zone(target_spawned=target_spawned_current)
+            
+            # Draw wind arrows
+            self.draw_wind(self.env.wind_x, self.env.wind_y)
+            
+            # Draw drone
+            self.draw_drone(self.env.x, self.env.y, self.env.vx, self.env.vy)
+            
+            # Get in_target from info if available, otherwise compute
+            if not paused and 'in_target' in locals():
+                current_in_target = in_target
+            else:
+                from env.drone_env import TARGET_X_MIN, TARGET_X_MAX, TARGET_Y_MIN, TARGET_Y_MAX
+                current_in_target = (
+                    TARGET_X_MIN <= self.env.x <= TARGET_X_MAX and
+                    TARGET_Y_MIN <= self.env.y <= TARGET_Y_MAX
+                )
+            
+            # Draw info
+            self.draw_info(step_count, reward if not paused else 0, action, current_in_target)
+            
+            # Draw pause indicator
+            if paused:
+                text = self.font.render("PAUSED (SPACE to resume)", True, YELLOW)
+                text_rect = text.get_rect(center=(WINDOW_WIDTH // 2, 30))
+                self.screen.blit(text, text_rect)
+            
+            # Draw controls
+            controls_y = WINDOW_HEIGHT - 80
+            controls = [
+                "SPACE: Pause/Resume",
+                "R: Reset",
+                "ESC: Quit"
+            ]
+            for i, control in enumerate(controls):
+                text = self.small_font.render(control, True, GRAY)
+                self.screen.blit(text, (10, controls_y + i * 20))
+            
+            pygame.display.flip()
+            
+            # Control speed
+            if not paused:
+                self.clock.tick(FPS * speed)
+            else:
+                self.clock.tick(10)
+            
+            # Auto-reset on done/truncated
+            if (done or truncated) and not paused:
+                pygame.time.wait(1000)  # Wait 1 second before reset
+                obs, info = self.env.reset()
+                done = False
+                truncated = False
+                step_count = 0
+        
+        pygame.quit()
+
+
+def main():
+    """Main function to run the visualization."""
+    import argparse
+    
+    parser = argparse.ArgumentParser(description="Visualize drone environment")
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default="models/liquid_policy.zip",
+        help="Path to trained model (default: models/liquid_policy.zip)"
+    )
+    parser.add_argument(
+        "--random",
+        action="store_true",
+        help="Use random actions instead of trained model"
+    )
+    parser.add_argument(
+        "--max-steps",
+        type=int,
+        default=500,
+        help="Maximum steps per episode (default: 500)"
+    )
+    parser.add_argument(
+        "--speed",
+        type=float,
+        default=1.0,
+        help="Animation speed multiplier (default: 1.0)"
+    )
+    
+    args = parser.parse_args()
+    
+    # Create environment
+    env = DroneWindEnv()
+    
+    # Load model if specified
+    model = None
+    if not args.random:
+        if os.path.exists(args.model_path):
+            print(f"Loading model from {args.model_path}...")
+            model = PPO.load(args.model_path, env=env)
+            print("Model loaded successfully!")
+        else:
+            print(f"Model not found at {args.model_path}, using random actions")
+    
+    # Create visualizer
+    visualizer = DroneVisualizer(env, model)
+    
+    # Run visualization
+    print("\nStarting visualization...")
+    print("Controls:")
+    print("  SPACE: Pause/Resume")
+    print("  R: Reset episode")
+    print("  ESC: Quit")
+    print()
+    
+    visualizer.run(max_steps=args.max_steps, speed=args.speed)
+
+
+if __name__ == "__main__":
+    main()
+

demo_3d.py

@@ -0,0 +1,72 @@
+import numpy as np
+import time
+import os
+import matplotlib.pyplot as plt
+from mpl_toolkits.mplot3d import Axes3D
+from env.drone_3d import Drone3DEnv
+from models.liquid_ppo import make_liquid_ppo
+from stable_baselines3 import PPO
+
+def run_demo():
+    print("Initializing Project Neuro-Flyt 3D Demo (Matplotlib Mode)...")
+    
+    env = Drone3DEnv(render_mode="human", wind_scale=5.0, wind_speed=2.0)
+    
+    model_path = "liquid_ppo_drone_final.zip"
+    if os.path.exists(model_path):
+        print(f"Loading trained model from {model_path}...")
+        model = PPO.load(model_path, env=env)
+    else:
+        print("No trained model found. Using untrained Liquid Brain.")
+        model = make_liquid_ppo(env, verbose=1)
+    
+    print("\n=== DEMO STARTING ===")
+    
+    obs, info = env.reset()
+    
+    # Setup Plot
+    fig = plt.figure(figsize=(10, 8))
+    ax = fig.add_subplot(111, projection='3d')
+    
+    from matplotlib.animation import FuncAnimation
+    
+    def update(frame):
+        nonlocal obs, info
+        action, _ = model.predict(obs, deterministic=True)
+        obs, reward, term, trunc, info = env.step(action)
+        
+        ax.clear()
+        ax.set_xlim(-20, 20)
+        ax.set_ylim(-20, 20)
+        ax.set_zlim(0, 20)
+        ax.set_xlabel('X')
+        ax.set_ylabel('Y')
+        ax.set_zlabel('Z')
+        ax.set_title(f'Neuro-Flyt 3D | Step: {frame}')
+        
+        pos = obs[0:3]
+        wind = info.get("wind", np.zeros(3))
+        
+        # Draw Drone
+        ax.scatter(pos[0], pos[1], pos[2], c='blue', s=100, label='Drone')
+        
+        # Draw Wind Vector
+        ax.quiver(pos[0], pos[1], pos[2], wind[0], wind[1], wind[2], length=1.0, color='red', label='Wind Force')
+        
+        # Draw Target
+        ax.scatter(0, 0, 10, c='green', marker='x', s=100, label='Target')
+        
+        ax.legend()
+        
+        if term or trunc:
+            obs, info = env.reset()
+            
+    print("Generating Animation (demo.gif)...")
+    anim = FuncAnimation(fig, update, frames=200, interval=50)
+    anim.save('demo.gif', writer='pillow', fps=20)
+    print("Animation saved to demo.gif")
+    
+    env.close()
+
+if __name__ == "__main__":
+    run_demo()

demo_interactive.py

@@ -0,0 +1,124 @@
+import numpy as np
+import time
+import os
+import matplotlib.pyplot as plt
+from matplotlib.gridspec import GridSpec
+from env.drone_3d import Drone3DEnv
+from models.liquid_ppo import make_liquid_ppo
+from stable_baselines3 import PPO
+
+def run_interactive_demo():
+    print("Initializing Interactive Dashboard...")
+    
+    env = Drone3DEnv(render_mode="human", wind_scale=5.0, wind_speed=2.0)
+    
+    model_path = "liquid_ppo_drone_final.zip"
+    if os.path.exists(model_path):
+        print(f"Loading trained model from {model_path}...")
+        model = PPO.load(model_path, env=env)
+    else:
+        print("No trained model found. Using untrained Liquid Brain.")
+        model = make_liquid_ppo(env, verbose=1)
+    
+    obs, info = env.reset()
+    
+    # Setup Dashboard
+    plt.ion()
+    fig = plt.figure(figsize=(14, 8))
+    gs = GridSpec(2, 2, width_ratios=[2, 1])
+    
+    # 3D View (Left, spanning both rows)
+    ax_3d = fig.add_subplot(gs[:, 0], projection='3d')
+    
+    # Altitude Plot (Top Right)
+    ax_alt = fig.add_subplot(gs[0, 1])
+    ax_alt.set_title("Altitude (Z)")
+    ax_alt.set_ylim(0, 15)
+    ax_alt.set_xlim(0, 100)
+    line_alt, = ax_alt.plot([], [], 'b-')
+    
+    # Wind Speed Plot (Bottom Right)
+    ax_wind = fig.add_subplot(gs[1, 1])
+    ax_wind.set_title("Wind Magnitude")
+    ax_wind.set_ylim(0, 10)
+    ax_wind.set_xlim(0, 100)
+    line_wind, = ax_wind.plot([], [], 'r-')
+    
+    # Data Buffers
+    history_len = 100
+    alt_history = [10.0] * history_len
+    wind_history = [0.0] * history_len
+    
+    print("\n=== DASHBOARD LIVE ===")
+    print("Close the window to exit.")
+    
+    try:
+        step = 0
+        while True:
+            # Predict & Step
+            action, _ = model.predict(obs, deterministic=True)
+            obs, reward, term, trunc, info = env.step(action)
+            
+            # Update Data
+            pos = obs[0:3]
+            wind = info.get("wind", np.zeros(3))
+            wind_mag = np.linalg.norm(wind)
+            
+            alt_history.append(pos[2])
+            alt_history.pop(0)
+            wind_history.append(wind_mag)
+            wind_history.pop(0)
+            
+            # --- Render 3D View ---
+            ax_3d.clear()
+            ax_3d.set_xlim(-20, 20)
+            ax_3d.set_ylim(-20, 20)
+            ax_3d.set_zlim(0, 20)
+            ax_3d.set_xlabel('X')
+            ax_3d.set_ylabel('Y')
+            ax_3d.set_zlabel('Z')
+            ax_3d.set_title(f'Neuro-Flyt 3D | Step: {step}')
+            
+            # Drone
+            ax_3d.scatter(pos[0], pos[1], pos[2], c='blue', s=100, label='Drone')
+            # Wind Vector
+            ax_3d.quiver(pos[0], pos[1], pos[2], wind[0], wind[1], wind[2], length=1.0, color='red', label='Wind')
+            
+            # Target
+            target = info.get("target", np.array([0, 0, 10.0]))
+            ax_3d.scatter(target[0], target[1], target[2], c='green', marker='x', s=100, label='Target')
+            ax_3d.legend(loc='upper left')
+            
+            # --- Render Stats ---
+            line_alt.set_ydata(alt_history)
+            line_alt.set_xdata(range(history_len))
+            
+            line_wind.set_ydata(wind_history)
+            line_wind.set_xdata(range(history_len))
+            
+            # Stats Text
+            stats = f"Alt: {pos[2]:.2f}m\nWind: {wind_mag:.2f} N\nDrift: {np.linalg.norm(pos[:2]):.2f}m"
+            ax_3d.text2D(0.05, 0.95, stats, transform=ax_3d.transAxes, fontsize=12, bbox=dict(facecolor='white', alpha=0.7))
+            
+            plt.draw()
+            plt.pause(0.01)
+            
+            if term or trunc:
+                obs, info = env.reset()
+                
+            step += 1
+            
+            # Check if window is closed
+            if not plt.fignum_exists(fig.number):
+                break
+                
+    except KeyboardInterrupt:
+        print("Interrupted.")
+    except Exception as e:
+        print(f"Error: {e}")
+    finally:
+        plt.close()
+        env.close()
+
+if __name__ == "__main__":
+    run_interactive_demo()

demo_log.txt

@@ -0,0 +1,43 @@
+Initializing Project Neuro-Flyt 3D Demo (Matplotlib Mode)...
+No trained model found. Using untrained Liquid Brain.
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+Traceback (most recent call last):
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 72, in <module>
+    run_demo()
+    ~~~~~~~~^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 21, in run_demo
+    model = make_liquid_ppo(env, verbose=1)
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/models/liquid_ppo.py", line 67, in make_liquid_ppo
+    model = PPO(
+        "MlpPolicy",
+    ...<9 lines>...
+        clip_range=0.2,
+    )
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/ppo/ppo.py", line 171, in __init__
+    self._setup_model()
+    ~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/ppo/ppo.py", line 174, in _setup_model
+    super()._setup_model()
+    ~~~~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 135, in _setup_model
+    self.policy = self.policy_class(  # type: ignore[assignment]
+                  ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+        self.observation_space, self.action_space, self.lr_schedule, use_sde=self.use_sde, **self.policy_kwargs
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 507, in __init__
+    self.features_extractor = self.make_features_extractor()
+                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 120, in make_features_extractor
+    return self.features_extractor_class(self.observation_space, **self.features_extractor_kwargs)
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/models/liquid_ppo.py", line 18, in __init__
+    self.features_dim = features_dim
+    ^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 2071, in __setattr__
+    super().__setattr__(name, value)
+    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
+AttributeError: property 'features_dim' of 'LTCFeatureExtractor' object has no setter

demo_log_2.txt

@@ -0,0 +1,43 @@
+Initializing Project Neuro-Flyt 3D Demo (Matplotlib Mode)...
+No trained model found. Using untrained Liquid Brain.
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+Traceback (most recent call last):
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 72, in <module>
+    run_demo()
+    ~~~~~~~~^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 21, in run_demo
+    model = make_liquid_ppo(env, verbose=1)
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/models/liquid_ppo.py", line 67, in make_liquid_ppo
+    model = PPO(
+        "MlpPolicy",
+    ...<9 lines>...
+        clip_range=0.2,
+    )
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/ppo/ppo.py", line 171, in __init__
+    self._setup_model()
+    ~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/ppo/ppo.py", line 174, in _setup_model
+    super()._setup_model()
+    ~~~~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 135, in _setup_model
+    self.policy = self.policy_class(  # type: ignore[assignment]
+                  ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+        self.observation_space, self.action_space, self.lr_schedule, use_sde=self.use_sde, **self.policy_kwargs
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 507, in __init__
+    self.features_extractor = self.make_features_extractor()
+                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 120, in make_features_extractor
+    return self.features_extractor_class(self.observation_space, **self.features_extractor_kwargs)
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/models/liquid_ppo.py", line 22, in __init__
+    wiring = AutoNCP(features_dim, output_size=features_dim)
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/wirings/wirings.py", line 622, in __init__
+    raise ValueError(
+        f"Output size must be less than the number of units-2 (given {units} units, {output_size} output size)"
+    )
+ValueError: Output size must be less than the number of units-2 (given 32 units, 32 output size)

demo_log_3.txt

@@ -0,0 +1,87 @@
+Initializing Project Neuro-Flyt 3D Demo (Matplotlib Mode)...
+No trained model found. Using untrained Liquid Brain.
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+
+=== DEMO STARTING ===
+Generating Animation (demo.gif)...
+Traceback (most recent call last):
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 224, in saving
+    yield self
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1109, in save
+    anim._init_draw()  # Clear the initial frame
+    ~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1770, in _init_draw
+    self._draw_frame(frame_data)
+    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1789, in _draw_frame
+    self._drawn_artists = self._func(framedata, *self._args)
+                          ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 35, in update
+    action, _ = model.predict(obs, deterministic=True)
+                ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/base_class.py", line 557, in predict
+    return self.policy.predict(observation, state, episode_start, deterministic)
+           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 368, in predict
+    actions = self._predict(obs_tensor, deterministic=deterministic)
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 717, in _predict
+    return self.get_distribution(observation).get_actions(deterministic=deterministic)
+           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 750, in get_distribution
+    features = super().extract_features(obs, self.pi_features_extractor)
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 131, in extract_features
+    return features_extractor(preprocessed_obs)
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
+    return self._call_impl(*args, **kwargs)
+           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
+    return forward_call(*args, **kwargs)
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/models/liquid_ppo.py", line 54, in forward
+    output, self.hx = self.ltc(observations, self.hx)
+                      ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
+    return self._call_impl(*args, **kwargs)
+           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
+    return forward_call(*args, **kwargs)
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc.py", line 185, in forward
+    h_out, h_state = self.rnn_cell.forward(inputs, h_state, ts)
+                     ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc_cell.py", line 282, in forward
+    next_state = self._ode_solver(inputs, states, elapsed_time)
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc_cell.py", line 230, in _ode_solver
+    w_activation = w_param * self._sigmoid(
+                             ~~~~~~~~~~~~~^
+        v_pre, self._params["mu"], self._params["sigma"]
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc_cell.py", line 199, in _sigmoid
+    mues = v_pre - mu
+           ~~~~~~^~~~
+RuntimeError: The size of tensor a (32) must match the size of tensor b (48) at non-singleton dimension 1
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 72, in <module>
+    run_demo()
+    ~~~~~~~~^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 66, in run_demo
+    anim.save('demo.gif', writer='pillow', fps=20)
+    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1098, in save
+    with (writer.saving(self._fig, filename, dpi),
+          ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/lib64/python3.14/contextlib.py", line 162, in __exit__
+    self.gen.throw(value)
+    ~~~~~~~~~~~~~~^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 226, in saving
+    self.finish()
+    ~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 506, in finish
+    self._frames[0].save(
+    ~~~~~~~~~~~~^^^
+IndexError: list index out of range

demo_log_4.txt

@@ -0,0 +1,87 @@
+Initializing Project Neuro-Flyt 3D Demo (Matplotlib Mode)...
+No trained model found. Using untrained Liquid Brain.
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+
+=== DEMO STARTING ===
+Generating Animation (demo.gif)...
+Traceback (most recent call last):
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 224, in saving
+    yield self
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1109, in save
+    anim._init_draw()  # Clear the initial frame
+    ~~~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1770, in _init_draw
+    self._draw_frame(frame_data)
+    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1789, in _draw_frame
+    self._drawn_artists = self._func(framedata, *self._args)
+                          ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 35, in update
+    action, _ = model.predict(obs, deterministic=True)
+                ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/base_class.py", line 557, in predict
+    return self.policy.predict(observation, state, episode_start, deterministic)
+           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 368, in predict
+    actions = self._predict(obs_tensor, deterministic=deterministic)
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 717, in _predict
+    return self.get_distribution(observation).get_actions(deterministic=deterministic)
+           ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 750, in get_distribution
+    features = super().extract_features(obs, self.pi_features_extractor)
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/policies.py", line 131, in extract_features
+    return features_extractor(preprocessed_obs)
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
+    return self._call_impl(*args, **kwargs)
+           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
+    return forward_call(*args, **kwargs)
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/models/liquid_ppo.py", line 54, in forward
+    output, self.hx = self.ltc(observations, self.hx)
+                      ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
+    return self._call_impl(*args, **kwargs)
+           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
+    return forward_call(*args, **kwargs)
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc.py", line 185, in forward
+    h_out, h_state = self.rnn_cell.forward(inputs, h_state, ts)
+                     ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc_cell.py", line 282, in forward
+    next_state = self._ode_solver(inputs, states, elapsed_time)
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc_cell.py", line 230, in _ode_solver
+    w_activation = w_param * self._sigmoid(
+                             ~~~~~~~~~~~~~^
+        v_pre, self._params["mu"], self._params["sigma"]
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/ncps/torch/ltc_cell.py", line 199, in _sigmoid
+    mues = v_pre - mu
+           ~~~~~~^~~~
+RuntimeError: The size of tensor a (32) must match the size of tensor b (48) at non-singleton dimension 1
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 72, in <module>
+    run_demo()
+    ~~~~~~~~^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/demo_3d.py", line 66, in run_demo
+    anim.save('demo.gif', writer='pillow', fps=20)
+    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 1098, in save
+    with (writer.saving(self._fig, filename, dpi),
+          ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/usr/lib64/python3.14/contextlib.py", line 162, in __exit__
+    self.gen.throw(value)
+    ~~~~~~~~~~~~~~^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 226, in saving
+    self.finish()
+    ~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/matplotlib/animation.py", line 506, in finish
+    self._frames[0].save(
+    ~~~~~~~~~~~~^^^
+IndexError: list index out of range

demo_log_5.txt

@@ -0,0 +1,9 @@
+Initializing Project Neuro-Flyt 3D Demo (Matplotlib Mode)...
+No trained model found. Using untrained Liquid Brain.
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+
+=== DEMO STARTING ===
+Generating Animation (demo.gif)...
+Animation saved to demo.gif

deploy_log.txt

@@ -0,0 +1,254 @@
+Cloning Space...
+Cloning into 'hf_deploy'...
+Copying files...
+sending incremental file list
+.gitignore
+
            536 100%    0.00kB/s    0:00:00  
            536 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=76/78)
+Dockerfile
+
            622 100%  607.42kB/s    0:00:00  
            622 100%  607.42kB/s    0:00:00 (xfr#2, to-chk=75/78)
+README.md
+
            924 100%  902.34kB/s    0:00:00  
            924 100%  902.34kB/s    0:00:00 (xfr#3, to-chk=74/78)
+README_HF.md
+
          3,598 100%    3.43MB/s    0:00:00  
          3,598 100%    3.43MB/s    0:00:00 (xfr#4, to-chk=73/78)
+debug_imports.py
+
            620 100%  605.47kB/s    0:00:00  
            620 100%  605.47kB/s    0:00:00 (xfr#5, to-chk=72/78)
+demo.gif
+
         32,768   4%   31.25MB/s    0:00:00  
        692,490 100%  330.20MB/s    0:00:00 (xfr#6, to-chk=71/78)
+demo_3d.py
+
          2,198 100%  715.49kB/s    0:00:00  
          2,198 100%  715.49kB/s    0:00:00 (xfr#7, to-chk=70/78)
+demo_interactive.py
+
          4,050 100%    1.29MB/s    0:00:00  
          4,050 100%    1.29MB/s    0:00:00 (xfr#8, to-chk=69/78)
+demo_log.txt
+
          2,580 100%  839.84kB/s    0:00:00  
          2,580 100%  839.84kB/s    0:00:00 (xfr#9, to-chk=68/78)
+demo_log_2.txt
+
          2,652 100%  863.28kB/s    0:00:00  
          2,652 100%  863.28kB/s    0:00:00 (xfr#10, to-chk=67/78)
+demo_log_3.txt
+
          5,276 100%    1.68MB/s    0:00:00  
          5,276 100%    1.68MB/s    0:00:00 (xfr#11, to-chk=66/78)
+demo_log_4.txt
+
          5,276 100%    1.68MB/s    0:00:00  
          5,276 100%    1.68MB/s    0:00:00 (xfr#12, to-chk=65/78)
+demo_log_5.txt
+
            295 100%   96.03kB/s    0:00:00  
            295 100%   96.03kB/s    0:00:00 (xfr#13, to-chk=64/78)
+deploy_log.txt
+
          1,740 100%  566.41kB/s    0:00:00  
          1,740 100%  566.41kB/s    0:00:00 (xfr#14, to-chk=63/78)
+install_log.txt
+
         14,311 100%    4.55MB/s    0:00:00  
         14,311 100%    4.55MB/s    0:00:00 (xfr#15, to-chk=62/78)
+interactive_log.txt
+
            236 100%   76.82kB/s    0:00:00  
            236 100%   76.82kB/s    0:00:00 (xfr#16, to-chk=61/78)
+main.py
+
          4,348 100%    1.38MB/s    0:00:00  
          4,348 100%    1.38MB/s    0:00:00 (xfr#17, to-chk=60/78)
+output.txt
+
            281 100%   91.47kB/s    0:00:00  
            281 100%   91.47kB/s    0:00:00 (xfr#18, to-chk=59/78)
+pip_log.txt
+
          5,014 100%    1.59MB/s    0:00:00  
          5,014 100%    1.59MB/s    0:00:00 (xfr#19, to-chk=58/78)
+pybullet_bin_log.txt
+
            223 100%   72.59kB/s    0:00:00  
            223 100%   72.59kB/s    0:00:00 (xfr#20, to-chk=57/78)
+pybullet_log.txt
+
         32,768  25%   10.42MB/s    0:00:00  
        129,509 100%   30.88MB/s    0:00:00 (xfr#21, to-chk=56/78)
+pygame_log.txt
+
            219 100%   53.47kB/s    0:00:00  
            219 100%   53.47kB/s    0:00:00 (xfr#22, to-chk=55/78)
+requirements.txt
+
            194 100%   47.36kB/s    0:00:00  
            194 100%   47.36kB/s    0:00:00 (xfr#23, to-chk=54/78)
+run_demo.sh
+
            178 100%   43.46kB/s    0:00:00  
            178 100%   43.46kB/s    0:00:00 (xfr#24, to-chk=53/78)
+setup.sh
+
            978 100%  238.77kB/s    0:00:00  
            978 100%  238.77kB/s    0:00:00 (xfr#25, to-chk=52/78)
+test_log.txt
+
            964 100%  235.35kB/s    0:00:00  
            964 100%  235.35kB/s    0:00:00 (xfr#26, to-chk=51/78)
+test_physics.py
+
          1,311 100%  320.07kB/s    0:00:00  
          1,311 100%  320.07kB/s    0:00:00 (xfr#27, to-chk=50/78)
+test_random_1.txt
+
            318 100%   77.64kB/s    0:00:00  
            318 100%   77.64kB/s    0:00:00 (xfr#28, to-chk=49/78)
+test_random_2.txt
+
            315 100%   76.90kB/s    0:00:00  
            315 100%   76.90kB/s    0:00:00 (xfr#29, to-chk=48/78)
+test_result_final.txt
+
            263 100%   64.21kB/s    0:00:00  
            263 100%   64.21kB/s    0:00:00 (xfr#30, to-chk=47/78)
+test_success.txt
+
            263 100%   64.21kB/s    0:00:00  
            263 100%   64.21kB/s    0:00:00 (xfr#31, to-chk=46/78)
+train.py
+
          1,205 100%  294.19kB/s    0:00:00  
          1,205 100%  294.19kB/s    0:00:00 (xfr#32, to-chk=45/78)
+train_hf.py
+
          2,277 100%  555.91kB/s    0:00:00  
          2,277 100%  555.91kB/s    0:00:00 (xfr#33, to-chk=44/78)
+train_log_500k.txt
+
         32,768  39%    7.81MB/s    0:00:00  
         83,713 100%   19.96MB/s    0:00:00 (xfr#34, to-chk=43/78)
+train_log_full.txt
+
          3,044 100%  743.16kB/s    0:00:00  
          3,044 100%  743.16kB/s    0:00:00 (xfr#35, to-chk=42/78)
+train_log_retry.txt
+
         14,709 100%    3.51MB/s    0:00:00  
         14,709 100%    3.51MB/s    0:00:00 (xfr#36, to-chk=41/78)
+checkpoints/
+checkpoints/liquid_ppo_drone_100000_steps.zip
+
         32,768   9%    6.25MB/s    0:00:00  
        361,759 100%   57.50MB/s    0:00:00 (xfr#37, to-chk=33/78)
+checkpoints/liquid_ppo_drone_10000_steps.zip
+
         32,768   9%    4.46MB/s    0:00:00  
        358,037 100%   48.78MB/s    0:00:00 (xfr#38, to-chk=32/78)
+checkpoints/liquid_ppo_drone_110000_steps.zip
+
         32,768   9%    3.91MB/s    0:00:00  
        361,803 100%   38.34MB/s    0:00:00 (xfr#39, to-chk=31/78)
+checkpoints/liquid_ppo_drone_120000_steps.zip
+
         32,768   9%    3.12MB/s    0:00:00  
        361,803 100%   34.50MB/s    0:00:00 (xfr#40, to-chk=30/78)
+checkpoints/liquid_ppo_drone_130000_steps.zip
+
         32,768   9%    2.60MB/s    0:00:00  
        361,803 100%   28.75MB/s    0:00:00 (xfr#41, to-chk=29/78)
+checkpoints/liquid_ppo_drone_140000_steps.zip
+
         32,768   9%    2.40MB/s    0:00:00  
        361,803 100%   24.65MB/s    0:00:00 (xfr#42, to-chk=28/78)
+checkpoints/liquid_ppo_drone_150000_steps.zip
+
         32,768   9%    2.08MB/s    0:00:00  
        361,813 100%   23.00MB/s    0:00:00 (xfr#43, to-chk=27/78)
+checkpoints/liquid_ppo_drone_160000_steps.zip
+
         32,768   9%    1.84MB/s    0:00:00  
        361,803 100%   20.30MB/s    0:00:00 (xfr#44, to-chk=26/78)
+checkpoints/liquid_ppo_drone_170000_steps.zip
+
         32,768   9%    1.74MB/s    0:00:00  
        361,803 100%   18.16MB/s    0:00:00 (xfr#45, to-chk=25/78)
+checkpoints/liquid_ppo_drone_180000_steps.zip
+
         32,768   9%    1.56MB/s    0:00:00  
        361,803 100%   16.43MB/s    0:00:00 (xfr#46, to-chk=24/78)
+checkpoints/liquid_ppo_drone_20000_steps.zip
+
         32,768   9%    1.42MB/s    0:00:00  
        358,453 100%   14.86MB/s    0:00:00 (xfr#47, to-chk=23/78)
+checkpoints/liquid_ppo_drone_30000_steps.zip
+
         32,768   9%    1.30MB/s    0:00:00  
        358,866 100%   14.26MB/s    0:00:00 (xfr#48, to-chk=22/78)
+checkpoints/liquid_ppo_drone_40000_steps.zip
+
         32,768   9%    1.20MB/s    0:00:00  
        359,278 100%   13.18MB/s    0:00:00 (xfr#49, to-chk=21/78)
+checkpoints/liquid_ppo_drone_50000_steps.zip
+
         32,768   9%    1.12MB/s    0:00:00  
        359,694 100%   11.83MB/s    0:00:00 (xfr#50, to-chk=20/78)
+checkpoints/liquid_ppo_drone_60000_steps.zip
+
         32,768   9%    1.04MB/s    0:00:00  
        360,106 100%   11.08MB/s    0:00:00 (xfr#51, to-chk=19/78)
+checkpoints/liquid_ppo_drone_70000_steps.zip
+
         32,768   9%  969.70kB/s    0:00:00  
        360,518 100%   10.11MB/s    0:00:00 (xfr#52, to-chk=18/78)
+checkpoints/liquid_ppo_drone_80000_steps.zip
+
         32,768   9%  888.89kB/s    0:00:00  
        360,934 100%    9.30MB/s    0:00:00 (xfr#53, to-chk=17/78)
+checkpoints/liquid_ppo_drone_90000_steps.zip
+
         32,768   9%  842.11kB/s    0:00:00  
        361,356 100%    8.84MB/s    0:00:00 (xfr#54, to-chk=16/78)
+demo/
+demo/__init__.py
+
             39 100%    0.98kB/s    0:00:00  
             39 100%    0.98kB/s    0:00:00 (xfr#55, to-chk=15/78)
+demo/visualize_drone.py
+
         16,089 100%  402.87kB/s    0:00:00  
         16,089 100%  402.87kB/s    0:00:00 (xfr#56, to-chk=14/78)
+env/
+env/__init__.py
+
             62 100%    1.55kB/s    0:00:00  
             62 100%    1.55kB/s    0:00:00 (xfr#57, to-chk=13/78)
+env/drone_3d.py
+
          5,304 100%  132.81kB/s    0:00:00  
          5,304 100%  132.81kB/s    0:00:00 (xfr#58, to-chk=12/78)
+env/drone_env.py
+
         10,099 100%  252.88kB/s    0:00:00  
         10,099 100%  252.88kB/s    0:00:00 (xfr#59, to-chk=11/78)
+eval/
+eval/__init__.py
+
             41 100%    1.03kB/s    0:00:00  
             41 100%    1.03kB/s    0:00:00 (xfr#60, to-chk=10/78)
+eval/eval_liquid_policy.py
+
          5,742 100%  143.78kB/s    0:00:00  
          5,742 100%  143.78kB/s    0:00:00 (xfr#61, to-chk=9/78)
+eval/eval_mlp_baseline.py
+
          5,570 100%  139.47kB/s    0:00:00  
          5,570 100%  139.47kB/s    0:00:00 (xfr#62, to-chk=8/78)
+models/
+models/__init__.py
+
             44 100%    1.10kB/s    0:00:00  
             44 100%    1.10kB/s    0:00:00 (xfr#63, to-chk=7/78)
+models/liquid_cell.py
+
          3,168 100%   79.33kB/s    0:00:00  
          3,168 100%   79.33kB/s    0:00:00 (xfr#64, to-chk=6/78)
+models/liquid_policy.py
+
          2,690 100%   67.36kB/s    0:00:00  
          2,690 100%   67.36kB/s    0:00:00 (xfr#65, to-chk=5/78)
+models/liquid_ppo.py
+
          4,141 100%  103.69kB/s    0:00:00  
          4,141 100%  103.69kB/s    0:00:00 (xfr#66, to-chk=4/78)
+train/
+train/__init__.py
+
             39 100%    0.98kB/s    0:00:00  
             39 100%    0.98kB/s    0:00:00 (xfr#67, to-chk=3/78)
+train/train_liquid_ppo.py
+
          6,619 100%  165.74kB/s    0:00:00  
          6,619 100%  165.74kB/s    0:00:00 (xfr#68, to-chk=2/78)
+train/train_mlp_ppo.py
+
          4,442 100%  111.23kB/s    0:00:00  
          4,442 100%  111.23kB/s    0:00:00 (xfr#69, to-chk=1/78)
+utils/
+utils/__init__.py
+
             38 100%    0.95kB/s    0:00:00  
             38 100%    0.95kB/s    0:00:00 (xfr#70, to-chk=0/78)
+
+sent 7,551,292 bytes  received 1,390 bytes  15,105,364.00 bytes/sec
+total size is 7,542,644  speedup is 1.00
+[main 8ef522c] Deploy Neuro-Flyt 3D Training
+ 70 files changed, 9247 insertions(+), 10 deletions(-)
+ create mode 100644 .gitignore
+ create mode 100644 Dockerfile
+ create mode 100644 README_HF.md
+ create mode 100644 checkpoints/liquid_ppo_drone_100000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_10000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_110000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_120000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_130000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_140000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_150000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_160000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_170000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_180000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_20000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_30000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_40000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_50000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_60000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_70000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_80000_steps.zip
+ create mode 100644 checkpoints/liquid_ppo_drone_90000_steps.zip
+ create mode 100644 debug_imports.py
+ create mode 100644 demo.gif
+ create mode 100644 demo/__init__.py
+ create mode 100644 demo/visualize_drone.py
+ create mode 100644 demo_3d.py
+ create mode 100644 demo_interactive.py
+ create mode 100644 demo_log.txt
+ create mode 100644 demo_log_2.txt
+ create mode 100644 demo_log_3.txt
+ create mode 100644 demo_log_4.txt
+ create mode 100644 demo_log_5.txt
+ create mode 100644 deploy_log.txt
+ create mode 100644 env/__init__.py
+ create mode 100644 env/drone_3d.py
+ create mode 100644 env/drone_env.py
+ create mode 100644 eval/__init__.py
+ create mode 100644 eval/eval_liquid_policy.py
+ create mode 100644 eval/eval_mlp_baseline.py
+ create mode 100644 install_log.txt
+ create mode 100644 interactive_log.txt
+ create mode 100644 main.py
+ create mode 100644 models/__init__.py
+ create mode 100644 models/liquid_cell.py
+ create mode 100644 models/liquid_policy.py
+ create mode 100644 models/liquid_ppo.py
+ create mode 100644 output.txt
+ create mode 100644 pip_log.txt
+ create mode 100644 pybullet_bin_log.txt
+ create mode 100644 pybullet_log.txt
+ create mode 100644 pygame_log.txt
+ create mode 100644 requirements.txt
+ create mode 100755 run_demo.sh
+ create mode 100755 setup.sh
+ create mode 100644 test_log.txt
+ create mode 100644 test_physics.py
+ create mode 100644 test_random_1.txt
+ create mode 100644 test_random_2.txt
+ create mode 100644 test_result_final.txt
+ create mode 100644 test_success.txt
+ create mode 100644 train.py
+ create mode 100644 train/__init__.py
+ create mode 100644 train/train_liquid_ppo.py
+ create mode 100644 train/train_mlp_ppo.py
+ create mode 100644 train_hf.py
+ create mode 100644 train_log_500k.txt
+ create mode 100644 train_log_full.txt
+ create mode 100644 train_log_retry.txt
+ create mode 100644 utils/__init__.py
+remote: -------------------------------------------------------------------------        
+remote: Your push was rejected because it contains binary files.        
+remote: Please use https://huggingface.co/docs/hub/xet to store binary files.        
+remote: See also: https://huggingface.co/docs/hub/xet/using-xet-storage#git        
+remote:         
+remote: Offending files:        
+remote:   - checkpoints/liquid_ppo_drone_100000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_10000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_110000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_120000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_130000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_140000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_150000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_160000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_170000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_180000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_20000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_30000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_40000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_50000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_60000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_70000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_80000_steps.zip (ref: refs/heads/main)        
+remote:   - checkpoints/liquid_ppo_drone_90000_steps.zip (ref: refs/heads/main)        
+remote:   - demo.gif (ref: refs/heads/main)        
+remote: -------------------------------------------------------------------------        
+To https://huggingface.co/spaces/iteratehack/neuro-flyt-training
+ ! [remote rejected] main -> main (pre-receive hook declined)
+error: failed to push some refs to 'https://huggingface.co/spaces/iteratehack/neuro-flyt-training'

deploy_log_2.txt

@@ -0,0 +1,30 @@
+Cloning Space...
+Cloning into 'hf_deploy'...
+Copying files...
+sending incremental file list
+.gitignore
+
            536 100%    0.00kB/s    0:00:00  
            536 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=57/59)
+Dockerfile
+
            622 100%  607.42kB/s    0:00:00  
            622 100%  607.42kB/s    0:00:00 (xfr#2, to-chk=56/59)
+README.md
+
            924 100%  902.34kB/s    0:00:00  
            924 100%  902.34kB/s    0:00:00 (xfr#3, to-chk=55/59)
+README_HF.md
+
          3,598 100%    3.43MB/s    0:00:00  
          3,598 100%    3.43MB/s    0:00:00 (xfr#4, to-chk=54/59)
+debug_imports.py
+
            620 100%  605.47kB/s    0:00:00  
            620 100%  605.47kB/s    0:00:00 (xfr#5, to-chk=53/59)
+demo_3d.py
+
          2,198 100%    2.10MB/s    0:00:00  
          2,198 100%    2.10MB/s    0:00:00 (xfr#6, to-chk=52/59)
+demo_interactive.py
+
          4,050 100%    3.86MB/s    0:00:00  
          4,050 100%    3.86MB/s    0:00:00 (xfr#7, to-chk=51/59)
+demo_log.txt
+
          2,580 100%    2.46MB/s    0:00:00  
          2,580 100%    2.46MB/s    0:00:00 (xfr#8, to-chk=50/59)
+demo_log_2.txt
+
          2,652 100%    2.53MB/s    0:00:00  
          2,652 100%    2.53MB/s    0:00:00 (xfr#9, to-chk=49/59)
+demo_log_3.txt
+
          5,276 100%    5.03MB/s    0:00:00  
          5,276 100%    5.03MB/s    0:00:00 (xfr#10, to-chk=48/59)
+demo_log_4.txt
+
          5,276 100%    5.03MB/s    0:00:00  
          5,276 100%    5.03MB/s    0:00:00 (xfr#11, to-chk=47/59)
+demo_log_5.txt
+
            295 100%  288.09kB/s    0:00:00  
            295 100%  288.09kB/s    0:00:00 (xfr#12, to-chk=46/59)
+deploy_log.txt
+
         15,625 100%   14.90MB/s    0:00:00  
         15,625 100%   14.90MB/s    0:00:00 (xfr#13, to-chk=45/59)

env/__init__.py

@@ -0,0 +1,2 @@
+"""Environment modules for custom gymnasium environments."""
+

env/drone_3d.py

@@ -0,0 +1,143 @@
+import numpy as np
+import gymnasium as gym
+from opensimplex import OpenSimplex
+import time
+
+# Try to import PyFlyt, else fallback
+try:
+    from PyFlyt.gym_envs.quadx_envs.quadx_waypoints_env import QuadXWaypointsEnv
+    PYFLYT_AVAILABLE = True
+except ImportError:
+    PYFLYT_AVAILABLE = False
+    # Create a dummy base class if PyFlyt is missing
+    class QuadXWaypointsEnv(gym.Env):
+        metadata = {"render_modes": ["human", "rgb_array"]}
+        def __init__(self, render_mode=None):
+            self.render_mode = render_mode
+
+class WindField:
+    def __init__(self, seed=42, scale=0.1, speed=1.0):
+        self.noise = OpenSimplex(seed=seed)
+        self.scale = scale
+        self.speed = speed
+        self.time_offset = 0.0
+
+    def get_wind(self, x, y, z, dt):
+        self.time_offset += dt * self.speed
+        u = self.noise.noise4(x * self.scale, y * self.scale, z * self.scale, self.time_offset)
+        v = self.noise.noise4(x * self.scale + 100, y * self.scale + 100, z * self.scale, self.time_offset)
+        w = self.noise.noise4(x * self.scale + 200, y * self.scale + 200, z * self.scale, self.time_offset)
+        return np.array([u, v, w])
+
+class Drone3DEnv(gym.Env):
+    def __init__(self, render_mode=None, wind_scale=10.0, wind_speed=1.0):
+        super().__init__()
+        self.render_mode = render_mode
+        self.wind_field = WindField(scale=0.05, speed=wind_speed)
+        self.wind_strength = wind_scale
+        
+        # Define spaces
+        # Obs: [x, y, z, roll, pitch, yaw, u, v, w, p, q, r]
+        self.observation_space = gym.spaces.Box(low=-np.inf, high=np.inf, shape=(12,), dtype=np.float32)
+        # Action: [motor1, motor2, motor3, motor4] or [thrust, roll, pitch, yaw]
+        # We'll assume simple [thrust_x, thrust_y, thrust_z, yaw] for the mock
+        self.action_space = gym.spaces.Box(low=-1, high=1, shape=(4,), dtype=np.float32)
+        
+        self.state = np.zeros(12)
+        self.dt = 0.05
+        self.step_count = 0
+        self.max_steps = 1000
+
+    def reset(self, seed=None, options=None):
+        super().reset(seed=seed)
+        self.state = np.zeros(12)
+        self.state[2] = 10.0 # Start at 10m height
+        self.step_count = 0
+        
+        # Randomize Target
+        # x, y in [-5, 5], z in [5, 15]
+        self.target = np.random.uniform(low=[-5, -5, 5], high=[5, 5, 15])
+        
+        # Randomize Wind
+        # We can just re-initialize the noise with a random seed
+        new_seed = np.random.randint(0, 10000)
+        self.wind_field = WindField(seed=new_seed, scale=0.05, speed=self.wind_field.speed)
+        
+        return self.state.astype(np.float32), {}
+
+    def step(self, action):
+        self.step_count += 1
+        
+        # Unpack state
+        pos = self.state[0:3]
+        vel = self.state[6:9]
+        
+        # Get Wind
+        raw_wind = self.wind_field.get_wind(pos[0], pos[1], pos[2], self.dt)
+        wind_force = raw_wind * self.wind_strength
+        
+        # Simple Kinematics (Double Integrator)
+        # Action is roughly acceleration command
+        # We need enough authority to fight gravity (9.81) + wind
+        # Let's say max thrust is 20 m/s^2 (~2G)
+        # Action [-1, 1] -> [-20, 20] ? 
+        # No, usually thrust is positive 0..Max. 
+        # But for simplified "QuadX" control often inputs are roll/pitch/yaw/thrust.
+        # Here we are abstracting to "Force/Accel command in 3D".
+        # Let's map action [-1, 1] to [-15, 15] acceleration.
+        accel = action[:3] * 15.0 
+        
+        # Gravity
+        gravity = np.array([0, 0, -9.81])
+        
+        # Total Force = Control + Wind + Gravity
+        # Note: We REMOVED the "anti-gravity" offset. 
+        # The agent MUST output positive Z acceleration to hover.
+        # If action[2] is 0, accel[2] is 0, and it falls due to gravity.
+        total_accel = accel + wind_force + gravity
+        
+        # Update State
+        vel += total_accel * self.dt
+        pos += vel * self.dt
+        
+        # Floor collision
+        if pos[2] < 0:
+            pos[2] = 0
+            vel[2] = 0 # Crash stop
+        
+        # Drag (Damping)
+        vel *= 0.95
+        
+        self.state[0:3] = pos
+        self.state[6:9] = vel
+        
+        # Reward: Stay close to Target
+        dist = np.linalg.norm(pos - self.target)
+        
+        # Smoothness: Penalty for high velocity (instability)
+        vel_mag = np.linalg.norm(vel)
+        
+        # Components:
+        # 1. Distance Reward: Higher is better (closer to 0)
+        r_dist = -dist
+        
+        # 2. Stability Penalty: Penalize erratic high-speed movements if far from target
+        # But we need speed to get there. Let's just penalize extreme speed.
+        r_vel = -0.01 * vel_mag
+        
+        # 3. Survival Reward: Bonus for not crashing
+        r_survive = 0.1
+        
+        reward = r_dist + r_vel + r_survive
+        
+        # Terminate if crashed or too far
+        term = False # Let it crash and stay on floor
+        trunc = self.step_count >= self.max_steps
+        
+        info = {"wind": wind_force, "target": self.target}
+        
+        return self.state.astype(np.float32), reward, term, trunc, info
+
+    def render(self):
+        # We will handle rendering in the demo script using matplotlib
+        pass

env/drone_env.py

@@ -0,0 +1,286 @@
+"""
+A 2D drone environment with dynamic wind forces for reinforcement learning.
+The drone can apply discrete thrust actions while being affected by smoothly varying wind.
+The goal is to navigate and survive within the bounded world.
+"""
+
+import numpy as np
+import gymnasium as gym
+from gymnasium import spaces
+from typing import Optional, Tuple, Dict, Any
+
+
+# Constants
+DT = 0.1  # Time step
+MAX_VEL = 2.0  # Maximum velocity magnitude
+WIND_MAX = 2.0  # Maximum wind magnitude
+WIND_SMOOTHING = 0.05  # Wind interpolation rate toward target
+WIND_TARGET_INTERVAL = 50  # Steps between sampling new wind target
+MAX_STEPS = 500  # Maximum episode length
+POSITION_MIN = 0.0  # Minimum position (x, y)
+POSITION_MAX = 1.0  # Maximum position (x, y)
+THRUST = 0.25  # Thrust magnitude per action (slightly higher for control authority)
+
+# Target zone (box) constants
+TARGET_X_MIN = 0.7  # Target box left edge
+TARGET_X_MAX = 0.9  # Target box right edge
+TARGET_Y_MIN = 0.3  # Target box bottom edge
+TARGET_Y_MAX = 0.7  # Target box top edge
+TARGET_REWARD = 2.0  # Bonus reward for being in target zone
+TARGET_SPAWN_DELAY = 50  # Steps before target zone appears (after wind starts)
+
+# Stabilization and shaping
+DRAG_COEFF = 0.3  # Linear velocity drag coefficient
+SPEED_PENALTY_COEFF = 0.05  # Penalize high speeds to encourage smooth control
+EDGE_MARGIN = 0.06  # Margin near boundaries where penalty increases
+EDGE_PENALTY_COEFF = 0.5  # Strength of boundary proximity penalty
+
+
+class DroneWindEnv(gym.Env):
+    """
+    A 2D drone environment with dynamic wind.
+    
+    Observation: [x, y, vx, vy, wind_x, wind_y]
+    Action: Discrete(5) - 0: no thrust, 1: up, 2: down, 3: left, 4: right
+    """
+    
+    metadata = {"render_modes": ["human", "rgb_array"], "render_fps": 10}
+    
+    def __init__(self):
+        super().__init__()
+        
+        # Observation space: [x, y, vx, vy, wind_x, wind_y]
+        self.observation_space = spaces.Box(
+            low=np.array([POSITION_MIN, POSITION_MIN, -MAX_VEL, -MAX_VEL, -WIND_MAX, -WIND_MAX], dtype=np.float32),
+            high=np.array([POSITION_MAX, POSITION_MAX, MAX_VEL, MAX_VEL, WIND_MAX, WIND_MAX], dtype=np.float32),
+            dtype=np.float32
+        )
+        
+        # Action space: 5 discrete thrust directions
+        self.action_space = spaces.Discrete(5)
+        
+        # Internal state
+        self.x: float = 0.0
+        self.y: float = 0.0
+        self.vx: float = 0.0
+        self.vy: float = 0.0
+        self.wind_x: float = 0.0
+        self.wind_y: float = 0.0
+        self.wind_target_x: float = 0.0
+        self.wind_target_y: float = 0.0
+        self.step_count: int = 0
+        
+    def reset(
+        self, 
+        *, 
+        seed: Optional[int] = None, 
+        options: Optional[Dict[str, Any]] = None
+    ) -> Tuple[np.ndarray, Dict[str, Any]]:
+        """
+        Reset the environment to initial state.
+        
+        Args:
+            seed: Optional random seed
+            options: Optional reset options
+            
+        Returns:
+            observation: Initial observation array
+            info: Empty info dict
+        """
+        # Always call super().reset to ensure seeding and np_random are initialized
+        super().reset(seed=seed)
+        
+        # Initialize state
+        self.x = 0.1
+        self.y = 0.5
+        self.vx = 0.0
+        self.vy = 0.0
+        self.wind_x = 0.0
+        self.wind_y = 0.0
+        self.wind_target_x = 0.0
+        self.wind_target_y = 0.0
+        self.step_count = 0
+        
+        # Build observation
+        obs = self._get_observation()
+        info = {}
+        
+        return obs, info
+    
+    def step(self, action: int) -> Tuple[np.ndarray, float, bool, bool, Dict[str, Any]]:
+        """
+        Execute one environment step.
+        
+        Args:
+            action: Discrete action (0-4)
+            
+        Returns:
+            observation: New observation array
+            reward: Reward for this step
+            terminated: Whether episode ended due to boundary crash
+            truncated: Whether episode ended due to max steps
+            info: Info dict with step_count
+        """
+        # Increment step count
+        self.step_count += 1
+        
+        # Update wind model
+        self._update_wind()
+        
+        # Apply physics update
+        self._apply_physics(action)
+        
+        # Compute reward
+        base_reward = 1.0  # Survival reward
+        
+        # Check if drone is in target zone (only if target has spawned)
+        target_spawned = self.step_count >= TARGET_SPAWN_DELAY
+        in_target = False
+        if target_spawned:
+            in_target = (
+                TARGET_X_MIN <= self.x <= TARGET_X_MAX and
+                TARGET_Y_MIN <= self.y <= TARGET_Y_MAX
+            )
+        target_bonus = TARGET_REWARD if in_target else 0.0
+        
+        # Speed penalty (discourage excessive velocity)
+        speed_sq = self.vx * self.vx + self.vy * self.vy
+        speed_penalty = -SPEED_PENALTY_COEFF * float(speed_sq)
+        # Boundary proximity penalty (discourage hovering near walls)
+        dist_left = self.x - POSITION_MIN
+        dist_right = POSITION_MAX - self.x
+        dist_bottom = self.y - POSITION_MIN
+        dist_top = POSITION_MAX - self.y
+        min_dist = min(dist_left, dist_right, dist_bottom, dist_top)
+        edge_penalty = 0.0
+        if min_dist < EDGE_MARGIN:
+            edge_penalty = -EDGE_PENALTY_COEFF * (EDGE_MARGIN - float(min_dist)) / EDGE_MARGIN
+        
+        reward = base_reward + target_bonus + speed_penalty + edge_penalty
+        
+        # Check termination (boundary crash)
+        terminated = (
+            self.x <= POSITION_MIN or 
+            self.x >= POSITION_MAX or 
+            self.y <= POSITION_MIN or 
+            self.y >= POSITION_MAX
+        )
+        
+        # Check truncation (max steps)
+        truncated = self.step_count >= MAX_STEPS
+        
+        # Build observation
+        obs = self._get_observation()
+        
+        # Check if in target zone (only if target has spawned)
+        target_spawned = self.step_count >= TARGET_SPAWN_DELAY
+        in_target = False
+        if target_spawned:
+            in_target = (
+                TARGET_X_MIN <= self.x <= TARGET_X_MAX and
+                TARGET_Y_MIN <= self.y <= TARGET_Y_MAX
+            )
+        info = {"step_count": self.step_count, "in_target": in_target, "target_spawned": target_spawned}
+        
+        return obs, reward, terminated, truncated, info
+    
+    def _update_wind(self) -> None:
+        """Update wind by smoothly moving toward target, resampling target periodically."""
+        # Resample wind target every WIND_TARGET_INTERVAL steps
+        if self.step_count % WIND_TARGET_INTERVAL == 0:
+            self.wind_target_x = self.np_random.uniform(-WIND_MAX, WIND_MAX)
+            self.wind_target_y = self.np_random.uniform(-WIND_MAX, WIND_MAX)
+        
+        # Smoothly interpolate wind toward target
+        self.wind_x += WIND_SMOOTHING * (self.wind_target_x - self.wind_x)
+        self.wind_y += WIND_SMOOTHING * (self.wind_target_y - self.wind_y)
+        
+        # Clamp wind to bounds
+        self.wind_x = np.clip(self.wind_x, -WIND_MAX, WIND_MAX)
+        self.wind_y = np.clip(self.wind_y, -WIND_MAX, WIND_MAX)
+    
+    def _apply_physics(self, action: int) -> None:
+        """Apply physics update: convert action to thrust, update velocity and position."""
+        # Convert action to thrust vector
+        if action == 0:  # No thrust
+            ax, ay = 0.0, 0.0
+        elif action == 1:  # Thrust up
+            ax, ay = 0.0, THRUST
+        elif action == 2:  # Thrust down
+            ax, ay = 0.0, -THRUST
+        elif action == 3:  # Thrust left
+            ax, ay = -THRUST, 0.0
+        elif action == 4:  # Thrust right
+            ax, ay = THRUST, 0.0
+        else:
+            raise ValueError(f"Invalid action: {action}. Must be in [0, 4]")
+        
+        # Update velocity with thrust and wind
+        self.vx = self.vx + ax + self.wind_x * DT
+        self.vy = self.vy + ay + self.wind_y * DT
+        # Apply linear drag (proportional to velocity) for stability
+        self.vx -= DRAG_COEFF * self.vx * DT
+        self.vy -= DRAG_COEFF * self.vy * DT
+        
+        # Clamp velocity
+        self.vx = np.clip(self.vx, -MAX_VEL, MAX_VEL)
+        self.vy = np.clip(self.vy, -MAX_VEL, MAX_VEL)
+        
+        # Update position
+        self.x = self.x + self.vx * DT
+        self.y = self.y + self.vy * DT
+        
+        # Clamp position to bounds
+        self.x = np.clip(self.x, POSITION_MIN, POSITION_MAX)
+        self.y = np.clip(self.y, POSITION_MIN, POSITION_MAX)
+    
+    def _get_observation(self) -> np.ndarray:
+        """Build observation array from current state."""
+        return np.array(
+            [self.x, self.y, self.vx, self.vy, self.wind_x, self.wind_y],
+            dtype=np.float32
+        )
+    
+    def render(self) -> None:
+        """
+        Render the environment state (stub implementation for Phase 1).
+        Prints state to stdout.
+        """
+        print(
+            f"Step {self.step_count}: "
+            f"x={self.x:.2f}, y={self.y:.2f}, "
+            f"vx={self.vx:.2f}, vy={self.vy:.2f}, "
+            f"wind=({self.wind_x:.2f}, {self.wind_y:.2f})"
+        )
+
+
+def make_drone_env() -> DroneWindEnv:
+    """Helper function to create a DroneWindEnv instance."""
+    return DroneWindEnv()
+
+
+if __name__ == "__main__":
+    # Manual test block
+    print("Testing DroneWindEnv...")
+    print("=" * 60)
+    
+    env = make_drone_env()
+    obs, info = env.reset(seed=42)
+    print(f"Initial observation: {obs}")
+    print()
+    
+    for t in range(200):
+        action = env.action_space.sample()
+        obs, reward, terminated, truncated, info = env.step(action)
+        env.render()
+        
+        if terminated:
+            print(f"\nEpisode terminated at step {t} (boundary crash)")
+            break
+        if truncated:
+            print(f"\nEpisode truncated at step {t} (max steps reached)")
+            break
+    
+    print("=" * 60)
+    print("Test completed!")
+

eval/__init__.py

@@ -0,0 +1,2 @@
+"""Evaluation scripts and utilities."""
+

eval/eval_liquid_policy.py

@@ -0,0 +1,177 @@
+"""
+Evaluate a trained PPO Liquid Neural Network policy on the DroneWindEnv environment.
+
+This script loads a saved PPO model with liquid policy and runs evaluation episodes,
+printing statistics about average reward and episode length.
+"""
+
+import os
+import sys
+import argparse
+import numpy as np
+from stable_baselines3 import PPO
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from env.drone_env import DroneWindEnv
+
+
+def main():
+    """Main evaluation function."""
+    parser = argparse.ArgumentParser(description="Evaluate PPO Liquid NN agent on DroneWindEnv")
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default="models/liquid_policy.zip",
+        help="Path to the trained model (default: models/liquid_policy.zip)"
+    )
+    parser.add_argument(
+        "--episodes",
+        type=int,
+        default=10,
+        help="Number of evaluation episodes (default: 10)"
+    )
+    parser.add_argument(
+        "--render",
+        action="store_true",
+        help="Print environment state to console during evaluation"
+    )
+    parser.add_argument(
+        "--seed",
+        type=int,
+        default=None,
+        help="Random seed for evaluation (default: None)"
+    )
+    
+    args = parser.parse_args()
+    
+    print("=" * 60)
+    print("Evaluating PPO Liquid NN Agent on DroneWindEnv")
+    print("=" * 60)
+    print(f"Model path: {args.model_path}")
+    print(f"Number of episodes: {args.episodes}")
+    print("=" * 60)
+    
+    # Check if model file exists
+    if not os.path.exists(args.model_path):
+        print(f"\nError: Model file not found at {args.model_path}")
+        print("Please train a model first using:")
+        print("  python train/train_liquid_ppo.py")
+        return
+    
+    # Create environment
+    print("\nCreating environment...")
+    env = DroneWindEnv()
+    
+    # Load the model
+    print(f"Loading model from {args.model_path}...")
+    try:
+        model = PPO.load(args.model_path, env=env)
+        print("Model loaded successfully!")
+    except Exception as e:
+        print(f"\nError loading model: {e}")
+        return
+    
+    # Run evaluation episodes
+    print(f"\nRunning {args.episodes} evaluation episodes...")
+    print("-" * 60)
+    
+    rewards = []
+    episode_lengths = []
+    
+    for episode in range(args.episodes):
+        obs, info = env.reset(seed=args.seed)
+        done = False
+        truncated = False
+        total_reward = 0.0
+        step_count = 0
+        
+        if args.render:
+            print(f"\nEpisode {episode + 1}:")
+            env.render()
+        
+        while not (done or truncated):
+            # Get action from the model (deterministic)
+            action, _ = model.predict(obs, deterministic=True)
+            
+            # Step the environment
+            obs, reward, done, truncated, info = env.step(action)
+            
+            total_reward += reward
+            step_count += 1
+            
+            if args.render:
+                env.render()
+        
+        rewards.append(total_reward)
+        episode_lengths.append(step_count)
+        
+        status = "terminated" if done else "truncated"
+        print(f"Episode {episode + 1}: Reward = {total_reward:.2f}, "
+              f"Length = {step_count} steps ({status})")
+    
+    # Print statistics
+    print("\n" + "=" * 60)
+    print("Evaluation Results")
+    print("=" * 60)
+    print(f"Average reward: {np.mean(rewards):.2f} ± {np.std(rewards):.2f}")
+    print(f"Average episode length: {np.mean(episode_lengths):.1f} ± {np.std(episode_lengths):.1f}")
+    print(f"Average survival time: {np.mean(episode_lengths):.1f} steps")
+    print(f"Min reward: {np.min(rewards):.2f}")
+    print(f"Max reward: {np.max(rewards):.2f}")
+    print(f"Min episode length: {np.min(episode_lengths)}")
+    print(f"Max episode length: {np.max(episode_lengths)}")
+    print("=" * 60)
+    
+    # Print per-episode rewards
+    print("\nPer-episode rewards:")
+    for i, reward in enumerate(rewards, 1):
+        print(f"  Episode {i}: {reward:.2f}")
+    
+    # Optional: Try to plot if matplotlib is available
+    try:
+        import matplotlib.pyplot as plt
+        
+        plt.figure(figsize=(10, 5))
+        
+        # Plot 1: Episode rewards
+        plt.subplot(1, 2, 1)
+        plt.plot(range(1, len(rewards) + 1), rewards, 'o-', linewidth=2, markersize=6)
+        plt.axhline(y=np.mean(rewards), color='r', linestyle='--', label=f'Mean: {np.mean(rewards):.2f}')
+        plt.xlabel('Episode')
+        plt.ylabel('Total Reward')
+        plt.title('Episode Rewards (Liquid NN)')
+        plt.grid(True, alpha=0.3)
+        plt.legend()
+        
+        # Plot 2: Episode lengths
+        plt.subplot(1, 2, 2)
+        plt.plot(range(1, len(episode_lengths) + 1), episode_lengths, 's-', 
+                linewidth=2, markersize=6, color='green')
+        plt.axhline(y=np.mean(episode_lengths), color='r', linestyle='--', 
+                   label=f'Mean: {np.mean(episode_lengths):.1f}')
+        plt.xlabel('Episode')
+        plt.ylabel('Episode Length')
+        plt.title('Episode Lengths (Liquid NN)')
+        plt.grid(True, alpha=0.3)
+        plt.legend()
+        
+        plt.tight_layout()
+        plt.savefig('eval_liquid_results.png', dpi=150, bbox_inches='tight')
+        print("\n✓ Evaluation plots saved to eval_liquid_results.png")
+        print("  (Close the plot window to continue)")
+        plt.show(block=False)
+        plt.pause(2)  # Show for 2 seconds
+        plt.close()
+        
+    except ImportError:
+        # Matplotlib not available, skip plotting
+        pass
+    except Exception as e:
+        print(f"\nNote: Could not generate plots: {e}")
+
+
+if __name__ == "__main__":
+    main()
+

eval/eval_mlp_baseline.py

@@ -0,0 +1,176 @@
+"""
+Evaluate a trained PPO MLP baseline on the DroneWindEnv environment.
+
+This script loads a saved PPO model and runs evaluation episodes,
+printing statistics about average reward and episode length.
+"""
+
+import os
+import sys
+import argparse
+import numpy as np
+from stable_baselines3 import PPO
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from env.drone_env import DroneWindEnv
+
+
+def main():
+    """Main evaluation function."""
+    parser = argparse.ArgumentParser(description="Evaluate PPO agent on DroneWindEnv")
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default="models/mlp_baseline.zip",
+        help="Path to the trained model (default: models/mlp_baseline.zip)"
+    )
+    parser.add_argument(
+        "--episodes",
+        type=int,
+        default=10,
+        help="Number of evaluation episodes (default: 10)"
+    )
+    parser.add_argument(
+        "--render",
+        action="store_true",
+        help="Print environment state to console during evaluation"
+    )
+    parser.add_argument(
+        "--seed",
+        type=int,
+        default=None,
+        help="Random seed for evaluation (default: None)"
+    )
+    
+    args = parser.parse_args()
+    
+    print("=" * 60)
+    print("Evaluating PPO Agent on DroneWindEnv")
+    print("=" * 60)
+    print(f"Model path: {args.model_path}")
+    print(f"Number of episodes: {args.episodes}")
+    print("=" * 60)
+    
+    # Check if model file exists
+    if not os.path.exists(args.model_path):
+        print(f"\nError: Model file not found at {args.model_path}")
+        print("Please train a model first using:")
+        print("  python train/train_mlp_ppo.py")
+        return
+    
+    # Create environment
+    print("\nCreating environment...")
+    env = DroneWindEnv()
+    
+    # Load the model
+    print(f"Loading model from {args.model_path}...")
+    try:
+        model = PPO.load(args.model_path, env=env)
+        print("Model loaded successfully!")
+    except Exception as e:
+        print(f"\nError loading model: {e}")
+        return
+    
+    # Run evaluation episodes
+    print(f"\nRunning {args.episodes} evaluation episodes...")
+    print("-" * 60)
+    
+    rewards = []
+    episode_lengths = []
+    
+    for episode in range(args.episodes):
+        obs, info = env.reset(seed=args.seed)
+        done = False
+        truncated = False
+        total_reward = 0.0
+        step_count = 0
+        
+        if args.render:
+            print(f"\nEpisode {episode + 1}:")
+            env.render()
+        
+        while not (done or truncated):
+            # Get action from the model (deterministic)
+            action, _ = model.predict(obs, deterministic=True)
+            
+            # Step the environment
+            obs, reward, done, truncated, info = env.step(action)
+            
+            total_reward += reward
+            step_count += 1
+            
+            if args.render:
+                env.render()
+        
+        rewards.append(total_reward)
+        episode_lengths.append(step_count)
+        
+        status = "terminated" if done else "truncated"
+        print(f"Episode {episode + 1}: Reward = {total_reward:.2f}, "
+              f"Length = {step_count} steps ({status})")
+    
+    # Print statistics
+    print("\n" + "=" * 60)
+    print("Evaluation Results")
+    print("=" * 60)
+    print(f"Average reward: {np.mean(rewards):.2f} ± {np.std(rewards):.2f}")
+    print(f"Average episode length: {np.mean(episode_lengths):.1f} ± {np.std(episode_lengths):.1f}")
+    print(f"Min reward: {np.min(rewards):.2f}")
+    print(f"Max reward: {np.max(rewards):.2f}")
+    print(f"Min episode length: {np.min(episode_lengths)}")
+    print(f"Max episode length: {np.max(episode_lengths)}")
+    print("=" * 60)
+    
+    # Print per-episode rewards
+    print("\nPer-episode rewards:")
+    for i, reward in enumerate(rewards, 1):
+        print(f"  Episode {i}: {reward:.2f}")
+    
+    # Optional: Try to plot if matplotlib is available
+    try:
+        import matplotlib.pyplot as plt
+        
+        plt.figure(figsize=(10, 5))
+        
+        # Plot 1: Episode rewards
+        plt.subplot(1, 2, 1)
+        plt.plot(range(1, len(rewards) + 1), rewards, 'o-', linewidth=2, markersize=6)
+        plt.axhline(y=np.mean(rewards), color='r', linestyle='--', label=f'Mean: {np.mean(rewards):.2f}')
+        plt.xlabel('Episode')
+        plt.ylabel('Total Reward')
+        plt.title('Episode Rewards')
+        plt.grid(True, alpha=0.3)
+        plt.legend()
+        
+        # Plot 2: Episode lengths
+        plt.subplot(1, 2, 2)
+        plt.plot(range(1, len(episode_lengths) + 1), episode_lengths, 's-', 
+                linewidth=2, markersize=6, color='green')
+        plt.axhline(y=np.mean(episode_lengths), color='r', linestyle='--', 
+                   label=f'Mean: {np.mean(episode_lengths):.1f}')
+        plt.xlabel('Episode')
+        plt.ylabel('Episode Length')
+        plt.title('Episode Lengths')
+        plt.grid(True, alpha=0.3)
+        plt.legend()
+        
+        plt.tight_layout()
+        plt.savefig('eval_results.png', dpi=150, bbox_inches='tight')
+        print("\n✓ Evaluation plots saved to eval_results.png")
+        print("  (Close the plot window to continue)")
+        plt.show(block=False)
+        plt.pause(2)  # Show for 2 seconds
+        plt.close()
+        
+    except ImportError:
+        # Matplotlib not available, skip plotting
+        pass
+    except Exception as e:
+        print(f"\nNote: Could not generate plots: {e}")
+
+
+if __name__ == "__main__":
+    main()
+

install_log.txt

@@ -0,0 +1,197 @@
+Defaulting to user installation because normal site-packages is not writeable
+Collecting PyFlyt
+  Downloading pyflyt-0.29.0-py3-none-any.whl.metadata (5.2 kB)
+Downloading pyflyt-0.29.0-py3-none-any.whl (215 kB)
+Installing collected packages: PyFlyt
+Successfully installed PyFlyt-0.29.0
+Defaulting to user installation because normal site-packages is not writeable
+Requirement already satisfied: numpy in /usr/lib64/python3.14/site-packages (2.3.5)
+Collecting gymnasium
+  Using cached gymnasium-1.2.2-py3-none-any.whl.metadata (10 kB)
+Collecting opensimplex
+  Downloading opensimplex-0.4.5.1-py3-none-any.whl.metadata (10 kB)
+Collecting ncps
+  Downloading ncps-1.0.1-py3-none-any.whl.metadata (702 bytes)
+Collecting stable-baselines3
+  Downloading stable_baselines3-2.7.0-py3-none-any.whl.metadata (4.8 kB)
+Collecting rich
+  Downloading rich-14.2.0-py3-none-any.whl.metadata (18 kB)
+Collecting tqdm
+  Downloading tqdm-4.67.1-py3-none-any.whl.metadata (57 kB)
+Collecting cloudpickle>=1.2.0 (from gymnasium)
+  Downloading cloudpickle-3.1.2-py3-none-any.whl.metadata (7.1 kB)
+Requirement already satisfied: typing-extensions>=4.3.0 in /usr/lib/python3.14/site-packages (from gymnasium) (4.15.0)
+Collecting farama-notifications>=0.0.1 (from gymnasium)
+  Downloading Farama_Notifications-0.0.4-py3-none-any.whl.metadata (558 bytes)
+Collecting future (from ncps)
+  Downloading future-1.0.0-py3-none-any.whl.metadata (4.0 kB)
+Requirement already satisfied: packaging in /usr/lib/python3.14/site-packages (from ncps) (25.0)
+Collecting torch<3.0,>=2.3 (from stable-baselines3)
+  Using cached torch-2.9.1-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (30 kB)
+Collecting pandas (from stable-baselines3)
+  Downloading pandas-2.3.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.metadata (91 kB)
+Collecting matplotlib (from stable-baselines3)
+  Downloading matplotlib-3.10.7-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (11 kB)
+Collecting filelock (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading filelock-3.20.0-py3-none-any.whl.metadata (2.1 kB)
+Requirement already satisfied: setuptools in /usr/lib/python3.14/site-packages (from torch<3.0,>=2.3->stable-baselines3) (78.1.1)
+Collecting sympy>=1.13.3 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading sympy-1.14.0-py3-none-any.whl.metadata (12 kB)
+Collecting networkx>=2.5.1 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading networkx-3.6-py3-none-any.whl.metadata (6.8 kB)
+Requirement already satisfied: jinja2 in /usr/lib/python3.14/site-packages (from torch<3.0,>=2.3->stable-baselines3) (3.1.6)
+Collecting fsspec>=0.8.5 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading fsspec-2025.10.0-py3-none-any.whl.metadata (10 kB)
+Collecting nvidia-cuda-nvrtc-cu12==12.8.93 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-cuda-runtime-cu12==12.8.90 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-cuda-cupti-cu12==12.8.90 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-cudnn-cu12==9.10.2.21 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
+Collecting nvidia-cublas-cu12==12.8.4.1 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-cufft-cu12==11.3.3.83 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-curand-cu12==10.3.9.90 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-cusolver-cu12==11.7.3.90 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl.metadata (1.8 kB)
+Collecting nvidia-cusparse-cu12==12.5.8.93 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
+Collecting nvidia-cusparselt-cu12==0.7.1 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl.metadata (7.0 kB)
+Collecting nvidia-nccl-cu12==2.27.5 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_nccl_cu12-2.27.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.0 kB)
+Collecting nvidia-nvshmem-cu12==3.3.20 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_nvshmem_cu12-3.3.20-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.1 kB)
+Collecting nvidia-nvtx-cu12==12.8.90 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.8 kB)
+Collecting nvidia-nvjitlink-cu12==12.8.93 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl.metadata (1.7 kB)
+Collecting nvidia-cufile-cu12==1.13.1.3 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (1.7 kB)
+Collecting triton==3.5.1 (from torch<3.0,>=2.3->stable-baselines3)
+  Downloading triton-3.5.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (1.7 kB)
+Collecting markdown-it-py>=2.2.0 (from rich)
+  Downloading markdown_it_py-4.0.0-py3-none-any.whl.metadata (7.3 kB)
+Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/lib/python3.14/site-packages (from rich) (2.19.1)
+Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich)
+  Downloading mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
+Collecting mpmath<1.4,>=1.1.0 (from sympy>=1.13.3->torch<3.0,>=2.3->stable-baselines3)
+  Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
+Requirement already satisfied: MarkupSafe>=2.0 in /usr/lib64/python3.14/site-packages (from jinja2->torch<3.0,>=2.3->stable-baselines3) (3.0.2)
+Collecting contourpy>=1.0.1 (from matplotlib->stable-baselines3)
+  Downloading contourpy-1.3.3-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (5.5 kB)
+Collecting cycler>=0.10 (from matplotlib->stable-baselines3)
+  Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
+Collecting fonttools>=4.22.0 (from matplotlib->stable-baselines3)
+  Downloading fonttools-4.61.0-cp314-cp314-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl.metadata (113 kB)
+Collecting kiwisolver>=1.3.1 (from matplotlib->stable-baselines3)
+  Downloading kiwisolver-1.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (6.3 kB)
+Requirement already satisfied: pillow>=8 in /usr/lib64/python3.14/site-packages (from matplotlib->stable-baselines3) (11.3.0)
+Requirement already satisfied: pyparsing>=3 in /usr/lib/python3.14/site-packages (from matplotlib->stable-baselines3) (3.1.2)
+Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.14/site-packages (from matplotlib->stable-baselines3) (2.9.0.post0)
+Requirement already satisfied: six>=1.5 in /usr/lib/python3.14/site-packages (from python-dateutil>=2.7->matplotlib->stable-baselines3) (1.17.0)
+Collecting pytz>=2020.1 (from pandas->stable-baselines3)
+  Downloading pytz-2025.2-py2.py3-none-any.whl.metadata (22 kB)
+Collecting tzdata>=2022.7 (from pandas->stable-baselines3)
+  Downloading tzdata-2025.2-py2.py3-none-any.whl.metadata (1.4 kB)
+Downloading gymnasium-1.2.2-py3-none-any.whl (952 kB)
+   ━━ 9… 1… eta 0:0…
+      kB M…         
+Downloading opensimplex-0.4.5.1-py3-none-any.whl (267 kB)
+Downloading ncps-1.0.1-py3-none-any.whl (60 kB)
+Downloading stable_baselines3-2.7.0-py3-none-any.whl (187 kB)
+Downloading torch-2.9.1-cp314-cp314-manylinux_2_28_x86_64.whl (899.7 MB)
+   ━━ 8… 2… eta 0:0…
+      MB M…         
+Downloading nvidia_cublas_cu12-12.8.4.1-py3-none-manylinux_2_27_x86_64.whl (594.3 MB)
+   ━━ 5… 2… eta 0:0…
+      MB M…         
+Downloading nvidia_cuda_cupti_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (10.2 MB)
+   ━━ 1… 3… eta 0:0…
+      MB M…         
+Downloading nvidia_cuda_nvrtc_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (88.0 MB)
+   ━━ 8… 3… eta 0:0…
+      MB M…         
+Downloading nvidia_cuda_runtime_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (954 kB)
+   ━━ 9… 2… eta 0:0…
+      kB M…         
+Downloading nvidia_cudnn_cu12-9.10.2.21-py3-none-manylinux_2_27_x86_64.whl (706.8 MB)
+   ━━ 7… 2… eta 0:0…
+      MB M…         
+Downloading nvidia_cufft_cu12-11.3.3.83-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (193.1 MB)
+   ━━ 1… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_cufile_cu12-1.13.1.3-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.2 MB)
+   ━━ 1… 7… eta 0:0…
+      MB M…         
+Downloading nvidia_curand_cu12-10.3.9.90-py3-none-manylinux_2_27_x86_64.whl (63.6 MB)
+   ━━ 6… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_cusolver_cu12-11.7.3.90-py3-none-manylinux_2_27_x86_64.whl (267.5 MB)
+   ━━ 2… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_cusparse_cu12-12.5.8.93-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (288.2 MB)
+   ━━ 2… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_cusparselt_cu12-0.7.1-py3-none-manylinux2014_x86_64.whl (287.2 MB)
+   ━━ 2… 2… eta 0:0…
+      MB M…         
+Downloading nvidia_nccl_cu12-2.27.5-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (322.3 MB)
+   ━━ 3… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_nvjitlink_cu12-12.8.93-py3-none-manylinux2010_x86_64.manylinux_2_12_x86_64.whl (39.3 MB)
+   ━━ 3… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_nvshmem_cu12-3.3.20-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (124.7 MB)
+   ━━ 1… 1… eta 0:0…
+      MB M…         
+Downloading nvidia_nvtx_cu12-12.8.90-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (89 kB)
+Downloading triton-3.5.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (170.5 MB)
+   ━━ 1… 3… eta 0:0…
+      MB M…         
+Downloading rich-14.2.0-py3-none-any.whl (243 kB)
+Downloading tqdm-4.67.1-py3-none-any.whl (78 kB)
+Downloading cloudpickle-3.1.2-py3-none-any.whl (22 kB)
+Downloading Farama_Notifications-0.0.4-py3-none-any.whl (2.5 kB)
+Downloading fsspec-2025.10.0-py3-none-any.whl (200 kB)
+Downloading markdown_it_py-4.0.0-py3-none-any.whl (87 kB)
+Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
+Downloading networkx-3.6-py3-none-any.whl (2.1 MB)
+   ━━ 2… 3… eta 0:0…
+      MB M…         
+Downloading sympy-1.14.0-py3-none-any.whl (6.3 MB)
+   ━━ 6… 3… eta 0:0…
+      MB M…         
+Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
+   ━━ 5… 1… eta 0:0…
+      kB M…         
+Downloading filelock-3.20.0-py3-none-any.whl (16 kB)
+Downloading future-1.0.0-py3-none-any.whl (491 kB)
+Downloading matplotlib-3.10.7-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (9.8 MB)
+   ━━ 9… 3… eta 0:0…
+      MB M…         
+Downloading contourpy-1.3.3-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (363 kB)
+Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB)
+Downloading fonttools-4.61.0-cp314-cp314-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl (4.9 MB)
+   ━━ 4… 3… eta 0:0…
+      MB M…         
+Downloading kiwisolver-1.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (1.5 MB)
+   ━━ 1… 2… eta 0:0…
+      MB M…         
+Downloading pandas-2.3.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (12.3 MB)
+   ━━ 1… 3… eta 0:0…
+      MB M…         
+Downloading pytz-2025.2-py2.py3-none-any.whl (509 kB)
+Downloading tzdata-2025.2-py2.py3-none-any.whl (347 kB)
+Installing collected packages: pytz, nvidia-cusparselt-cu12, mpmath, farama-notifications, tzdata, triton, tqdm, sympy, opensimplex, nvidia-nvtx-cu12, nvidia-nvshmem-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufile-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, networkx, mdurl, kiwisolver, future, fsspec, fonttools, filelock, cycler, contourpy, cloudpickle, pandas, nvidia-cusparse-cu12, nvidia-cufft-cu12, nvidia-cudnn-cu12, ncps, matplotlib, markdown-it-py, gymnasium, rich, nvidia-cusolver-cu12, torch, stable-baselines3
+
+ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
+pyflyt 0.29.0 requires numba, which is not installed.
+pyflyt 0.29.0 requires pettingzoo, which is not installed.
+pyflyt 0.29.0 requires pybullet, which is not installed.
+pyflyt 0.29.0 requires numpy<2.0.0, but you have numpy 2.3.5 which is incompatible.
+Successfully installed cloudpickle-3.1.2 contourpy-1.3.3 cycler-0.12.1 farama-notifications-0.0.4 filelock-3.20.0 fonttools-4.61.0 fsspec-2025.10.0 future-1.0.0 gymnasium-1.2.2 kiwisolver-1.4.9 markdown-it-py-4.0.0 matplotlib-3.10.7 mdurl-0.1.2 mpmath-1.3.0 ncps-1.0.1 networkx-3.6 nvidia-cublas-cu12-12.8.4.1 nvidia-cuda-cupti-cu12-12.8.90 nvidia-cuda-nvrtc-cu12-12.8.93 nvidia-cuda-runtime-cu12-12.8.90 nvidia-cudnn-cu12-9.10.2.21 nvidia-cufft-cu12-11.3.3.83 nvidia-cufile-cu12-1.13.1.3 nvidia-curand-cu12-10.3.9.90 nvidia-cusolver-cu12-11.7.3.90 nvidia-cusparse-cu12-12.5.8.93 nvidia-cusparselt-cu12-0.7.1 nvidia-nccl-cu12-2.27.5 nvidia-nvjitlink-cu12-12.8.93 nvidia-nvshmem-cu12-3.3.20 nvidia-nvtx-cu12-12.8.90 opensimplex-0.4.5.1 pandas-2.3.3 pytz-2025.2 rich-14.2.0 stable-baselines3-2.7.0 sympy-1.14.0 torch-2.9.1 tqdm-4.67.1 triton-3.5.1 tzdata-2025.2

interactive_log.txt

@@ -0,0 +1,8 @@
+Initializing Interactive Dashboard...
+No trained model found. Using untrained Liquid Brain.
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+
+=== DASHBOARD LIVE ===
+Close the window to exit.

main.py

@@ -0,0 +1,144 @@
+"""
+Main entry point for the Drone RL project.
+Tests imports and basic functionality.
+"""
+
+import sys
+import numpy as np
+
+print("=" * 50)
+print("Drone RL Project - Import Test")
+print("=" * 50)
+
+# Test PyTorch
+try:
+    import torch
+    print(f"✓ PyTorch {torch.__version__} imported successfully")
+    print(f"  CUDA available: {torch.cuda.is_available()}")
+except ImportError as e:
+    print(f"✗ PyTorch import failed: {e}")
+    sys.exit(1)
+
+# Test NumPy
+try:
+    print(f"✓ NumPy {np.__version__} imported successfully")
+except ImportError as e:
+    print(f"✗ NumPy import failed: {e}")
+    sys.exit(1)
+
+# Test Gymnasium
+try:
+    import gymnasium as gym
+    print(f"✓ Gymnasium {gym.__version__} imported successfully")
+    
+    # Test creating a simple environment
+    try:
+        env = gym.make("CartPole-v1")
+        print(f"✓ Gymnasium environment 'CartPole-v1' created successfully")
+        env.close()
+    except Exception as e:
+        print(f"⚠ Could not create test environment: {e}")
+except ImportError as e:
+    print(f"✗ Gymnasium import failed: {e}")
+    sys.exit(1)
+
+# Test Stable-Baselines3
+try:
+    import stable_baselines3
+    print(f"✓ Stable-Baselines3 imported successfully")
+except ImportError as e:
+    print(f"⚠ Stable-Baselines3 import failed: {e}")
+    print("  (This is optional, you can use cleanrl instead)")
+
+# Test Pygame
+pygame_works = False
+try:
+    import pygame
+    print(f"✓ Pygame {pygame.__version__} imported successfully")
+    
+    # Try to initialize pygame
+    try:
+        pygame.init()
+        print("✓ Pygame initialized successfully")
+        
+        # Create a test window
+        screen = pygame.display.set_mode((640, 480))
+        pygame.display.set_caption("Drone RL - Pygame Test")
+        print("✓ Pygame window created successfully")
+        print("  Window should be visible. Close it to continue...")
+        
+        # Keep window open briefly, then close
+        import time
+        clock = pygame.time.Clock()
+        running = True
+        start_time = time.time()
+        
+        while running and (time.time() - start_time) < 2.0:  # Show for 2 seconds
+            for event in pygame.event.get():
+                if event.type == pygame.QUIT:
+                    running = False
+            
+            screen.fill((50, 50, 50))  # Dark gray background
+            font = pygame.font.Font(None, 36)
+            text = font.render("Pygame Works!", True, (255, 255, 255))
+            text_rect = text.get_rect(center=(320, 240))
+            screen.blit(text, text_rect)
+            pygame.display.flip()
+            clock.tick(60)
+        
+        pygame.quit()
+        pygame_works = True
+        print("✓ Pygame window test completed successfully")
+        
+    except Exception as e:
+        print(f"⚠ Pygame initialization failed: {e}")
+        pygame.quit()
+        
+except ImportError as e:
+    print(f"✗ Pygame import failed: {e}")
+
+# Fallback to matplotlib if pygame failed
+if not pygame_works:
+    print("\n" + "=" * 50)
+    print("Falling back to matplotlib animation...")
+    print("=" * 50)
+    try:
+        import matplotlib.pyplot as plt
+        import matplotlib.animation as animation
+        
+        print("✓ Matplotlib imported successfully")
+        
+        # Create a simple animation
+        fig, ax = plt.subplots()
+        ax.set_xlim(0, 10)
+        ax.set_ylim(0, 10)
+        ax.set_title("Drone RL - Matplotlib Test")
+        
+        line, = ax.plot([], [], 'o-', lw=2)
+        
+        def animate(frame):
+            x = np.linspace(0, 10, 100)
+            y = np.sin(x + frame * 0.1) * 5 + 5
+            line.set_data(x, y)
+            return line,
+        
+        anim = animation.FuncAnimation(fig, animate, frames=100, interval=50, blit=True)
+        print("✓ Matplotlib animation created")
+        print("  Animation window should be visible. Close it to continue...")
+        plt.show(block=False)
+        
+        # Keep it open briefly
+        import time
+        time.sleep(2)
+        plt.close()
+        
+        print("✓ Matplotlib animation test completed successfully")
+        
+    except ImportError as e:
+        print(f"✗ Matplotlib import failed: {e}")
+        print("  Please install matplotlib: pip install matplotlib")
+
+print("\n" + "=" * 50)
+print("All tests completed!")
+print("=" * 50)
+

models/__init__.py

@@ -0,0 +1,2 @@
+"""Model definitions and architectures."""
+

models/liquid_cell.py

@@ -0,0 +1,86 @@
+"""
+Liquid Neural Network Cell - Discrete-time approximation of continuous-time dynamics.
+
+Implements a liquid cell with learnable per-neuron time constants.
+The cell updates hidden state using a differential equation approximation.
+"""
+
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+
+class LiquidCell(nn.Module):
+    """
+    A discrete-time liquid neural network cell.
+    
+    Hidden state update rule:
+        h_{t+1,i} = h_{t,i} + dt / tau_i * ( tanh( W_hh[i]·h_t + W_xh[i]·x_t + b[i] ) - h_{t,i} )
+    
+    where tau_i is a learnable per-neuron time constant.
+    
+    Args:
+        hidden_size: Number of hidden neurons
+        input_size: Size of input vector
+        dt: Time step for discrete approximation (default: 0.1)
+    """
+    
+    def __init__(self, hidden_size: int, input_size: int, dt: float = 0.1):
+        super().__init__()
+        self.hidden_size = hidden_size
+        self.input_size = input_size
+        self.dt = dt
+        
+        # Recurrent weight matrix: (hidden_size, hidden_size)
+        self.W_hh = nn.Parameter(torch.randn(hidden_size, hidden_size) * 0.1)
+        
+        # Input weight matrix: (hidden_size, input_size)
+        self.W_xh = nn.Parameter(torch.randn(hidden_size, input_size) * 0.1)
+        
+        # Bias vector: (hidden_size,)
+        self.b = nn.Parameter(torch.zeros(hidden_size))
+        
+        # Raw time constants (will be transformed to positive values)
+        # Shape: (hidden_size,)
+        self.tau_raw = nn.Parameter(torch.ones(hidden_size))
+        
+    def forward(self, h: torch.Tensor, x: torch.Tensor) -> torch.Tensor:
+        """
+        Forward pass through the liquid cell.
+        
+        Args:
+            h: Hidden state tensor of shape (batch, hidden_size)
+            x: Input tensor of shape (batch, input_size)
+            
+        Returns:
+            Next hidden state tensor of shape (batch, hidden_size)
+        """
+        # Compute time constants: tau = softplus(tau_raw) + 1e-3
+        # This ensures tau is always positive
+        tau = F.softplus(self.tau_raw) + 1e-3
+        
+        # Compute preactivation:
+        # preact = tanh( W_hh @ h^T + W_xh @ x^T + b )
+        # Using batch matrix multiplication
+        
+        # W_hh @ h^T: (hidden_size, hidden_size) @ (hidden_size, batch) -> (hidden_size, batch)
+        # Then transpose to (batch, hidden_size)
+        h_proj = torch.matmul(h, self.W_hh.t())  # (batch, hidden_size)
+        
+        # W_xh @ x^T: (hidden_size, input_size) @ (input_size, batch) -> (hidden_size, batch)
+        # Then transpose to (batch, hidden_size)
+        x_proj = torch.matmul(x, self.W_xh.t())  # (batch, hidden_size)
+        
+        # Add bias and apply tanh
+        preact = torch.tanh(h_proj + x_proj + self.b)  # (batch, hidden_size)
+        
+        # Update hidden state:
+        # h_next = h + dt * (preact - h) / tau
+        # tau is (hidden_size,), so we need to broadcast
+        h_next = h + self.dt * (preact - h) / tau.unsqueeze(0)  # (batch, hidden_size)
+        
+        # Clamp to reasonable range for stability
+        h_next = torch.clamp(h_next, -5.0, 5.0)
+        
+        return h_next
+

models/liquid_policy.py

@@ -0,0 +1,81 @@
+"""
+Liquid Neural Network Policy for Stable-Baselines3.
+
+Implements a custom feature extractor using LiquidCell that can be used
+with PPO and other SB3 algorithms.
+"""
+
+import torch
+import torch.nn as nn
+import gymnasium as gym
+from stable_baselines3.common.torch_layers import BaseFeaturesExtractor
+
+from models.liquid_cell import LiquidCell
+
+
+class LiquidFeatureExtractor(BaseFeaturesExtractor):
+    """
+    Feature extractor using a Liquid Neural Network cell.
+    
+    This extractor processes observations through a liquid cell to produce
+    rich temporal features suitable for policy/value networks.
+    
+    Args:
+        observation_space: Gymnasium observation space
+        features_dim: Output feature dimension (default: 32)
+        hidden_size: Number of hidden neurons in liquid cell (default: 32)
+        dt: Time step for liquid cell (default: 0.1)
+    """
+    
+    def __init__(
+        self,
+        observation_space: gym.Space,
+        features_dim: int = 32,
+        hidden_size: int = 32,
+        dt: float = 0.1,
+    ):
+        super().__init__(observation_space, features_dim)
+        
+        # Get observation dimension
+        if isinstance(observation_space, gym.spaces.Box):
+            obs_dim = observation_space.shape[0]
+        else:
+            raise ValueError(f"Unsupported observation space: {observation_space}")
+        
+        self.hidden_size = hidden_size
+        self.dt = dt
+        
+        # Input projection layer: maps observation to hidden space
+        self.input_layer = nn.Linear(obs_dim, hidden_size)
+        
+        # Liquid cell: processes hidden state
+        self.liquid_cell = LiquidCell(hidden_size, hidden_size, dt)
+        
+        # Output projection: maps liquid cell output to feature dimension
+        self.output_layer = nn.Linear(hidden_size, features_dim)
+        
+    def forward(self, observations: torch.Tensor) -> torch.Tensor:
+        """
+        Forward pass through the liquid feature extractor.
+        
+        Args:
+            observations: Input tensor of shape (batch, obs_dim)
+            
+        Returns:
+            Feature tensor of shape (batch, features_dim)
+        """
+        # Project input to hidden space and apply tanh
+        x = torch.tanh(self.input_layer(observations))  # (batch, hidden_size)
+        
+        # Initialize hidden state from input
+        h = x
+        
+        # Apply one liquid cell step
+        # The liquid cell uses both the hidden state and the input
+        h = self.liquid_cell(h, x)  # (batch, hidden_size)
+        
+        # Project to output feature dimension
+        features = self.output_layer(h)  # (batch, features_dim)
+        
+        return features
+

models/liquid_ppo.py

@@ -0,0 +1,100 @@
+import torch
+import torch.nn as nn
+import gymnasium as gym
+from stable_baselines3 import PPO
+from stable_baselines3.common.torch_layers import BaseFeaturesExtractor
+from stable_baselines3.common.env_util import make_vec_env
+from stable_baselines3.common.vec_env import SubprocVecEnv
+from ncps.torch import LTC
+from ncps.wirings import AutoNCP
+from env.drone_3d import Drone3DEnv
+
+class LTCFeatureExtractor(BaseFeaturesExtractor):
+    """
+    Custom Feature Extractor using Liquid Time-Constant (LTC) Cells.
+    This allows the agent to handle irregular time-steps and stiff dynamics better than standard MLPs or LSTMs.
+    """
+    def __init__(self, observation_space: gym.spaces.Box, features_dim: int = 32):
+        super().__init__(observation_space, features_dim)
+        
+        input_size = observation_space.shape[0]
+        # self.features_dim is already set by super().__init__
+        
+        # Neural Circuit Policy (NCP) wiring for structured connectivity
+        # We use a small wiring to keep inference fast (< 10ms)
+        # AutoNCP requires units > output_size. Let's use 48 units for 32 outputs.
+        wiring = AutoNCP(48, output_size=features_dim)
+        
+        self.ltc = LTC(input_size, wiring, batch_first=True)
+        
+        # Hidden state for the LTC
+        self.hx = None
+
+    def forward(self, observations: torch.Tensor) -> torch.Tensor:
+        # LTC expects (batch, time, features)
+        # SB3 provides (batch, features), so we add a time dimension
+        if observations.dim() == 2:
+            observations = observations.unsqueeze(1)
+            
+        # Initialize hidden state if needed or if batch size changes
+        batch_size = observations.size(0)
+        if self.hx is None or self.hx.size(0) != batch_size:
+            self.hx = torch.zeros(batch_size, self.ltc.state_size, device=observations.device)
+            
+        # Forward pass through LTC
+        # Note: In a real recurrent setting with SB3, we'd need to manage hidden states 
+        # more carefully (e.g. using RecurrentPPO from sb3-contrib).
+        # For this demo, we are using a simplified approach where we treat the LTC 
+        # as a stateful feature extractor that maintains state between calls within a batch.
+        # However, standard PPO assumes stateless policies. 
+        # To make this truly "Liquid" in a standard PPO loop without sb3-contrib, 
+        # we approximate by running the LTC on the current step.
+        # A better approach for production would be RecurrentPPO.
+        # Given the constraints and the goal of a "demo", we will use the LTC 
+        # but reset state if we detect a new episode (which is hard here).
+        # So we will let the LTC evolve.
+        
+        # Detach hidden state from previous graph to prevent "backward through graph a second time" error
+        if self.hx is not None:
+            self.hx = self.hx.detach()
+            
+        output, self.hx = self.ltc(observations, self.hx)
+        
+        # Remove time dimension
+        return output.squeeze(1)
+
+def make_liquid_ppo(env, verbose=1):
+    """
+    Factory function to create a PPO agent with Liquid Brain.
+    """
+    # Parallel Environments for High-Performance Training
+    # A100/A10G are data hungry. We need to run physics on many CPU cores to feed them.
+    # We will use 16 parallel environments.
+    n_envs = 16
+    env = make_vec_env(
+        lambda: Drone3DEnv(render_mode=None, wind_scale=10.0, wind_speed=5.0),
+        n_envs=n_envs,
+        vec_env_cls=SubprocVecEnv
+    )
+    
+    # Create Model with optimized hyperparameters for A100
+    policy_kwargs = dict(
+        features_extractor_class=LTCFeatureExtractor,
+        features_extractor_kwargs=dict(features_dim=32),
+    )
+    
+    model = PPO(
+        "MlpPolicy",
+        env,
+        policy_kwargs=policy_kwargs,
+        verbose=1,
+        learning_rate=3e-4,
+        n_steps=2048, # 2048 * 16 = 32,768 steps per update
+        batch_size=4096, # Large batch size for A100
+        n_epochs=10,
+        gamma=0.99,
+        gae_lambda=0.95,
+        clip_range=0.2,
+        device="cuda" # Force CUDA
+    )
+    return model

output.txt

@@ -0,0 +1,6 @@
+Python version: 3.14.0 (main, Oct 17 2025, 00:00:00) [GCC 15.2.1 20251022 (Red Hat 15.2.1-3)]
+Numpy imported
+Gymnasium failed: No module named 'gymnasium'
+PyFlyt failed: No module named 'PyFlyt'
+Opensimplex failed: No module named 'opensimplex'
+Ncps failed: No module named 'ncps'

pip_log.txt

@@ -0,0 +1,93 @@
+Defaulting to user installation because normal site-packages is not writeable
+Collecting torch>=2.0.0 (from -r requirements.txt (line 1))
+  Using cached torch-2.9.1-cp314-cp314-manylinux_2_28_x86_64.whl.metadata (30 kB)
+Collecting gymnasium>=0.29.0 (from -r requirements.txt (line 2))
+  Using cached gymnasium-1.2.2-py3-none-any.whl.metadata (10 kB)
+Collecting pygame>=2.5.0 (from -r requirements.txt (line 3))
+  Using cached pygame-2.6.1.tar.gz (14.8 MB)
+  Installing build dependencies: started
+  Installing build dependencies: finished with status 'done'
+  Getting requirements to build wheel: started
+  Getting requirements to build wheel: finished with status 'error'
+  error: subprocess-exited-with-error
+  
+  × Getting requirements to build wheel did not run successfully.
+  │ exit code: 1
+  ╰─> [67 lines of output]
+      Skipping Cython compilation
+      
+      
+      WARNING, No "Setup" File Exists, Running "buildconfig/config.py"
+      Using UNIX configuration...
+      
+      /bin/sh: line 1: dpkg-architecture: command not found
+      /bin/sh: line 1: gcc: command not found
+      /bin/sh: line 1: gcc: command not found
+      /bin/sh: line 1: sdl2-config: command not found
+      /bin/sh: line 1: sdl2-config: command not found
+      /bin/sh: line 1: sdl2-config: command not found
+      Package freetype2 was not found in the pkg-config search path.
+      Perhaps you should add the directory containing `freetype2.pc'
+      to the PKG_CONFIG_PATH environment variable
+      Package 'freetype2' not found
+      Package freetype2 was not found in the pkg-config search path.
+      Perhaps you should add the directory containing `freetype2.pc'
+      to the PKG_CONFIG_PATH environment variable
+      Package 'freetype2' not found
+      Package freetype2 was not found in the pkg-config search path.
+      Perhaps you should add the directory containing `freetype2.pc'
+      to the PKG_CONFIG_PATH environment variable
+      Package 'freetype2' not found
+      /bin/sh: line 1: freetype-config: command not found
+      /bin/sh: line 1: freetype-config: command not found
+      /bin/sh: line 1: freetype-config: command not found
+      
+      Hunting dependencies...
+      WARNING: "sdl2-config" failed!
+      WARNING: "pkg-config freetype2" failed!
+      WARNING: "freetype-config" failed!
+      
+      ---
+      For help with compilation see:
+          https://www.pygame.org/wiki/Compilation
+      To contribute to pygame development see:
+          https://www.pygame.org/contribute.html
+      ---
+      
+      Traceback (most recent call last):
+        File "/usr/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
+          main()
+          ~~~~^^
+        File "/usr/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
+          json_out["return_val"] = hook(**hook_input["kwargs"])
+                                   ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
+        File "/usr/lib/python3.14/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
+          return hook(config_settings)
+        File "/tmp/pip-build-env-9_tn2xvg/overlay/lib/python3.14/site-packages/setuptools/build_meta.py", line 331, in get_requires_for_build_wheel
+          return self._get_build_requires(config_settings, requirements=[])
+                 ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+        File "/tmp/pip-build-env-9_tn2xvg/overlay/lib/python3.14/site-packages/setuptools/build_meta.py", line 301, in _get_build_requires
+          self.run_setup()
+          ~~~~~~~~~~~~~~^^
+        File "/tmp/pip-build-env-9_tn2xvg/overlay/lib/python3.14/site-packages/setuptools/build_meta.py", line 512, in run_setup
+          super().run_setup(setup_script=setup_script)
+          ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
+        File "/tmp/pip-build-env-9_tn2xvg/overlay/lib/python3.14/site-packages/setuptools/build_meta.py", line 317, in run_setup
+          exec(code, locals())
+          ~~~~^^^^^^^^^^^^^^^^
+        File "<string>", line 432, in <module>
+        File "/tmp/pip-install-tmizgydo/pygame_66dd0a904d484e059768cb5e32cd4cf6/buildconfig/config.py", line 234, in main
+          deps = CFG.main(**kwds, auto_config=auto)
+        File "/tmp/pip-install-tmizgydo/pygame_66dd0a904d484e059768cb5e32cd4cf6/buildconfig/config_unix.py", line 245, in main
+          raise RuntimeError('Unable to run "sdl-config". Please make sure a development version of SDL is installed.')
+      RuntimeError: Unable to run "sdl-config". Please make sure a development version of SDL is installed.
+      [end of output]
+  
+  note: This error originates from a subprocess, and is likely not a problem with pip.
+error: subprocess-exited-with-error
+
+× Getting requirements to build wheel did not run successfully.
+│ exit code: 1
+╰─> See above for output.
+
+note: This error originates from a subprocess, and is likely not a problem with pip.

pybullet_bin_log.txt

@@ -0,0 +1,3 @@
+Defaulting to user installation because normal site-packages is not writeable
+ERROR: Could not find a version that satisfies the requirement pybullet (from versions: none)
+ERROR: No matching distribution found for pybullet

pygame_log.txt

@@ -0,0 +1,3 @@
+Defaulting to user installation because normal site-packages is not writeable
+ERROR: Could not find a version that satisfies the requirement pygame (from versions: none)
+ERROR: No matching distribution found for pygame

requirements.txt

@@ -0,0 +1,13 @@
+torch>=2.0.0
+gymnasium>=0.29.0
+pygame>=2.5.0
+numpy>=1.24.0
+stable-baselines3>=2.0.0
+matplotlib>=3.7.0
+tensorboard>=2.14.0
+tqdm>=4.65.0
+rich>=13.0.0
+
+PyFlyt>=0.3.0
+opensimplex>=0.4.4
+ncps>=0.0.7

run_demo.sh

@@ -0,0 +1,9 @@
+#!/bin/bash
+set -e
+
+echo "=== Project Neuro-Flyt 3D Setup ==="
+echo "Installing dependencies..."
+pip install -r requirements.txt
+
+echo "=== Launching Demo ==="
+python demo_3d.py

setup.sh

@@ -0,0 +1,45 @@
+#!/bin/bash
+# Setup script for Drone RL project
+
+echo "Setting up Drone RL project..."
+echo ""
+
+# Check Python version
+PYTHON_CMD="python3.10"
+if ! command -v $PYTHON_CMD &> /dev/null; then
+    PYTHON_CMD="python3.11"
+fi
+
+if ! command -v $PYTHON_CMD &> /dev/null; then
+    echo "Error: Python 3.10+ not found. Please install Python 3.10 or higher."
+    exit 1
+fi
+
+echo "Using: $($PYTHON_CMD --version)"
+echo ""
+
+# Create virtual environment if it doesn't exist
+if [ ! -d "venv" ]; then
+    echo "Creating virtual environment..."
+    $PYTHON_CMD -m venv venv
+fi
+
+# Activate virtual environment
+echo "Activating virtual environment..."
+source venv/bin/activate
+
+# Upgrade pip
+echo "Upgrading pip..."
+pip install --upgrade pip
+
+# Install requirements
+echo "Installing requirements..."
+pip install -r requirements.txt
+
+echo ""
+echo "Setup complete! To activate the environment, run:"
+echo "  source venv/bin/activate"
+echo ""
+echo "To test the setup, run:"
+echo "  python main.py"
+

test_log.txt

@@ -0,0 +1,14 @@
+Testing Wind Physics...
+Traceback (most recent call last):
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/test_physics.py", line 41, in <module>
+    test_wind_physics()
+    ~~~~~~~~~~~~~~~~~^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/test_physics.py", line 11, in test_wind_physics
+    obs, _, _, _, info = env.step(np.zeros(4)) # Hover action (approx)
+                         ~~~~~~~~^^^^^^^^^^^^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/env/drone_3d.py", line 66, in step
+    raw_wind = self.wind_field.get_wind(pos[0], pos[1], pos[2], self.dt)
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/env/drone_3d.py", line 27, in get_wind
+    u = self.noise.noise4d(x * self.scale, y * self.scale, z * self.scale, self.time_offset)
+        ^^^^^^^^^^^^^^^^^^
+AttributeError: 'OpenSimplex' object has no attribute 'noise4d'. Did you mean: 'noise4'?

test_physics.py

@@ -0,0 +1,43 @@
+import numpy as np
+import time
+from env.drone_3d import Drone3DEnv
+
+def test_wind_physics():
+    print("Testing Wind Physics...")
+    env = Drone3DEnv(render_mode=None, wind_scale=10.0, wind_speed=5.0)
+    env.reset()
+    
+    # 1. Test Non-Zero Wind
+    obs, _, _, _, info = env.step(np.zeros(4)) # Hover action (approx)
+    wind_0 = info.get("wind", np.zeros(3))
+    target_0 = info.get("target", np.zeros(3))
+    print(f"Initial Wind Vector: {wind_0}")
+    print(f"Target Location: {target_0}")
+    
+    if np.linalg.norm(wind_0) == 0:
+        print("WARNING: Wind vector is zero. Check noise generation.")
+    else:
+        print("SUCCESS: Wind vector is non-zero.")
+
+    # 2. Test Temporal Variation
+    print("Stepping environment to test temporal variation...")
+    winds = []
+    for _ in range(10):
+        _, _, _, _, info = env.step(np.zeros(4))
+        winds.append(info["wind"])
+        
+    winds = np.array(winds)
+    # Check if wind changes
+    diffs = np.diff(winds, axis=0)
+    mean_diff = np.mean(np.abs(diffs))
+    print(f"Mean frame-to-frame wind change: {mean_diff:.4f}")
+    
+    if mean_diff > 0:
+        print("SUCCESS: Wind varies over time.")
+    else:
+        print("FAILURE: Wind is static.")
+
+    print("Physics Test Complete.")
+
+if __name__ == "__main__":
+    test_wind_physics()

test_random_1.txt

@@ -0,0 +1,8 @@
+Testing Wind Physics...
+Initial Wind Vector: [ 2.32584927 -5.50170588  4.15986258]
+Target Location: [-0.26455097  1.92920148 12.09442991]
+SUCCESS: Wind vector is non-zero.
+Stepping environment to test temporal variation...
+Mean frame-to-frame wind change: 1.1686
+SUCCESS: Wind varies over time.
+Physics Test Complete.

test_random_2.txt

@@ -0,0 +1,8 @@
+Testing Wind Physics...
+Initial Wind Vector: [2.32121281 1.08356691 0.06257384]
+Target Location: [ 1.08129528  4.75833083 13.20958488]
+SUCCESS: Wind vector is non-zero.
+Stepping environment to test temporal variation...
+Mean frame-to-frame wind change: 1.0202
+SUCCESS: Wind varies over time.
+Physics Test Complete.

test_result_final.txt

@@ -0,0 +1,7 @@
+Testing Wind Physics...
+Initial Wind Vector: [ 1.25934332 -0.34450611  4.63735668]
+SUCCESS: Wind vector is non-zero.
+Stepping environment to test temporal variation...
+Mean frame-to-frame wind change: 1.0813
+SUCCESS: Wind varies over time.
+Physics Test Complete.

test_success.txt

@@ -0,0 +1,7 @@
+Testing Wind Physics...
+Initial Wind Vector: [ 1.25934332 -0.34450611  4.63735668]
+SUCCESS: Wind vector is non-zero.
+Stepping environment to test temporal variation...
+Mean frame-to-frame wind change: 1.0813
+SUCCESS: Wind varies over time.
+Physics Test Complete.

train.py

@@ -0,0 +1,35 @@
+import os
+import gymnasium as gym
+from stable_baselines3 import PPO
+from stable_baselines3.common.callbacks import CheckpointCallback
+from env.drone_3d import Drone3DEnv
+from models.liquid_ppo import make_liquid_ppo
+
+def train():
+    print("Setting up Training Environment...")
+    # Create environment
+    # We use a lower wind scale for training initially to help it learn
+    env = Drone3DEnv(render_mode=None, wind_scale=2.0, wind_speed=1.0)
+    
+    print("Creating Liquid PPO Agent...")
+    model = make_liquid_ppo(env, verbose=1)
+    
+    # Create checkpoints directory
+    os.makedirs("checkpoints", exist_ok=True)
+    checkpoint_callback = CheckpointCallback(
+        save_freq=10000,
+        save_path="./checkpoints/",
+        name_prefix="liquid_ppo_drone"
+    )
+    
+    print("Starting Training (This may take a while)...")
+    # Training for 500,000 steps (500 episodes) as requested for proper convergence
+    total_timesteps = 500000
+    model.learn(total_timesteps=total_timesteps, callback=checkpoint_callback)
+    
+    print("Training Complete.")
+    model.save("liquid_ppo_drone_final")
+    print("Model saved to 'liquid_ppo_drone_final.zip'")
+
+if __name__ == "__main__":
+    train()

train/__init__.py

@@ -0,0 +1,2 @@
+"""Training scripts and utilities."""
+

train/train_liquid_ppo.py

@@ -0,0 +1,209 @@
+"""
+Train a PPO agent with Liquid Neural Network policy on the DroneWindEnv environment.
+
+This script uses stable-baselines3 PPO with a Liquid Neural Network feature extractor
+to train an agent to survive and navigate in the 2D drone environment with wind.
+The trained model is saved to models/liquid_policy.zip and TensorBoard logs
+are written to logs/ppo_liquid/.
+"""
+
+import os
+import sys
+import argparse
+from typing import Optional
+import gymnasium as gym
+from stable_baselines3 import PPO
+from stable_baselines3.common.vec_env import DummyVecEnv
+from stable_baselines3.common.monitor import Monitor
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from env.drone_env import DroneWindEnv
+from models.liquid_policy import LiquidFeatureExtractor
+
+
+def make_env(seed: Optional[int] = None) -> gym.Env:
+    """
+    Create and wrap a DroneWindEnv instance with Monitor.
+    
+    Args:
+        seed: Optional random seed for the environment
+        
+    Returns:
+        Wrapped Gymnasium environment
+    """
+    env = DroneWindEnv()
+    env = Monitor(env)
+    if seed is not None:
+        env.reset(seed=seed)
+    return env
+
+
+def make_vec_env(num_envs: int = 4) -> DummyVecEnv:
+    """
+    Create a vectorized environment with multiple parallel instances.
+    
+    Args:
+        num_envs: Number of parallel environments
+        
+    Returns:
+        Vectorized environment
+    """
+    def make_vec_env_fn(seed: Optional[int] = None):
+        def _init():
+            return make_env(seed)
+        return _init
+    
+    vec_env = DummyVecEnv([make_vec_env_fn(seed=i) for i in range(num_envs)])
+    return vec_env
+
+
+def main():
+    """Main training function."""
+    parser = argparse.ArgumentParser(description="Train PPO agent with Liquid NN on DroneWindEnv")
+    parser.add_argument(
+        "--timesteps",
+        type=int,
+        default=100_000,
+        help="Total number of training timesteps (default: 100000)"
+    )
+    parser.add_argument(
+        "--seed",
+        type=int,
+        default=0,
+        help="Random seed (default: 0)"
+    )
+    parser.add_argument(
+        "--logdir",
+        type=str,
+        default="logs/ppo_liquid",
+        help="Directory for TensorBoard logs (default: logs/ppo_liquid)"
+    )
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default="models/liquid_policy.zip",
+        help="Path to save the trained model (default: models/liquid_policy.zip)"
+    )
+    parser.add_argument(
+        "--num-envs",
+        type=int,
+        default=4,
+        help="Number of parallel environments (default: 4)"
+    )
+    parser.add_argument(
+        "--hidden-size",
+        type=int,
+        default=32,
+        help="Hidden size for liquid cell (default: 32)"
+    )
+    parser.add_argument(
+        "--dt",
+        type=float,
+        default=0.1,
+        help="Time step for liquid cell (default: 0.1)"
+    )
+    
+    args = parser.parse_args()
+    
+    # Create directories if they don't exist
+    os.makedirs(os.path.dirname(args.model_path), exist_ok=True)
+    os.makedirs(args.logdir, exist_ok=True)
+    
+    print("=" * 60)
+    print("Training PPO Agent with Liquid NN on DroneWindEnv")
+    print("=" * 60)
+    print(f"Total timesteps: {args.timesteps:,}")
+    print(f"Number of parallel environments: {args.num_envs}")
+    print(f"Liquid cell hidden size: {args.hidden_size}")
+    print(f"Liquid cell dt: {args.dt}")
+    print(f"Model will be saved to: {args.model_path}")
+    print(f"TensorBoard logs: {args.logdir}")
+    print("=" * 60)
+    
+    # Create vectorized environment
+    print("Creating vectorized environment...")
+    vec_env = make_vec_env(num_envs=args.num_envs)
+    
+    # Get observation space for feature extractor
+    obs_space = vec_env.observation_space
+    
+    # Configure policy with liquid feature extractor
+    policy_kwargs = dict(
+        features_extractor_class=LiquidFeatureExtractor,
+        features_extractor_kwargs=dict(
+            features_dim=args.hidden_size,
+            hidden_size=args.hidden_size,
+            dt=args.dt,
+        ),
+        net_arch=dict(pi=[64], vf=[64]),  # Policy and value heads with 64 hidden units
+    )
+    
+    # Create PPO agent
+    print("Initializing PPO agent with Liquid NN...")
+    model = PPO(
+        policy="MlpPolicy",
+        env=vec_env,
+        policy_kwargs=policy_kwargs,
+        n_steps=1024,
+        batch_size=64,
+        gamma=0.99,
+        learning_rate=3e-4,
+        gae_lambda=0.95,
+        clip_range=0.2,
+        ent_coef=0.01,
+        verbose=1,
+        tensorboard_log=args.logdir,
+        seed=args.seed,
+    )
+    
+    # Training with curriculum (commented out for now - use fixed mild wind)
+    # For curriculum learning, you could do:
+    # 
+    # # Phase 1: Mild wind (0-30k steps)
+    # if args.timesteps > 30000:
+    #     print("Training phase 1: Mild wind (0-30k steps)...")
+    #     model.learn(total_timesteps=30000, progress_bar=True)
+    #     
+    #     # Phase 2: Medium wind (30k-60k steps)
+    #     if args.timesteps > 60000:
+    #         print("Training phase 2: Medium wind (30k-60k steps)...")
+    #         # Would need to recreate env with difficulty=1
+    #         model.learn(total_timesteps=30000, progress_bar=True, reset_num_timesteps=False)
+    #         
+    #         # Phase 3: Strong wind (60k+ steps)
+    #         if args.timesteps > 60000:
+    #             print("Training phase 3: Strong wind (60k+ steps)...")
+    #             # Would need to recreate env with difficulty=2
+    #             model.learn(total_timesteps=args.timesteps - 60000, progress_bar=True, reset_num_timesteps=False)
+    #     else:
+    #         model.learn(total_timesteps=args.timesteps - 30000, progress_bar=True, reset_num_timesteps=False)
+    # else:
+    #     model.learn(total_timesteps=args.timesteps, progress_bar=True)
+    
+    # For now, train on fixed mild wind
+    print("\nStarting training...")
+    model.learn(
+        total_timesteps=args.timesteps,
+        progress_bar=True
+    )
+    
+    # Save the model
+    print(f"\nSaving model to {args.model_path}...")
+    model.save(args.model_path)
+    
+    print("\n" + "=" * 60)
+    print("Training completed successfully!")
+    print(f"Model saved to: {args.model_path}")
+    print(f"TensorBoard logs available at: {args.logdir}")
+    print("=" * 60)
+    print("\nTo view training progress, run:")
+    print(f"  tensorboard --logdir {args.logdir}")
+    print("\nTo evaluate the model, run:")
+    print(f"  python eval/eval_liquid_policy.py --model-path {args.model_path}")
+
+
+if __name__ == "__main__":
+    main()
+

train/train_mlp_ppo.py

@@ -0,0 +1,159 @@
+"""
+Train a PPO agent with MLP policy on the DroneWindEnv environment.
+
+This script uses stable-baselines3 PPO with a 2-layer MLP (64, 64) to train
+an agent to survive and navigate in the 2D drone environment with wind.
+The trained model is saved to models/mlp_baseline.zip and TensorBoard logs
+are written to logs/ppo_mlp/.
+"""
+
+import os
+import sys
+import argparse
+from typing import Optional
+import gymnasium as gym
+from stable_baselines3 import PPO
+from stable_baselines3.common.vec_env import DummyVecEnv
+from stable_baselines3.common.monitor import Monitor
+
+# Add project root to path
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+from env.drone_env import DroneWindEnv
+
+
+def make_env(seed: Optional[int] = None) -> gym.Env:
+    """
+    Create and wrap a DroneWindEnv instance with Monitor.
+    
+    Args:
+        seed: Optional random seed for the environment
+        
+    Returns:
+        Wrapped Gymnasium environment
+    """
+    env = DroneWindEnv()
+    env = Monitor(env)
+    if seed is not None:
+        env.reset(seed=seed)
+    return env
+
+
+def make_vec_env(num_envs: int = 4) -> DummyVecEnv:
+    """
+    Create a vectorized environment with multiple parallel instances.
+    
+    Args:
+        num_envs: Number of parallel environments
+        
+    Returns:
+        Vectorized environment
+    """
+    def make_vec_env_fn(seed: Optional[int] = None):
+        def _init():
+            return make_env(seed)
+        return _init
+    
+    vec_env = DummyVecEnv([make_vec_env_fn(seed=i) for i in range(num_envs)])
+    return vec_env
+
+
+def main():
+    """Main training function."""
+    parser = argparse.ArgumentParser(description="Train PPO agent on DroneWindEnv")
+    parser.add_argument(
+        "--timesteps",
+        type=int,
+        default=100_000,
+        help="Total number of training timesteps (default: 100000)"
+    )
+    parser.add_argument(
+        "--seed",
+        type=int,
+        default=0,
+        help="Random seed (default: 0)"
+    )
+    parser.add_argument(
+        "--logdir",
+        type=str,
+        default="logs/ppo_mlp",
+        help="Directory for TensorBoard logs (default: logs/ppo_mlp)"
+    )
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default="models/mlp_baseline.zip",
+        help="Path to save the trained model (default: models/mlp_baseline.zip)"
+    )
+    parser.add_argument(
+        "--num-envs",
+        type=int,
+        default=4,
+        help="Number of parallel environments (default: 4)"
+    )
+    
+    args = parser.parse_args()
+    
+    # Create directories if they don't exist
+    os.makedirs(os.path.dirname(args.model_path), exist_ok=True)
+    os.makedirs(args.logdir, exist_ok=True)
+    
+    print("=" * 60)
+    print("Training PPO Agent on DroneWindEnv")
+    print("=" * 60)
+    print(f"Total timesteps: {args.timesteps:,}")
+    print(f"Number of parallel environments: {args.num_envs}")
+    print(f"Model will be saved to: {args.model_path}")
+    print(f"TensorBoard logs: {args.logdir}")
+    print("=" * 60)
+    
+    # Create vectorized environment
+    print("Creating vectorized environment...")
+    vec_env = make_vec_env(num_envs=args.num_envs)
+    
+    # Configure policy (2-layer MLP with 64 hidden units each)
+    policy_kwargs = dict(net_arch=[64, 64])
+    
+    # Create PPO agent
+    print("Initializing PPO agent...")
+    model = PPO(
+        policy="MlpPolicy",
+        env=vec_env,
+        policy_kwargs=policy_kwargs,
+        n_steps=1024,
+        batch_size=64,
+        gamma=0.99,
+        learning_rate=3e-4,
+        gae_lambda=0.95,
+        clip_range=0.2,
+        ent_coef=0.0,
+        verbose=1,
+        tensorboard_log=args.logdir,
+        seed=args.seed,
+    )
+    
+    # Train the agent
+    print("\nStarting training...")
+    model.learn(
+        total_timesteps=args.timesteps,
+        progress_bar=True
+    )
+    
+    # Save the model
+    print(f"\nSaving model to {args.model_path}...")
+    model.save(args.model_path)
+    
+    print("\n" + "=" * 60)
+    print("Training completed successfully!")
+    print(f"Model saved to: {args.model_path}")
+    print(f"TensorBoard logs available at: {args.logdir}")
+    print("=" * 60)
+    print("\nTo view training progress, run:")
+    print(f"  tensorboard --logdir {args.logdir}")
+    print("\nTo evaluate the model, run:")
+    print(f"  python eval/eval_mlp_baseline.py --model-path {args.model_path}")
+
+
+if __name__ == "__main__":
+    main()
+

train_hf.py

@@ -0,0 +1,68 @@
+import os
+import argparse
+from huggingface_hub import HfApi, login
+from stable_baselines3 import PPO
+from stable_baselines3.common.callbacks import CheckpointCallback
+from stable_baselines3.common.env_util import make_vec_env
+from stable_baselines3.common.vec_env import SubprocVecEnv
+from env.drone_3d import Drone3DEnv
+from models.liquid_ppo import make_liquid_ppo, LTCFeatureExtractor
+
+def train_hf(repo_id, token, total_timesteps=500000):
+    print(f"Starting HF Training for Repo: {repo_id}")
+    
+    # Login to HF
+    if token:
+        login(token=token)
+    
+    # Create Optimized Model (Parallel Envs + A100 Tuning)
+    # Note: make_liquid_ppo now handles env creation internally for parallelism
+    model = make_liquid_ppo(None, verbose=1) 
+    
+    # Checkpoint Callback
+    checkpoint_callback = CheckpointCallback(
+        save_freq=50000,
+        save_path='./checkpoints/',
+        name_prefix='liquid_ppo_drone'
+    )
+    
+    print(f"Training for {total_timesteps} steps...")
+    model.learn(total_timesteps=total_timesteps, callback=checkpoint_callback)
+    
+    # Save Final Model
+    model_path = "liquid_ppo_drone_final.zip"
+    model.save(model_path)
+    print(f"Model saved to {model_path}")
+    
+    # Push to Hub
+    print("Pushing to Hugging Face Hub...")
+    api = HfApi()
+    
+    try:
+        # Create repo if it doesn't exist
+        api.create_repo(repo_id=repo_id, exist_ok=True)
+        
+        # Upload Model
+        api.upload_file(
+            path_or_fileobj=model_path,
+            path_in_repo="liquid_ppo_drone_final.zip",
+            repo_id=repo_id,
+            repo_type="model"
+        )
+        print("Upload Complete!")
+        
+    except Exception as e:
+        print(f"Error uploading to Hub: {e}")
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--repo_id", type=str, required=True, help="HF Repo ID (e.g., username/neuro-flyt-3d)")
+    parser.add_argument("--token", type=str, help="HF Write Token")
+    parser.add_argument("--steps", type=int, default=500000, help="Total training steps")
+    
+    args = parser.parse_args()
+    
+    # Get token from env var if not provided
+    token = args.token or os.environ.get("HF_TOKEN")
+    
+    train_hf(args.repo_id, token, args.steps)

train_log_500k.txt

@@ -0,0 +1,1974 @@
+Setting up Training Environment...
+Creating Liquid PPO Agent...
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+Starting Training (This may take a while)...
+----------------------------------
+| rollout/           |           |
+|    ep_len_mean     | 1e+03     |
+|    ep_rew_mean     | -2.56e+04 |
+| time/              |           |
+|    fps             | 608       |
+|    iterations      | 1         |
+|    time_elapsed    | 3         |
+|    total_timesteps | 2048      |
+----------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.23e+04    |
+| time/                   |              |
+|    fps                  | 177          |
+|    iterations           | 2            |
+|    time_elapsed         | 23           |
+|    total_timesteps      | 4096         |
+| train/                  |              |
+|    approx_kl            | 0.0049500195 |
+|    clip_fraction        | 0.0341       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.67        |
+|    explained_variance   | -9.31e-05    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.15e+05     |
+|    n_updates            | 10           |
+|    policy_gradient_loss | -0.00313     |
+|    std                  | 0.999        |
+|    value_loss           | 1.99e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.19e+04   |
+| time/                   |             |
+|    fps                  | 129         |
+|    iterations           | 3           |
+|    time_elapsed         | 47          |
+|    total_timesteps      | 6144        |
+| train/                  |             |
+|    approx_kl            | 0.002073763 |
+|    clip_fraction        | 0.00542     |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.67       |
+|    explained_variance   | 2.62e-05    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 5.71e+04    |
+|    n_updates            | 20          |
+|    policy_gradient_loss | -0.000381   |
+|    std                  | 0.996       |
+|    value_loss           | 1.15e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.28e+04    |
+| time/                   |              |
+|    fps                  | 116          |
+|    iterations           | 4            |
+|    time_elapsed         | 70           |
+|    total_timesteps      | 8192         |
+| train/                  |              |
+|    approx_kl            | 0.0047560623 |
+|    clip_fraction        | 0.0275       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.64        |
+|    explained_variance   | 8.34e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 7.31e+04     |
+|    n_updates            | 30           |
+|    policy_gradient_loss | -0.00268     |
+|    std                  | 0.988        |
+|    value_loss           | 1.42e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.4e+04    |
+| time/                   |             |
+|    fps                  | 93          |
+|    iterations           | 5           |
+|    time_elapsed         | 109         |
+|    total_timesteps      | 10240       |
+| train/                  |             |
+|    approx_kl            | 0.004183922 |
+|    clip_fraction        | 0.0234      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.62       |
+|    explained_variance   | -4.77e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.35e+05    |
+|    n_updates            | 40          |
+|    policy_gradient_loss | -0.003      |
+|    std                  | 0.985       |
+|    value_loss           | 2.21e+05    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.28e+04   |
+| time/                   |             |
+|    fps                  | 98          |
+|    iterations           | 6           |
+|    time_elapsed         | 125         |
+|    total_timesteps      | 12288       |
+| train/                  |             |
+|    approx_kl            | 0.005293761 |
+|    clip_fraction        | 0.0418      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.61       |
+|    explained_variance   | -3.58e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.32e+05    |
+|    n_updates            | 50          |
+|    policy_gradient_loss | -0.00347    |
+|    std                  | 0.985       |
+|    value_loss           | 2.86e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.41e+04    |
+| time/                   |              |
+|    fps                  | 97           |
+|    iterations           | 7            |
+|    time_elapsed         | 146          |
+|    total_timesteps      | 14336        |
+| train/                  |              |
+|    approx_kl            | 0.0050999783 |
+|    clip_fraction        | 0.0295       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.59        |
+|    explained_variance   | -7.15e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 4.66e+04     |
+|    n_updates            | 60           |
+|    policy_gradient_loss | -0.00355     |
+|    std                  | 0.974        |
+|    value_loss           | 8.14e+04     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.6e+04     |
+| time/                   |              |
+|    fps                  | 103          |
+|    iterations           | 8            |
+|    time_elapsed         | 158          |
+|    total_timesteps      | 16384        |
+| train/                  |              |
+|    approx_kl            | 0.0042739166 |
+|    clip_fraction        | 0.0147       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.57        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.64e+05     |
+|    n_updates            | 70           |
+|    policy_gradient_loss | -0.0019      |
+|    std                  | 0.972        |
+|    value_loss           | 3.31e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.7e+04     |
+| time/                   |              |
+|    fps                  | 108          |
+|    iterations           | 9            |
+|    time_elapsed         | 170          |
+|    total_timesteps      | 18432        |
+| train/                  |              |
+|    approx_kl            | 0.0053871158 |
+|    clip_fraction        | 0.0297       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.56        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.43e+05     |
+|    n_updates            | 80           |
+|    policy_gradient_loss | -0.00304     |
+|    std                  | 0.972        |
+|    value_loss           | 5.33e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.81e+04    |
+| time/                   |              |
+|    fps                  | 112          |
+|    iterations           | 10           |
+|    time_elapsed         | 181          |
+|    total_timesteps      | 20480        |
+| train/                  |              |
+|    approx_kl            | 0.0035741455 |
+|    clip_fraction        | 0.0138       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.56        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.74e+05     |
+|    n_updates            | 90           |
+|    policy_gradient_loss | -0.00138     |
+|    std                  | 0.971        |
+|    value_loss           | 3.49e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.79e+04   |
+| time/                   |             |
+|    fps                  | 116         |
+|    iterations           | 11          |
+|    time_elapsed         | 193         |
+|    total_timesteps      | 22528       |
+| train/                  |             |
+|    approx_kl            | 0.004108442 |
+|    clip_fraction        | 0.0245      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.56       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.53e+05    |
+|    n_updates            | 100         |
+|    policy_gradient_loss | -0.00274    |
+|    std                  | 0.971       |
+|    value_loss           | 5.84e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.84e+04    |
+| time/                   |              |
+|    fps                  | 119          |
+|    iterations           | 12           |
+|    time_elapsed         | 205          |
+|    total_timesteps      | 24576        |
+| train/                  |              |
+|    approx_kl            | 0.0057261223 |
+|    clip_fraction        | 0.0375       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.53        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 8.12e+04     |
+|    n_updates            | 110          |
+|    policy_gradient_loss | -0.003       |
+|    std                  | 0.96         |
+|    value_loss           | 1.72e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.82e+04    |
+| time/                   |              |
+|    fps                  | 122          |
+|    iterations           | 13           |
+|    time_elapsed         | 217          |
+|    total_timesteps      | 26624        |
+| train/                  |              |
+|    approx_kl            | 0.0051155365 |
+|    clip_fraction        | 0.0225       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.5         |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.4e+05      |
+|    n_updates            | 120          |
+|    policy_gradient_loss | -0.00331     |
+|    std                  | 0.955        |
+|    value_loss           | 4.7e+05      |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.83e+04   |
+| time/                   |             |
+|    fps                  | 125         |
+|    iterations           | 14          |
+|    time_elapsed         | 228         |
+|    total_timesteps      | 28672       |
+| train/                  |             |
+|    approx_kl            | 0.005322621 |
+|    clip_fraction        | 0.042       |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.48       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.33e+05    |
+|    n_updates            | 130         |
+|    policy_gradient_loss | -0.00404    |
+|    std                  | 0.952       |
+|    value_loss           | 2.58e+05    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.78e+04   |
+| time/                   |             |
+|    fps                  | 127         |
+|    iterations           | 15          |
+|    time_elapsed         | 240         |
+|    total_timesteps      | 30720       |
+| train/                  |             |
+|    approx_kl            | 0.006120109 |
+|    clip_fraction        | 0.0588      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.46       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.32e+05    |
+|    n_updates            | 140         |
+|    policy_gradient_loss | -0.00445    |
+|    std                  | 0.942       |
+|    value_loss           | 2.73e+05    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.79e+04   |
+| time/                   |             |
+|    fps                  | 129         |
+|    iterations           | 16          |
+|    time_elapsed         | 253         |
+|    total_timesteps      | 32768       |
+| train/                  |             |
+|    approx_kl            | 0.004814163 |
+|    clip_fraction        | 0.0178      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.43       |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 5.79e+04    |
+|    n_updates            | 150         |
+|    policy_gradient_loss | -0.000932   |
+|    std                  | 0.939       |
+|    value_loss           | 1.42e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.82e+04    |
+| time/                   |              |
+|    fps                  | 130          |
+|    iterations           | 17           |
+|    time_elapsed         | 266          |
+|    total_timesteps      | 34816        |
+| train/                  |              |
+|    approx_kl            | 0.0027875581 |
+|    clip_fraction        | 0.0163       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.43        |
+|    explained_variance   | 1.19e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.62e+05     |
+|    n_updates            | 160          |
+|    policy_gradient_loss | -0.00144     |
+|    std                  | 0.94         |
+|    value_loss           | 3.25e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.84e+04    |
+| time/                   |              |
+|    fps                  | 132          |
+|    iterations           | 18           |
+|    time_elapsed         | 277          |
+|    total_timesteps      | 36864        |
+| train/                  |              |
+|    approx_kl            | 0.0035902325 |
+|    clip_fraction        | 0.0154       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.43        |
+|    explained_variance   | 1.19e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.59e+05     |
+|    n_updates            | 170          |
+|    policy_gradient_loss | -0.00172     |
+|    std                  | 0.942        |
+|    value_loss           | 3.88e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.83e+04    |
+| time/                   |              |
+|    fps                  | 134          |
+|    iterations           | 19           |
+|    time_elapsed         | 289          |
+|    total_timesteps      | 38912        |
+| train/                  |              |
+|    approx_kl            | 0.0044348813 |
+|    clip_fraction        | 0.025        |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.44        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.36e+05     |
+|    n_updates            | 180          |
+|    policy_gradient_loss | -0.00187     |
+|    std                  | 0.943        |
+|    value_loss           | 2.4e+05      |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.77e+04   |
+| time/                   |             |
+|    fps                  | 136         |
+|    iterations           | 20          |
+|    time_elapsed         | 300         |
+|    total_timesteps      | 40960       |
+| train/                  |             |
+|    approx_kl            | 0.003115609 |
+|    clip_fraction        | 0.0162      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.42       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.02e+05    |
+|    n_updates            | 190         |
+|    policy_gradient_loss | -0.00152    |
+|    std                  | 0.936       |
+|    value_loss           | 2.02e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.72e+04    |
+| time/                   |              |
+|    fps                  | 136          |
+|    iterations           | 21           |
+|    time_elapsed         | 314          |
+|    total_timesteps      | 43008        |
+| train/                  |              |
+|    approx_kl            | 0.0044121114 |
+|    clip_fraction        | 0.0311       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.42        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 4.9e+04      |
+|    n_updates            | 200          |
+|    policy_gradient_loss | -0.00261     |
+|    std                  | 0.941        |
+|    value_loss           | 1.09e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.69e+04    |
+| time/                   |              |
+|    fps                  | 138          |
+|    iterations           | 22           |
+|    time_elapsed         | 326          |
+|    total_timesteps      | 45056        |
+| train/                  |              |
+|    approx_kl            | 0.0050966754 |
+|    clip_fraction        | 0.0294       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.41        |
+|    explained_variance   | 1.19e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 5.13e+04     |
+|    n_updates            | 210          |
+|    policy_gradient_loss | -0.00221     |
+|    std                  | 0.933        |
+|    value_loss           | 1.11e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.71e+04    |
+| time/                   |              |
+|    fps                  | 139          |
+|    iterations           | 23           |
+|    time_elapsed         | 337          |
+|    total_timesteps      | 47104        |
+| train/                  |              |
+|    approx_kl            | 0.0042023044 |
+|    clip_fraction        | 0.0154       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.4         |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 5.53e+04     |
+|    n_updates            | 220          |
+|    policy_gradient_loss | -0.000932    |
+|    std                  | 0.934        |
+|    value_loss           | 1.32e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.74e+04    |
+| time/                   |              |
+|    fps                  | 140          |
+|    iterations           | 24           |
+|    time_elapsed         | 348          |
+|    total_timesteps      | 49152        |
+| train/                  |              |
+|    approx_kl            | 0.0060270163 |
+|    clip_fraction        | 0.0548       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.4         |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.27e+05     |
+|    n_updates            | 230          |
+|    policy_gradient_loss | -0.00514     |
+|    std                  | 0.932        |
+|    value_loss           | 2.93e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.77e+04   |
+| time/                   |             |
+|    fps                  | 141         |
+|    iterations           | 25          |
+|    time_elapsed         | 361         |
+|    total_timesteps      | 51200       |
+| train/                  |             |
+|    approx_kl            | 0.003641401 |
+|    clip_fraction        | 0.0161      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.4        |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.7e+05     |
+|    n_updates            | 240         |
+|    policy_gradient_loss | -0.00216    |
+|    std                  | 0.937       |
+|    value_loss           | 3.48e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.79e+04    |
+| time/                   |              |
+|    fps                  | 142          |
+|    iterations           | 26           |
+|    time_elapsed         | 372          |
+|    total_timesteps      | 53248        |
+| train/                  |              |
+|    approx_kl            | 0.0040730843 |
+|    clip_fraction        | 0.0225       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.41        |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.05e+05     |
+|    n_updates            | 250          |
+|    policy_gradient_loss | -0.00147     |
+|    std                  | 0.934        |
+|    value_loss           | 4.28e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.8e+04    |
+| time/                   |             |
+|    fps                  | 143         |
+|    iterations           | 27          |
+|    time_elapsed         | 384         |
+|    total_timesteps      | 55296       |
+| train/                  |             |
+|    approx_kl            | 0.003144626 |
+|    clip_fraction        | 0.00791     |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.42       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.71e+05    |
+|    n_updates            | 260         |
+|    policy_gradient_loss | -0.00195    |
+|    std                  | 0.94        |
+|    value_loss           | 3.93e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.83e+04    |
+| time/                   |              |
+|    fps                  | 144          |
+|    iterations           | 28           |
+|    time_elapsed         | 397          |
+|    total_timesteps      | 57344        |
+| train/                  |              |
+|    approx_kl            | 0.0052720373 |
+|    clip_fraction        | 0.0272       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.42        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.75e+05     |
+|    n_updates            | 270          |
+|    policy_gradient_loss | -0.00242     |
+|    std                  | 0.935        |
+|    value_loss           | 3.07e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.79e+04    |
+| time/                   |              |
+|    fps                  | 145          |
+|    iterations           | 29           |
+|    time_elapsed         | 409          |
+|    total_timesteps      | 59392        |
+| train/                  |              |
+|    approx_kl            | 0.0041839215 |
+|    clip_fraction        | 0.0244       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.4         |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.79e+05     |
+|    n_updates            | 280          |
+|    policy_gradient_loss | -0.00283     |
+|    std                  | 0.933        |
+|    value_loss           | 3.86e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.8e+04     |
+| time/                   |              |
+|    fps                  | 145          |
+|    iterations           | 30           |
+|    time_elapsed         | 421          |
+|    total_timesteps      | 61440        |
+| train/                  |              |
+|    approx_kl            | 0.0053371564 |
+|    clip_fraction        | 0.0308       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.35        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 3.19e+04     |
+|    n_updates            | 290          |
+|    policy_gradient_loss | -0.00282     |
+|    std                  | 0.915        |
+|    value_loss           | 6.26e+04     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.78e+04    |
+| time/                   |              |
+|    fps                  | 146          |
+|    iterations           | 31           |
+|    time_elapsed         | 433          |
+|    total_timesteps      | 63488        |
+| train/                  |              |
+|    approx_kl            | 0.0045930664 |
+|    clip_fraction        | 0.0416       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.31        |
+|    explained_variance   | 1.19e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.66e+05     |
+|    n_updates            | 300          |
+|    policy_gradient_loss | -0.00376     |
+|    std                  | 0.913        |
+|    value_loss           | 3.15e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.77e+04   |
+| time/                   |             |
+|    fps                  | 147         |
+|    iterations           | 32          |
+|    time_elapsed         | 445         |
+|    total_timesteps      | 65536       |
+| train/                  |             |
+|    approx_kl            | 0.006433362 |
+|    clip_fraction        | 0.0423      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.29       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 7.14e+04    |
+|    n_updates            | 310         |
+|    policy_gradient_loss | -0.00386    |
+|    std                  | 0.906       |
+|    value_loss           | 1.45e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.74e+04    |
+| time/                   |              |
+|    fps                  | 147          |
+|    iterations           | 33           |
+|    time_elapsed         | 457          |
+|    total_timesteps      | 67584        |
+| train/                  |              |
+|    approx_kl            | 0.0060111308 |
+|    clip_fraction        | 0.0567       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.27        |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 8.6e+04      |
+|    n_updates            | 320          |
+|    policy_gradient_loss | -0.004       |
+|    std                  | 0.904        |
+|    value_loss           | 1.9e+05      |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.71e+04   |
+| time/                   |             |
+|    fps                  | 148         |
+|    iterations           | 34          |
+|    time_elapsed         | 469         |
+|    total_timesteps      | 69632       |
+| train/                  |             |
+|    approx_kl            | 0.002752479 |
+|    clip_fraction        | 0.0267      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.28       |
+|    explained_variance   | -1.19e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 3.72e+04    |
+|    n_updates            | 330         |
+|    policy_gradient_loss | -0.000776   |
+|    std                  | 0.909       |
+|    value_loss           | 5.9e+04     |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.7e+04    |
+| time/                   |             |
+|    fps                  | 149         |
+|    iterations           | 35          |
+|    time_elapsed         | 480         |
+|    total_timesteps      | 71680       |
+| train/                  |             |
+|    approx_kl            | 0.004144692 |
+|    clip_fraction        | 0.0243      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.29       |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 4.48e+04    |
+|    n_updates            | 340         |
+|    policy_gradient_loss | -0.00131    |
+|    std                  | 0.907       |
+|    value_loss           | 1.06e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.69e+04    |
+| time/                   |              |
+|    fps                  | 149          |
+|    iterations           | 36           |
+|    time_elapsed         | 492          |
+|    total_timesteps      | 73728        |
+| train/                  |              |
+|    approx_kl            | 0.0060227686 |
+|    clip_fraction        | 0.043        |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.28        |
+|    explained_variance   | 1.19e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 9.28e+04     |
+|    n_updates            | 350          |
+|    policy_gradient_loss | -0.00293     |
+|    std                  | 0.903        |
+|    value_loss           | 1.91e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.68e+04   |
+| time/                   |             |
+|    fps                  | 150         |
+|    iterations           | 37          |
+|    time_elapsed         | 504         |
+|    total_timesteps      | 75776       |
+| train/                  |             |
+|    approx_kl            | 0.003745494 |
+|    clip_fraction        | 0.0147      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.29       |
+|    explained_variance   | -1.19e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 6.81e+04    |
+|    n_updates            | 360         |
+|    policy_gradient_loss | -0.00168    |
+|    std                  | 0.91        |
+|    value_loss           | 1.31e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.71e+04    |
+| time/                   |              |
+|    fps                  | 150          |
+|    iterations           | 38           |
+|    time_elapsed         | 516          |
+|    total_timesteps      | 77824        |
+| train/                  |              |
+|    approx_kl            | 0.0039524576 |
+|    clip_fraction        | 0.0286       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.3         |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.13e+05     |
+|    n_updates            | 370          |
+|    policy_gradient_loss | -0.00305     |
+|    std                  | 0.909        |
+|    value_loss           | 2.44e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.67e+04   |
+| time/                   |             |
+|    fps                  | 150         |
+|    iterations           | 39          |
+|    time_elapsed         | 529         |
+|    total_timesteps      | 79872       |
+| train/                  |             |
+|    approx_kl            | 0.005160669 |
+|    clip_fraction        | 0.0254      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.29       |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.6e+05     |
+|    n_updates            | 380         |
+|    policy_gradient_loss | -0.00292    |
+|    std                  | 0.907       |
+|    value_loss           | 2.55e+05    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.66e+04    |
+| time/                   |              |
+|    fps                  | 151          |
+|    iterations           | 40           |
+|    time_elapsed         | 540          |
+|    total_timesteps      | 81920        |
+| train/                  |              |
+|    approx_kl            | 0.0046265204 |
+|    clip_fraction        | 0.0285       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.27        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.09e+04     |
+|    n_updates            | 390          |
+|    policy_gradient_loss | -0.00145     |
+|    std                  | 0.902        |
+|    value_loss           | 3.81e+04     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.67e+04    |
+| time/                   |              |
+|    fps                  | 151          |
+|    iterations           | 41           |
+|    time_elapsed         | 553          |
+|    total_timesteps      | 83968        |
+| train/                  |              |
+|    approx_kl            | 0.0042863134 |
+|    clip_fraction        | 0.0239       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.26        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 9.31e+04     |
+|    n_updates            | 400          |
+|    policy_gradient_loss | -0.00143     |
+|    std                  | 0.9          |
+|    value_loss           | 1.85e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.68e+04   |
+| time/                   |             |
+|    fps                  | 151         |
+|    iterations           | 42          |
+|    time_elapsed         | 566         |
+|    total_timesteps      | 86016       |
+| train/                  |             |
+|    approx_kl            | 0.005065168 |
+|    clip_fraction        | 0.0253      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.24       |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.1e+05     |
+|    n_updates            | 410         |
+|    policy_gradient_loss | -0.0026     |
+|    std                  | 0.894       |
+|    value_loss           | 2.2e+05     |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.67e+04    |
+| time/                   |              |
+|    fps                  | 152          |
+|    iterations           | 43           |
+|    time_elapsed         | 577          |
+|    total_timesteps      | 88064        |
+| train/                  |              |
+|    approx_kl            | 0.0030657728 |
+|    clip_fraction        | 0.0121       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.22        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.52e+05     |
+|    n_updates            | 420          |
+|    policy_gradient_loss | -0.00173     |
+|    std                  | 0.892        |
+|    value_loss           | 3.03e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.73e+04    |
+| time/                   |              |
+|    fps                  | 153          |
+|    iterations           | 44           |
+|    time_elapsed         | 588          |
+|    total_timesteps      | 90112        |
+| train/                  |              |
+|    approx_kl            | 0.0051104454 |
+|    clip_fraction        | 0.0357       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.22        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 8.25e+04     |
+|    n_updates            | 430          |
+|    policy_gradient_loss | -0.00201     |
+|    std                  | 0.893        |
+|    value_loss           | 1.81e+05     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -2.8e+04     |
+| time/                   |              |
+|    fps                  | 153          |
+|    iterations           | 45           |
+|    time_elapsed         | 600          |
+|    total_timesteps      | 92160        |
+| train/                  |              |
+|    approx_kl            | 0.0051720007 |
+|    clip_fraction        | 0.033        |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.23        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 3.47e+05     |
+|    n_updates            | 440          |
+|    policy_gradient_loss | -0.00428     |
+|    std                  | 0.896        |
+|    value_loss           | 7.59e+05     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.9e+04    |
+| time/                   |             |
+|    fps                  | 153         |
+|    iterations           | 46          |
+|    time_elapsed         | 612         |
+|    total_timesteps      | 94208       |
+| train/                  |             |
+|    approx_kl            | 0.004487371 |
+|    clip_fraction        | 0.0192      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.23       |
+|    explained_variance   | 5.96e-08    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 6.84e+05    |
+|    n_updates            | 450         |
+|    policy_gradient_loss | -0.00245    |
+|    std                  | 0.895       |
+|    value_loss           | 1.3e+06     |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -2.98e+04   |
+| time/                   |             |
+|    fps                  | 153         |
+|    iterations           | 47          |
+|    time_elapsed         | 625         |
+|    total_timesteps      | 96256       |
+| train/                  |             |
+|    approx_kl            | 0.005325151 |
+|    clip_fraction        | 0.0271      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.22       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 9.17e+05    |
+|    n_updates            | 460         |
+|    policy_gradient_loss | -0.00341    |
+|    std                  | 0.892       |
+|    value_loss           | 1.81e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -3.08e+04   |
+| time/                   |             |
+|    fps                  | 154         |
+|    iterations           | 48          |
+|    time_elapsed         | 637         |
+|    total_timesteps      | 98304       |
+| train/                  |             |
+|    approx_kl            | 0.004731435 |
+|    clip_fraction        | 0.0245      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.21       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 8.34e+05    |
+|    n_updates            | 470         |
+|    policy_gradient_loss | -0.00319    |
+|    std                  | 0.888       |
+|    value_loss           | 1.53e+06    |
+-----------------------------------------
+----------------------------------------
+| rollout/                |            |
+|    ep_len_mean          | 1e+03      |
+|    ep_rew_mean          | -3.21e+04  |
+| time/                   |            |
+|    fps                  | 154        |
+|    iterations           | 49         |
+|    time_elapsed         | 648        |
+|    total_timesteps      | 100352     |
+| train/                  |            |
+|    approx_kl            | 0.00386829 |
+|    clip_fraction        | 0.00859    |
+|    clip_range           | 0.2        |
+|    entropy_loss         | -5.2       |
+|    explained_variance   | -1.19e-07  |
+|    learning_rate        | 0.0003     |
+|    loss                 | 1.05e+06   |
+|    n_updates            | 480        |
+|    policy_gradient_loss | -0.00151   |
+|    std                  | 0.887      |
+|    value_loss           | 2.04e+06   |
+----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -3.34e+04   |
+| time/                   |             |
+|    fps                  | 154         |
+|    iterations           | 50          |
+|    time_elapsed         | 660         |
+|    total_timesteps      | 102400      |
+| train/                  |             |
+|    approx_kl            | 0.005242249 |
+|    clip_fraction        | 0.0372      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | -1.19e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.95e+06    |
+|    n_updates            | 490         |
+|    policy_gradient_loss | -0.00506    |
+|    std                  | 0.889       |
+|    value_loss           | 3.17e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -3.5e+04    |
+| time/                   |             |
+|    fps                  | 155         |
+|    iterations           | 51          |
+|    time_elapsed         | 673         |
+|    total_timesteps      | 104448      |
+| train/                  |             |
+|    approx_kl            | 0.003204999 |
+|    clip_fraction        | 0.00566     |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.21       |
+|    explained_variance   | 5.96e-08    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.29e+06    |
+|    n_updates            | 500         |
+|    policy_gradient_loss | -0.000996   |
+|    std                  | 0.89        |
+|    value_loss           | 3.12e+06    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -3.71e+04    |
+| time/                   |              |
+|    fps                  | 155          |
+|    iterations           | 52           |
+|    time_elapsed         | 685          |
+|    total_timesteps      | 106496       |
+| train/                  |              |
+|    approx_kl            | 0.0037713286 |
+|    clip_fraction        | 0.0106       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.2         |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.73e+06     |
+|    n_updates            | 510          |
+|    policy_gradient_loss | -0.00205     |
+|    std                  | 0.889        |
+|    value_loss           | 3.39e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -3.89e+04   |
+| time/                   |             |
+|    fps                  | 155         |
+|    iterations           | 53          |
+|    time_elapsed         | 699         |
+|    total_timesteps      | 108544      |
+| train/                  |             |
+|    approx_kl            | 0.003621605 |
+|    clip_fraction        | 0.00576     |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.11e+06    |
+|    n_updates            | 520         |
+|    policy_gradient_loss | -0.00108    |
+|    std                  | 0.889       |
+|    value_loss           | 5.23e+06    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -4.07e+04    |
+| time/                   |              |
+|    fps                  | 155          |
+|    iterations           | 54           |
+|    time_elapsed         | 710          |
+|    total_timesteps      | 110592       |
+| train/                  |              |
+|    approx_kl            | 0.0037987605 |
+|    clip_fraction        | 0.0108       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.21        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.02e+06     |
+|    n_updates            | 530          |
+|    policy_gradient_loss | -0.00193     |
+|    std                  | 0.891        |
+|    value_loss           | 4.78e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -4.31e+04    |
+| time/                   |              |
+|    fps                  | 155          |
+|    iterations           | 55           |
+|    time_elapsed         | 722          |
+|    total_timesteps      | 112640       |
+| train/                  |              |
+|    approx_kl            | 0.0041659893 |
+|    clip_fraction        | 0.00801      |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.21        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.27e+06     |
+|    n_updates            | 540          |
+|    policy_gradient_loss | -0.00111     |
+|    std                  | 0.89         |
+|    value_loss           | 5e+06        |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -4.52e+04    |
+| time/                   |              |
+|    fps                  | 156          |
+|    iterations           | 56           |
+|    time_elapsed         | 734          |
+|    total_timesteps      | 114688       |
+| train/                  |              |
+|    approx_kl            | 0.0052787326 |
+|    clip_fraction        | 0.0296       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.21        |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 3.7e+06      |
+|    n_updates            | 550          |
+|    policy_gradient_loss | -0.00406     |
+|    std                  | 0.889        |
+|    value_loss           | 6.79e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -4.67e+04   |
+| time/                   |             |
+|    fps                  | 156         |
+|    iterations           | 57          |
+|    time_elapsed         | 746         |
+|    total_timesteps      | 116736      |
+| train/                  |             |
+|    approx_kl            | 0.004433933 |
+|    clip_fraction        | 0.0184      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | 5.96e-08    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.93e+06    |
+|    n_updates            | 560         |
+|    policy_gradient_loss | -0.00222    |
+|    std                  | 0.888       |
+|    value_loss           | 6.15e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -4.83e+04   |
+| time/                   |             |
+|    fps                  | 156         |
+|    iterations           | 58          |
+|    time_elapsed         | 758         |
+|    total_timesteps      | 118784      |
+| train/                  |             |
+|    approx_kl            | 0.004643922 |
+|    clip_fraction        | 0.025       |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.54e+06    |
+|    n_updates            | 570         |
+|    policy_gradient_loss | -0.00334    |
+|    std                  | 0.888       |
+|    value_loss           | 4.8e+06     |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -4.99e+04   |
+| time/                   |             |
+|    fps                  | 156         |
+|    iterations           | 59          |
+|    time_elapsed         | 769         |
+|    total_timesteps      | 120832      |
+| train/                  |             |
+|    approx_kl            | 0.004279623 |
+|    clip_fraction        | 0.00815     |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.58e+06    |
+|    n_updates            | 580         |
+|    policy_gradient_loss | -0.00133    |
+|    std                  | 0.89        |
+|    value_loss           | 4.7e+06     |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -5.18e+04   |
+| time/                   |             |
+|    fps                  | 157         |
+|    iterations           | 60          |
+|    time_elapsed         | 782         |
+|    total_timesteps      | 122880      |
+| train/                  |             |
+|    approx_kl            | 0.004928913 |
+|    clip_fraction        | 0.0252      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.06e+06    |
+|    n_updates            | 590         |
+|    policy_gradient_loss | -0.00314    |
+|    std                  | 0.886       |
+|    value_loss           | 4.64e+06    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -5.37e+04    |
+| time/                   |              |
+|    fps                  | 157          |
+|    iterations           | 61           |
+|    time_elapsed         | 793          |
+|    total_timesteps      | 124928       |
+| train/                  |              |
+|    approx_kl            | 0.0044577485 |
+|    clip_fraction        | 0.0167       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.19        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.75e+06     |
+|    n_updates            | 600          |
+|    policy_gradient_loss | -0.00216     |
+|    std                  | 0.884        |
+|    value_loss           | 5.37e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -5.54e+04    |
+| time/                   |              |
+|    fps                  | 157          |
+|    iterations           | 62           |
+|    time_elapsed         | 805          |
+|    total_timesteps      | 126976       |
+| train/                  |              |
+|    approx_kl            | 0.0033779903 |
+|    clip_fraction        | 0.00854      |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.19        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.75e+06     |
+|    n_updates            | 610          |
+|    policy_gradient_loss | -0.000881    |
+|    std                  | 0.887        |
+|    value_loss           | 5.11e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -5.84e+04    |
+| time/                   |              |
+|    fps                  | 157          |
+|    iterations           | 63           |
+|    time_elapsed         | 817          |
+|    total_timesteps      | 129024       |
+| train/                  |              |
+|    approx_kl            | 0.0036522774 |
+|    clip_fraction        | 0.00669      |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.2         |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.36e+06     |
+|    n_updates            | 620          |
+|    policy_gradient_loss | -0.00121     |
+|    std                  | 0.889        |
+|    value_loss           | 5.49e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -6.02e+04   |
+| time/                   |             |
+|    fps                  | 158         |
+|    iterations           | 64          |
+|    time_elapsed         | 829         |
+|    total_timesteps      | 131072      |
+| train/                  |             |
+|    approx_kl            | 0.005089692 |
+|    clip_fraction        | 0.0314      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | -1.19e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.64e+06    |
+|    n_updates            | 630         |
+|    policy_gradient_loss | -0.00405    |
+|    std                  | 0.887       |
+|    value_loss           | 5.26e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -6.17e+04   |
+| time/                   |             |
+|    fps                  | 158         |
+|    iterations           | 65          |
+|    time_elapsed         | 841         |
+|    total_timesteps      | 133120      |
+| train/                  |             |
+|    approx_kl            | 0.004889611 |
+|    clip_fraction        | 0.0176      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.2        |
+|    explained_variance   | 1.19e-07    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.92e+06    |
+|    n_updates            | 640         |
+|    policy_gradient_loss | -0.00247    |
+|    std                  | 0.888       |
+|    value_loss           | 4.65e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -6.28e+04   |
+| time/                   |             |
+|    fps                  | 158         |
+|    iterations           | 66          |
+|    time_elapsed         | 853         |
+|    total_timesteps      | 135168      |
+| train/                  |             |
+|    approx_kl            | 0.004658374 |
+|    clip_fraction        | 0.021       |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.19       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.67e+06    |
+|    n_updates            | 650         |
+|    policy_gradient_loss | -0.00222    |
+|    std                  | 0.886       |
+|    value_loss           | 3.85e+06    |
+-----------------------------------------
+----------------------------------------
+| rollout/                |            |
+|    ep_len_mean          | 1e+03      |
+|    ep_rew_mean          | -6.38e+04  |
+| time/                   |            |
+|    fps                  | 158        |
+|    iterations           | 67         |
+|    time_elapsed         | 866        |
+|    total_timesteps      | 137216     |
+| train/                  |            |
+|    approx_kl            | 0.00569404 |
+|    clip_fraction        | 0.0394     |
+|    clip_range           | 0.2        |
+|    entropy_loss         | -5.19      |
+|    explained_variance   | 5.96e-08   |
+|    learning_rate        | 0.0003     |
+|    loss                 | 1.44e+06   |
+|    n_updates            | 660        |
+|    policy_gradient_loss | -0.00414   |
+|    std                  | 0.887      |
+|    value_loss           | 2.78e+06   |
+----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -6.47e+04   |
+| time/                   |             |
+|    fps                  | 157         |
+|    iterations           | 68          |
+|    time_elapsed         | 887         |
+|    total_timesteps      | 139264      |
+| train/                  |             |
+|    approx_kl            | 0.004979359 |
+|    clip_fraction        | 0.0314      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.19       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 7.57e+05    |
+|    n_updates            | 670         |
+|    policy_gradient_loss | -0.00347    |
+|    std                  | 0.883       |
+|    value_loss           | 1.69e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -6.6e+04    |
+| time/                   |             |
+|    fps                  | 155         |
+|    iterations           | 69          |
+|    time_elapsed         | 911         |
+|    total_timesteps      | 141312      |
+| train/                  |             |
+|    approx_kl            | 0.003934146 |
+|    clip_fraction        | 0.0181      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.17       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 8.59e+05    |
+|    n_updates            | 680         |
+|    policy_gradient_loss | -0.002      |
+|    std                  | 0.882       |
+|    value_loss           | 1.66e+06    |
+-----------------------------------------
+----------------------------------------
+| rollout/                |            |
+|    ep_len_mean          | 1e+03      |
+|    ep_rew_mean          | -6.73e+04  |
+| time/                   |            |
+|    fps                  | 154        |
+|    iterations           | 70         |
+|    time_elapsed         | 929        |
+|    total_timesteps      | 143360     |
+| train/                  |            |
+|    approx_kl            | 0.00488944 |
+|    clip_fraction        | 0.0386     |
+|    clip_range           | 0.2        |
+|    entropy_loss         | -5.17      |
+|    explained_variance   | 0          |
+|    learning_rate        | 0.0003     |
+|    loss                 | 1.07e+06   |
+|    n_updates            | 690        |
+|    policy_gradient_loss | -0.00419   |
+|    std                  | 0.879      |
+|    value_loss           | 2.26e+06   |
+----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -6.88e+04    |
+| time/                   |              |
+|    fps                  | 151          |
+|    iterations           | 71           |
+|    time_elapsed         | 956          |
+|    total_timesteps      | 145408       |
+| train/                  |              |
+|    approx_kl            | 0.0039507896 |
+|    clip_fraction        | 0.026        |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.16        |
+|    explained_variance   | -2.38e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.23e+06     |
+|    n_updates            | 700          |
+|    policy_gradient_loss | -0.00263     |
+|    std                  | 0.879        |
+|    value_loss           | 2.33e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -7e+04       |
+| time/                   |              |
+|    fps                  | 151          |
+|    iterations           | 72           |
+|    time_elapsed         | 974          |
+|    total_timesteps      | 147456       |
+| train/                  |              |
+|    approx_kl            | 0.0048819017 |
+|    clip_fraction        | 0.0321       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.16        |
+|    explained_variance   | -1.19e-07    |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.66e+06     |
+|    n_updates            | 710          |
+|    policy_gradient_loss | -0.00324     |
+|    std                  | 0.877        |
+|    value_loss           | 3.4e+06      |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -7.08e+04    |
+| time/                   |              |
+|    fps                  | 150          |
+|    iterations           | 73           |
+|    time_elapsed         | 994          |
+|    total_timesteps      | 149504       |
+| train/                  |              |
+|    approx_kl            | 0.0051534167 |
+|    clip_fraction        | 0.0276       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.15        |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.15e+06     |
+|    n_updates            | 720          |
+|    policy_gradient_loss | -0.0034      |
+|    std                  | 0.877        |
+|    value_loss           | 2.73e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -7.17e+04    |
+| time/                   |              |
+|    fps                  | 150          |
+|    iterations           | 74           |
+|    time_elapsed         | 1009         |
+|    total_timesteps      | 151552       |
+| train/                  |              |
+|    approx_kl            | 0.0039522136 |
+|    clip_fraction        | 0.0215       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.17        |
+|    explained_variance   | 5.96e-08     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 8.39e+05     |
+|    n_updates            | 730          |
+|    policy_gradient_loss | -0.00278     |
+|    std                  | 0.885        |
+|    value_loss           | 1.82e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -7.25e+04    |
+| time/                   |              |
+|    fps                  | 150          |
+|    iterations           | 75           |
+|    time_elapsed         | 1023         |
+|    total_timesteps      | 153600       |
+| train/                  |              |
+|    approx_kl            | 0.0037896927 |
+|    clip_fraction        | 0.0131       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.18        |
+|    explained_variance   | 1.19e-07     |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.18e+06     |
+|    n_updates            | 740          |
+|    policy_gradient_loss | -0.00183     |
+|    std                  | 0.883        |
+|    value_loss           | 2.15e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -7.33e+04   |
+| time/                   |             |
+|    fps                  | 149         |
+|    iterations           | 76          |
+|    time_elapsed         | 1038        |
+|    total_timesteps      | 155648      |
+| train/                  |             |
+|    approx_kl            | 0.005035511 |
+|    clip_fraction        | 0.0261      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.16       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 6.91e+05    |
+|    n_updates            | 750         |
+|    policy_gradient_loss | -0.00347    |
+|    std                  | 0.877       |
+|    value_loss           | 1.49e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -7.45e+04   |
+| time/                   |             |
+|    fps                  | 149         |
+|    iterations           | 77          |
+|    time_elapsed         | 1052        |
+|    total_timesteps      | 157696      |
+| train/                  |             |
+|    approx_kl            | 0.005323178 |
+|    clip_fraction        | 0.0373      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.15       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 8.95e+05    |
+|    n_updates            | 760         |
+|    policy_gradient_loss | -0.00419    |
+|    std                  | 0.876       |
+|    value_loss           | 1.9e+06     |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -7.61e+04   |
+| time/                   |             |
+|    fps                  | 149         |
+|    iterations           | 78          |
+|    time_elapsed         | 1071        |
+|    total_timesteps      | 159744      |
+| train/                  |             |
+|    approx_kl            | 0.005339088 |
+|    clip_fraction        | 0.0279      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.13       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.56e+06    |
+|    n_updates            | 770         |
+|    policy_gradient_loss | -0.00348    |
+|    std                  | 0.871       |
+|    value_loss           | 3.12e+06    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -7.72e+04    |
+| time/                   |              |
+|    fps                  | 148          |
+|    iterations           | 79           |
+|    time_elapsed         | 1088         |
+|    total_timesteps      | 161792       |
+| train/                  |              |
+|    approx_kl            | 0.0021434259 |
+|    clip_fraction        | 0.00132      |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.13        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.39e+06     |
+|    n_updates            | 780          |
+|    policy_gradient_loss | -0.000295    |
+|    std                  | 0.874        |
+|    value_loss           | 3.14e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -7.88e+04   |
+| time/                   |             |
+|    fps                  | 148         |
+|    iterations           | 80          |
+|    time_elapsed         | 1106        |
+|    total_timesteps      | 163840      |
+| train/                  |             |
+|    approx_kl            | 0.004515986 |
+|    clip_fraction        | 0.0185      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.13       |
+|    explained_variance   | -2.38e-07   |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.26e+06    |
+|    n_updates            | 790         |
+|    policy_gradient_loss | -0.00181    |
+|    std                  | 0.872       |
+|    value_loss           | 2.6e+06     |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -8.02e+04   |
+| time/                   |             |
+|    fps                  | 147         |
+|    iterations           | 81          |
+|    time_elapsed         | 1125        |
+|    total_timesteps      | 165888      |
+| train/                  |             |
+|    approx_kl            | 0.004079287 |
+|    clip_fraction        | 0.0139      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.13       |
+|    explained_variance   | 5.96e-08    |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.86e+06    |
+|    n_updates            | 800         |
+|    policy_gradient_loss | -0.00189    |
+|    std                  | 0.873       |
+|    value_loss           | 3.26e+06    |
+-----------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -8.2e+04    |
+| time/                   |             |
+|    fps                  | 146         |
+|    iterations           | 82          |
+|    time_elapsed         | 1147        |
+|    total_timesteps      | 167936      |
+| train/                  |             |
+|    approx_kl            | 0.004386483 |
+|    clip_fraction        | 0.0127      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.13       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 1.87e+06    |
+|    n_updates            | 810         |
+|    policy_gradient_loss | -0.00205    |
+|    std                  | 0.873       |
+|    value_loss           | 3.56e+06    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -8.37e+04    |
+| time/                   |              |
+|    fps                  | 145          |
+|    iterations           | 83           |
+|    time_elapsed         | 1167         |
+|    total_timesteps      | 169984       |
+| train/                  |              |
+|    approx_kl            | 0.0041227336 |
+|    clip_fraction        | 0.0245       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.14        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.95e+06     |
+|    n_updates            | 820          |
+|    policy_gradient_loss | -0.00324     |
+|    std                  | 0.873        |
+|    value_loss           | 3.58e+06     |
+------------------------------------------
+----------------------------------------
+| rollout/                |            |
+|    ep_len_mean          | 1e+03      |
+|    ep_rew_mean          | -8.62e+04  |
+| time/                   |            |
+|    fps                  | 145        |
+|    iterations           | 84         |
+|    time_elapsed         | 1180       |
+|    total_timesteps      | 172032     |
+| train/                  |            |
+|    approx_kl            | 0.00430945 |
+|    clip_fraction        | 0.0171     |
+|    clip_range           | 0.2        |
+|    entropy_loss         | -5.13      |
+|    explained_variance   | 0          |
+|    learning_rate        | 0.0003     |
+|    loss                 | 2.18e+06   |
+|    n_updates            | 830        |
+|    policy_gradient_loss | -0.0021    |
+|    std                  | 0.872      |
+|    value_loss           | 3.97e+06   |
+----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -8.81e+04    |
+| time/                   |              |
+|    fps                  | 145          |
+|    iterations           | 85           |
+|    time_elapsed         | 1198         |
+|    total_timesteps      | 174080       |
+| train/                  |              |
+|    approx_kl            | 0.0027071913 |
+|    clip_fraction        | 0.0043       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.14        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.6e+06      |
+|    n_updates            | 840          |
+|    policy_gradient_loss | -0.000301    |
+|    std                  | 0.876        |
+|    value_loss           | 3.94e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -8.97e+04   |
+| time/                   |             |
+|    fps                  | 144         |
+|    iterations           | 86          |
+|    time_elapsed         | 1218        |
+|    total_timesteps      | 176128      |
+| train/                  |             |
+|    approx_kl            | 0.003243664 |
+|    clip_fraction        | 0.0133      |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.15       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.15e+06    |
+|    n_updates            | 850         |
+|    policy_gradient_loss | -0.00204    |
+|    std                  | 0.878       |
+|    value_loss           | 4.5e+06     |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -9.15e+04    |
+| time/                   |              |
+|    fps                  | 143          |
+|    iterations           | 87           |
+|    time_elapsed         | 1239         |
+|    total_timesteps      | 178176       |
+| train/                  |              |
+|    approx_kl            | 0.0029459428 |
+|    clip_fraction        | 0.00308      |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.16        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.44e+06     |
+|    n_updates            | 860          |
+|    policy_gradient_loss | -0.000751    |
+|    std                  | 0.88         |
+|    value_loss           | 4.24e+06     |
+------------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -9.36e+04    |
+| time/                   |              |
+|    fps                  | 143          |
+|    iterations           | 88           |
+|    time_elapsed         | 1258         |
+|    total_timesteps      | 180224       |
+| train/                  |              |
+|    approx_kl            | 0.0043680565 |
+|    clip_fraction        | 0.0208       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.17        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 2.09e+06     |
+|    n_updates            | 870          |
+|    policy_gradient_loss | -0.00237     |
+|    std                  | 0.881        |
+|    value_loss           | 4.39e+06     |
+------------------------------------------
+-----------------------------------------
+| rollout/                |             |
+|    ep_len_mean          | 1e+03       |
+|    ep_rew_mean          | -9.52e+04   |
+| time/                   |             |
+|    fps                  | 142         |
+|    iterations           | 89          |
+|    time_elapsed         | 1283        |
+|    total_timesteps      | 182272      |
+| train/                  |             |
+|    approx_kl            | 0.004063189 |
+|    clip_fraction        | 0.012       |
+|    clip_range           | 0.2         |
+|    entropy_loss         | -5.16       |
+|    explained_variance   | 0           |
+|    learning_rate        | 0.0003      |
+|    loss                 | 2.77e+06    |
+|    n_updates            | 880         |
+|    policy_gradient_loss | -0.00207    |
+|    std                  | 0.877       |
+|    value_loss           | 4.73e+06    |
+-----------------------------------------
+------------------------------------------
+| rollout/                |              |
+|    ep_len_mean          | 1e+03        |
+|    ep_rew_mean          | -9.69e+04    |
+| time/                   |              |
+|    fps                  | 141          |
+|    iterations           | 90           |
+|    time_elapsed         | 1306         |
+|    total_timesteps      | 184320       |
+| train/                  |              |
+|    approx_kl            | 0.0049707354 |
+|    clip_fraction        | 0.0279       |
+|    clip_range           | 0.2          |
+|    entropy_loss         | -5.13        |
+|    explained_variance   | 0            |
+|    learning_rate        | 0.0003       |
+|    loss                 | 1.65e+06     |
+|    n_updates            | 890          |
+|    policy_gradient_loss | -0.00372     |
+|    std                  | 0.871        |
+|    value_loss           | 3.68e+06     |
+------------------------------------------

train_log_full.txt

@@ -0,0 +1,64 @@
+Setting up Training Environment...
+Creating Liquid PPO Agent...
+Using cpu device
+Wrapping the env with a `Monitor` wrapper
+Wrapping the env in a DummyVecEnv.
+Starting Training (This may take a while)...
+----------------------------------
+| rollout/           |           |
+|    ep_len_mean     | 1e+03     |
+|    ep_rew_mean     | -2.12e+04 |
+| time/              |           |
+|    fps             | 464       |
+|    iterations      | 1         |
+|    time_elapsed    | 4         |
+|    total_timesteps | 2048      |
+----------------------------------
+Traceback (most recent call last):
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/train.py", line 35, in <module>
+    train()
+    ~~~~~^^
+  File "/home/ylop/Documents/drone go brr/Drone-go-brrrrr/Drone-go-brrrrr/train.py", line 28, in train
+    model.learn(total_timesteps=total_timesteps, callback=checkpoint_callback)
+    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/ppo/ppo.py", line 311, in learn
+    return super().learn(
+           ~~~~~~~~~~~~~^
+        total_timesteps=total_timesteps,
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    ...<4 lines>...
+        progress_bar=progress_bar,
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/common/on_policy_algorithm.py", line 337, in learn
+    self.train()
+    ~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/stable_baselines3/ppo/ppo.py", line 275, in train
+    loss.backward()
+    ~~~~~~~~~~~~~^^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/_tensor.py", line 625, in backward
+    torch.autograd.backward(
+    ~~~~~~~~~~~~~~~~~~~~~~~^
+        self, gradient, retain_graph, create_graph, inputs=inputs
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/autograd/__init__.py", line 354, in backward
+    _engine_run_backward(
+    ~~~~~~~~~~~~~~~~~~~~^
+        tensors,
+        ^^^^^^^^
+    ...<5 lines>...
+        accumulate_grad=True,
+        ^^^^^^^^^^^^^^^^^^^^^
+    )
+    ^
+  File "/home/ylop/.local/lib/python3.14/site-packages/torch/autograd/graph.py", line 841, in _engine_run_backward
+    return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+        t_outputs, *args, **kwargs
+        ^^^^^^^^^^^^^^^^^^^^^^^^^^
+    )  # Calls into the C++ engine to run the backward pass
+    ^
+RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.