Instructions to use AesSedai/Step-3.7-Flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AesSedai/Step-3.7-Flash-GGUF with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="AesSedai/Step-3.7-Flash-GGUF",
	filename="IQ2_S/Step-3.7-Flash-IQ2_S-00001-of-00003.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use AesSedai/Step-3.7-Flash-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

LM Studio
Jan
Ollama
How to use AesSedai/Step-3.7-Flash-GGUF with Ollama:
```
ollama run hf.co/AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
```

Unsloth Studio

How to use AesSedai/Step-3.7-Flash-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/Step-3.7-Flash-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AesSedai/Step-3.7-Flash-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AesSedai/Step-3.7-Flash-GGUF to start chatting

How to use AesSedai/Step-3.7-Flash-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AesSedai/Step-3.7-Flash-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use AesSedai/Step-3.7-Flash-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat new

OpenClaw new

How to use AesSedai/Step-3.7-Flash-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AesSedai/Step-3.7-Flash-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Docker Model Runner
How to use AesSedai/Step-3.7-Flash-GGUF with Docker Model Runner:
```
docker model run hf.co/AesSedai/Step-3.7-Flash-GGUF:Q4_K_M
```

Lemonade

How to use AesSedai/Step-3.7-Flash-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AesSedai/Step-3.7-Flash-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Step-3.7-Flash-GGUF-Q4_K_M

List all available models

lemonade list

Step-3.7-Flash-GGUF / kld_data /llm_quantization_data.csv

AesSedai

Super-squash branch 'main' using huggingface_hub

044920b about 1 month ago

Raw

History Blame Contribute Delete

27.2 kB

	model_name,file_size_gb,bpw,Mean KLD_mean,0.01.226.187 I common_memory_breakdown_print,0.01.241.186 I common_memory_breakdown_print,0.01.246.900 I common_params_fit_impl,0.01.246.905 I common_fit_params,0.01.254.635 I common_memory_breakdown_print,0.01.261.148 I common_params_fit_impl,0.01.261.154 I common_fit_params,0.01.267.573 I llama_model_loader,0.01.267.574 I llama_model_loader,0.01.267.587 I llama_model_loader,0.01.267.588 I llama_model_loader,0.01.267.589 I llama_model_loader,0.01.267.590 I llama_model_loader,0.01.267.597 I llama_model_loader,0.01.267.606 I llama_model_loader,0.01.267.607 I llama_model_loader,0.01.267.608 I llama_model_loader,0.01.267.609 I llama_model_loader,0.01.267.610 I llama_model_loader,0.01.267.611 I llama_model_loader,0.01.267.623 I llama_model_loader,0.01.274.576 I common_params_fit_impl,0.01.274.581 I common_fit_params,0.01.281.782 I llama_model_loader,0.01.281.783 I llama_model_loader,0.01.281.784 I llama_model_loader,0.01.281.799 I llama_model_loader,0.01.281.800 I llama_model_loader,0.01.281.801 I llama_model_loader,0.01.281.809 I llama_model_loader,0.01.281.816 I llama_model_loader,0.01.281.817 I llama_model_loader,0.01.281.818 I llama_model_loader,0.01.281.819 I llama_model_loader,0.01.281.831 I llama_model_loader,0.01.284.145 I llama_model_loader,0.01.284.146 I llama_model_loader,0.01.284.147 I llama_model_loader,0.01.284.148 I print_info,0.01.284.149 I print_info,0.01.295.299 I llama_model_loader,0.01.295.300 I llama_model_loader,0.01.295.301 I llama_model_loader,0.01.295.314 I llama_model_loader,0.01.295.315 I llama_model_loader,0.01.295.316 I llama_model_loader,0.01.295.326 I llama_model_loader,0.01.295.332 I llama_model_loader,0.01.295.333 I llama_model_loader,0.01.295.334 I llama_model_loader,0.01.295.366 I llama_model_loader,0.01.295.367 I llama_model_loader,0.01.295.368 I llama_model_loader,0.01.295.383 I llama_model_loader,0.01.297.892 I llama_model_loader,0.01.297.893 I llama_model_loader,0.01.297.896 I llama_model_loader,0.01.297.897 I llama_model_loader,0.01.297.898 I print_info,0.01.311.559 I llama_model_loader,0.01.311.560 I llama_model_loader,0.01.311.561 I llama_model_loader,0.01.311.563 I llama_model_loader,0.01.311.564 I llama_model_loader,0.01.311.565 I print_info,0.01.311.566 I print_info,0.01.321.836 I load,0.01.329.804 I load,0.01.329.805 I load,0.01.329.868 I load,0.01.335.287 I load,0.01.343.243 I load,0.01.343.244 I load,0.01.343.311 I load,0.01.349.155 I load,0.01.351.659 I load,0.01.351.674 I print_info,0.01.351.675 I print_info,0.01.351.676 I print_info,0.01.351.683 I print_info,0.01.351.685 I print_info,0.01.351.686 I print_info,0.01.351.688 I print_info,0.01.351.689 I print_info,0.01.351.690 I print_info,0.01.351.691 I print_info,0.01.351.692 I print_info,0.01.351.693 I print_info,0.01.351.694 I print_info,0.01.351.695 I print_info,0.01.351.696 I print_info,0.01.351.697 I print_info,0.01.351.698 I print_info,0.01.351.699 I print_info,0.01.351.700 I print_info,0.01.351.701 I print_info,0.01.351.702 I print_info,0.01.357.173 I load,0.01.357.174 I load,0.01.357.234 I load,0.01.364.998 I load,0.01.365.013 I print_info,0.01.365.014 I print_info,0.01.365.015 I print_info,0.01.365.023 I print_info,0.01.365.024 I print_info,0.01.365.025 I print_info,0.01.365.027 I print_info,0.01.365.029 I print_info,0.01.365.030 I print_info,0.01.365.032 I print_info,0.01.365.033 I print_info,0.01.365.034 I print_info,0.01.365.035 I print_info,0.01.365.050 I print_info,0.01.365.053 I print_info,0.01.365.054 I print_info,0.01.365.055 I print_info,0.01.365.056 I print_info,0.01.365.057 I print_info,0.01.365.058 I print_info,0.01.365.059 I print_info,0.01.365.060 I print_info,0.01.365.061 I print_info,0.01.365.062 I print_info,0.01.365.066 I print_info,0.01.365.067 I print_info,0.01.365.068 I print_info,0.01.365.069 I print_info,0.01.365.070 I print_info,0.01.365.071 I print_info,0.01.365.072 I print_info,0.01.378.991 I load,0.01.379.005 I print_info,0.01.379.006 I print_info,0.01.379.007 I print_info,0.01.379.015 I print_info,0.01.379.016 I print_info,0.01.379.017 I print_info,0.01.379.019 I print_info,0.01.379.021 I print_info,0.01.379.022 I print_info,0.01.379.023 I print_info,0.01.379.024 I print_info,0.01.379.025 I print_info,0.01.379.026 I print_info,0.01.379.027 I print_info,0.01.379.028 I print_info,0.01.379.029 I print_info,0.01.379.031 I print_info,0.01.379.032 I print_info,0.01.379.033 I print_info,0.01.379.034 I print_info,0.01.379.035 I print_info,0.01.379.036 I print_info,0.01.379.037 I print_info,0.01.887.699 I common_memory_breakdown_print,0.01.909.075 I common_params_fit_impl,0.01.909.080 I common_fit_params,0.01.929.961 I llama_model_loader,0.01.929.962 I llama_model_loader,0.01.929.974 I llama_model_loader,0.01.929.975 I llama_model_loader,0.01.929.976 I llama_model_loader,0.01.929.977 I llama_model_loader,0.01.929.985 I llama_model_loader,0.01.929.993 I llama_model_loader,0.01.929.994 I llama_model_loader,0.01.929.997 I llama_model_loader,0.01.929.999 I llama_model_loader,0.01.930.013 I llama_model_loader,0.01.946.146 I llama_model_loader,0.01.946.147 I llama_model_loader,0.01.946.148 I llama_model_loader,0.01.946.151 I llama_model_loader,0.01.946.152 I llama_model_loader,0.01.946.153 I print_info,0.01.983.955 I load,0.01.991.566 I common_memory_breakdown_print,0.01.991.994 I load,0.01.991.995 I load,0.01.992.061 I load,0.02.013.317 I common_params_fit_impl,0.02.013.321 I common_fit_params,0.02.013.774 I load,0.02.013.789 I print_info,0.02.013.790 I print_info,0.02.013.791 I print_info,0.02.013.801 I print_info,0.02.013.802 I print_info,0.02.013.803 I print_info,0.02.013.805 I print_info,0.02.013.807 I print_info,0.02.013.808 I print_info,0.02.013.809 I print_info,0.02.013.810 I print_info,0.02.013.811 I print_info,0.02.013.812 I print_info,0.02.013.813 I print_info,0.02.013.814 I print_info,0.02.013.815 I print_info,0.02.013.816 I print_info,0.02.013.817 I print_info,0.02.013.819 I print_info,0.02.013.820 I print_info,0.02.013.821 I print_info,0.02.034.314 I llama_model_loader,0.02.034.315 I llama_model_loader,0.02.034.328 I llama_model_loader,0.02.034.329 I llama_model_loader,0.02.034.330 I llama_model_loader,0.02.034.337 I llama_model_loader,0.02.034.342 I llama_model_loader,0.02.034.343 I llama_model_loader,0.02.034.344 I llama_model_loader,0.02.034.345 I llama_model_loader,0.02.034.361 I llama_model_loader,0.02.050.530 I llama_model_loader,0.02.050.531 I llama_model_loader,0.02.050.534 I llama_model_loader,0.02.050.535 I print_info,0.02.050.536 I print_info,0.02.088.221 I load,0.02.096.156 I load,0.02.096.157 I load,0.02.096.222 I load,0.02.118.190 I load,0.02.118.205 I print_info,0.02.118.206 I print_info,0.02.118.214 I print_info,0.02.118.215 I print_info,0.02.118.216 I print_info,0.02.118.218 I print_info,0.02.118.220 I print_info,0.02.118.221 I print_info,0.02.118.223 I print_info,0.02.118.224 I print_info,0.02.118.225 I print_info,0.02.118.226 I print_info,0.02.118.227 I print_info,0.02.118.228 I print_info,0.02.118.229 I print_info,0.02.118.230 I print_info,0.02.118.232 I print_info,0.02.118.233 I print_info,0.02.118.234 I print_info,0.02.293.742 I common_memory_breakdown_print,0.02.313.473 I common_params_fit_impl,0.02.313.478 I common_fit_params,0.02.334.361 I llama_model_loader,0.02.334.362 I llama_model_loader,0.02.334.375 I llama_model_loader,0.02.334.377 I llama_model_loader,0.02.334.378 I llama_model_loader,0.02.334.387 I llama_model_loader,0.02.334.402 I llama_model_loader,0.02.334.403 I llama_model_loader,0.02.334.404 I llama_model_loader,0.02.334.405 I llama_model_loader,0.02.334.441 I llama_model_loader,0.02.350.945 I llama_model_loader,0.02.350.946 I llama_model_loader,0.02.350.949 I llama_model_loader,0.02.350.951 I print_info,0.02.388.500 I load,0.02.396.641 I load,0.02.396.718 I load,0.02.418.378 I load,0.02.418.393 I print_info,0.02.418.394 I print_info,0.02.418.395 I print_info,0.02.418.403 I print_info,0.02.418.404 I print_info,0.02.418.405 I print_info,0.02.418.406 I print_info,0.02.418.408 I print_info,0.02.418.409 I print_info,0.02.418.410 I print_info,0.02.418.411 I print_info,0.02.418.412 I print_info,0.02.418.413 I print_info,0.02.418.414 I print_info,0.02.418.415 I print_info,0.02.418.416 I print_info,0.02.418.418 I print_info,0.02.418.419 I print_info,0.02.418.420 I print_info,0.02.418.421 I print_info,0.02.418.423 I print_info,0.02.418.424 I print_info,0.1% KLD,0.1% Δp,0.24.864.570 I load_tensors,0.24.864.576 I load_tensors,0.24.864.577 I load_tensors,0.24.864.578 I load_tensors,0.24.864.579 I load_tensors,0.28.741.786 I llama_context,0.28.741.787 I llama_context,0.28.741.792 I llama_context,0.28.741.793 I llama_context,0.28.744.871 I llama_context,0.28.744.881 I llama_kv_cache_iswa,0.28.745.178 I llama_kv_cache,0.28.745.410 I llama_kv_cache,0.28.745.641 I llama_kv_cache,0.28.745.840 I llama_kv_cache,0.28.746.041 I llama_kv_cache,0.28.746.237 I llama_kv_cache,0.28.746.450 I llama_kv_cache,0.28.746.678 I llama_kv_cache,0.28.746.723 I llama_kv_cache,0.28.746.724 I llama_kv_cache,0.28.746.725 I llama_kv_cache_iswa,0.28.747.026 I llama_kv_cache,0.28.747.297 I llama_kv_cache,0.28.747.573 I llama_kv_cache,0.28.747.826 I llama_kv_cache,0.28.748.069 I llama_kv_cache,0.28.748.327 I llama_kv_cache,0.28.748.611 I llama_kv_cache,0.28.748.863 I llama_kv_cache,0.28.748.949 I llama_kv_cache,0.28.825.896 I sched_reserve,0.28.825.909 I sched_reserve,0.28.825.910 I sched_reserve,0.28.825.911 I sched_reserve,0.28.825.912 I sched_reserve,0.28.825.913 I sched_reserve,0.28.825.914 I sched_reserve,0.28.903.914 I system_info,0.29.915.639 I kl_divergence,0.34.358.727 I load_tensors,0.34.358.738 I load_tensors,0.34.358.744 I load_tensors,0.34.358.745 I load_tensors,0.34.358.746 I load_tensors,0.34.358.753 I load_tensors,0.34.358.754 I load_tensors,0.34.400.842 I load_tensors,0.34.400.843 I load_tensors,0.34.400.848 I load_tensors,0.34.400.849 I load_tensors,0.34.400.850 I load_tensors,0.34.400.851 I load_tensors,0.37.621.931 I load_tensors,0.37.621.937 I load_tensors,0.37.621.938 I load_tensors,0.37.621.939 I load_tensors,0.37.621.940 I load_tensors,0.41.341.259 I llama_context,0.41.341.260 I llama_context,0.41.341.261 I llama_context,0.41.341.265 I llama_context,0.41.341.266 I llama_context,0.41.345.056 I llama_context,0.41.345.065 I llama_kv_cache_iswa,0.41.345.350 I llama_kv_cache,0.41.345.613 I llama_kv_cache,0.41.345.830 I llama_kv_cache,0.41.346.028 I llama_kv_cache,0.41.346.226 I llama_kv_cache,0.41.346.437 I llama_kv_cache,0.41.346.673 I llama_kv_cache,0.41.346.870 I llama_kv_cache,0.41.346.917 I llama_kv_cache,0.41.346.918 I llama_kv_cache,0.41.346.919 I llama_kv_cache_iswa,0.41.347.241 I llama_kv_cache,0.41.347.543 I llama_kv_cache,0.41.347.782 I llama_kv_cache,0.41.348.028 I llama_kv_cache,0.41.348.265 I llama_kv_cache,0.41.348.536 I llama_kv_cache,0.41.348.767 I llama_kv_cache,0.41.349.022 I llama_kv_cache,0.41.349.106 I llama_kv_cache,0.41.430.896 I sched_reserve,0.41.430.908 I sched_reserve,0.41.430.909 I sched_reserve,0.41.430.910 I sched_reserve,0.41.430.911 I sched_reserve,0.41.430.912 I sched_reserve,0.41.519.094 I system_info,0.41.534.735 I kl_divergence,0.43.673.874 I llama_context,0.43.673.875 I llama_context,0.43.673.876 I llama_context,0.43.673.881 I llama_context,0.43.673.882 I llama_context,0.43.677.130 I llama_context,0.43.677.140 I llama_kv_cache_iswa,0.43.677.459 I llama_kv_cache,0.43.677.678 I llama_kv_cache,0.43.677.875 I llama_kv_cache,0.43.678.071 I llama_kv_cache,0.43.678.269 I llama_kv_cache,0.43.678.489 I llama_kv_cache,0.43.678.688 I llama_kv_cache,0.43.678.874 I llama_kv_cache,0.43.678.914 I llama_kv_cache,0.43.678.915 I llama_kv_cache_iswa,0.43.679.212 I llama_kv_cache,0.43.679.502 I llama_kv_cache,0.43.679.749 I llama_kv_cache,0.43.679.994 I llama_kv_cache,0.43.680.224 I llama_kv_cache,0.43.680.815 I llama_kv_cache,0.43.681.044 I llama_kv_cache,0.43.681.284 I llama_kv_cache,0.43.681.379 I llama_kv_cache,0.43.681.380 I llama_kv_cache,0.43.766.572 I sched_reserve,0.43.766.586 I sched_reserve,0.43.766.587 I sched_reserve,0.43.766.588 I sched_reserve,0.43.766.589 I sched_reserve,0.43.766.590 I sched_reserve,0.43.766.591 I sched_reserve,0.43.846.652 I system_info,0.44.907.090 I load_tensors,0.44.907.096 I load_tensors,0.44.907.097 I load_tensors,0.44.907.098 I load_tensors,0.44.907.099 I load_tensors,0.45.782.539 I kl_divergence,0.51.525.796 I llama_context,0.51.525.797 I llama_context,0.51.525.798 I llama_context,0.51.525.802 I llama_context,0.51.525.803 I llama_context,0.51.529.119 I llama_context,0.51.529.127 I llama_kv_cache_iswa,0.51.529.395 I llama_kv_cache,0.51.529.612 I llama_kv_cache,0.51.529.814 I llama_kv_cache,0.51.530.016 I llama_kv_cache,0.51.530.215 I llama_kv_cache,0.51.530.403 I llama_kv_cache,0.51.530.602 I llama_kv_cache,0.51.530.789 I llama_kv_cache,0.51.530.834 I llama_kv_cache,0.51.530.835 I llama_kv_cache,0.51.530.836 I llama_kv_cache_iswa,0.51.531.181 I llama_kv_cache,0.51.531.435 I llama_kv_cache,0.51.531.670 I llama_kv_cache,0.51.531.920 I llama_kv_cache,0.51.532.182 I llama_kv_cache,0.51.532.427 I llama_kv_cache,0.51.532.663 I llama_kv_cache,0.51.532.912 I llama_kv_cache,0.51.532.997 I llama_kv_cache,0.51.620.447 I sched_reserve,0.51.620.460 I sched_reserve,0.51.620.461 I sched_reserve,0.51.620.462 I sched_reserve,0.51.620.463 I sched_reserve,0.51.620.464 I sched_reserve,0.51.620.465 I sched_reserve,0.51.620.466 I sched_reserve,0.51.620.467 I sched_reserve,0.51.703.860 I system_info,0.53.679.349 I kl_divergence,1.0% KLD,1.0% Δp,1.05.261.111 I load_tensors,1.05.261.112 I load_tensors,1.05.261.117 I load_tensors,1.05.261.118 I load_tensors,1.05.261.119 I load_tensors,1.05.261.120 I load_tensors,1.15.612.193 I llama_context,1.15.612.194 I llama_context,1.15.612.195 I llama_context,1.15.612.199 I llama_context,1.15.612.200 I llama_context,1.15.615.366 I llama_context,1.15.615.375 I llama_kv_cache_iswa,1.15.615.660 I llama_kv_cache,1.15.615.892 I llama_kv_cache,1.15.616.104 I llama_kv_cache,1.15.616.306 I llama_kv_cache,1.15.616.507 I llama_kv_cache,1.15.616.699 I llama_kv_cache,1.15.616.900 I llama_kv_cache,1.15.617.100 I llama_kv_cache,1.15.617.160 I llama_kv_cache,1.15.617.162 I llama_kv_cache_iswa,1.15.617.472 I llama_kv_cache,1.15.617.739 I llama_kv_cache,1.15.617.980 I llama_kv_cache,1.15.618.269 I llama_kv_cache,1.15.618.522 I llama_kv_cache,1.15.618.777 I llama_kv_cache,1.15.619.020 I llama_kv_cache,1.15.627.687 I llama_kv_cache,1.15.627.806 I llama_kv_cache,1.15.707.396 I sched_reserve,1.15.707.410 I sched_reserve,1.15.707.411 I sched_reserve,1.15.707.412 I sched_reserve,1.15.707.413 I sched_reserve,1.15.707.414 I sched_reserve,1.15.797.678 I system_info,1.16.130.613 I kl_divergence,1.18.838.235 I llama_context,1.18.838.236 I llama_context,1.18.838.241 I llama_context,1.18.838.242 I llama_context,1.18.842.512 I llama_context,1.18.842.520 I llama_kv_cache_iswa,1.18.842.873 I llama_kv_cache,1.18.852.621 I llama_kv_cache,1.18.859.213 I llama_kv_cache,1.18.866.376 I llama_kv_cache,1.18.878.894 I llama_kv_cache,1.18.880.736 I llama_kv_cache,1.18.880.955 I llama_kv_cache,1.18.881.169 I llama_kv_cache,1.18.881.212 I llama_kv_cache,1.18.881.213 I llama_kv_cache,1.18.881.215 I llama_kv_cache_iswa,1.18.881.708 I llama_kv_cache,1.18.881.985 I llama_kv_cache,1.18.882.232 I llama_kv_cache,1.18.882.489 I llama_kv_cache,1.18.882.757 I llama_kv_cache,1.18.883.005 I llama_kv_cache,1.18.883.237 I llama_kv_cache,1.18.883.487 I llama_kv_cache,1.18.883.573 I llama_kv_cache,1.18.963.096 I sched_reserve,1.18.963.109 I sched_reserve,1.18.963.110 I sched_reserve,1.18.963.111 I sched_reserve,1.18.963.112 I sched_reserve,1.18.963.113 I sched_reserve,1.18.963.114 I sched_reserve,1.19.064.073 I system_info,1.19.080.254 I kl_divergence,10.0% KLD,10.0% Δp,2.01.147.910 I llama_perf_context_print,2.01.147.914 I llama_perf_context_print,2.01.147.915 I llama_perf_context_print,2.01.148.129 I common_memory_breakdown_print,2.02.461.451 I llama_perf_context_print,2.02.461.456 I llama_perf_context_print,2.02.461.713 I common_memory_breakdown_print,2.07.127.592 I llama_perf_context_print,2.07.127.596 I llama_perf_context_print,2.07.127.815 I common_memory_breakdown_print,2.17.644.210 I llama_perf_context_print,2.17.644.213 I llama_perf_context_print,2.17.644.433 I common_memory_breakdown_print,2.40.769.870 I llama_perf_context_print,2.40.769.874 I llama_perf_context_print,2.40.770.081 I common_memory_breakdown_print,2.54.184.274 I llama_perf_context_print,2.54.184.277 I llama_perf_context_print,2.54.184.517 I common_memory_breakdown_print,25.0% Δp,5.0% KLD,5.0% Δp,75.0% Δp,90.0% KLD,90.0% Δp,95.0% KLD,95.0% Δp,99.0% KLD,99.0% Δp,99.9% KLD,99.9% Δp,"Cor(ln(PPL(Q)), ln(PPL(base)))",Maximum KLD,Maximum Δp,Mean KLD_std,Mean PPL(Q)-PPL(base)_mean,Mean PPL(Q)-PPL(base)_std,Mean PPL(Q)/PPL(base)_mean,Mean PPL(Q)/PPL(base)_std,Mean PPL(Q)_mean,Mean PPL(Q)_std,Mean PPL(base)_mean,Mean PPL(base)_std,Mean ln(PPL(Q)/PPL(base))_mean,Mean ln(PPL(Q)/PPL(base))_std,Mean Δp_mean,Mean Δp_std,Median KLD,Median Δp,Minimum KLD,Minimum Δp,RMS Δp_mean,RMS Δp_std,Same top p_mean,Same top p_std,file_path,file_size_gib,is_self_ref,mixture
	Step-3.7-Flash-IQ2_S (aes_sedai),69.18118572032002,2.78,0.623856,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-7344130321.0,97053.773503,0.91,-0.35,-3.2881,-6.1,-935.324096,-1035.3211264,-1435.322,-2135.328,-2335.32128,-2535.0,-2735.321,-3135.323,-45.3218,-49.6,-342.0,6.0,0.0,-128007.0,818.0,0.822,35.0,4096.0,48.0,6.496969664969697e+95,8.0,128.0,128.0,8.121212812121282e+83,1024.0,1024.0,0.0,0.0,0.0,0.0,2.0,5000000.0,1.0,262144.0,196.11,199.38,1.0,256.0,0.0,-99.377,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4949.0,413.03,6473.93,18864.55,48890.9,58864.55,75819.7,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1.3e-05,-95.885,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,8192.0,1.0,0.0,8192.0,7.87,-512.0,64.0,164.0,232.0,364.0,432.0,564.0,632.0,732.0,128.0,128.0,512.0,160.0,1128.0,2160.0,3128.0,4160.0,5128.0,6160.0,7128.0,128.0,3073.12,13073.12,23073.12,53073.12,74861.25,9.0,79.444,4.848561200112841e+50,561512819216.0,0.000796,-64.819,,,,,,,,,,,,,,,,,76712.5,34.0,-7344130321.0,-28.164,0.000175,-81.763,-0.057,1.87408,1.766,2.837528,10.103,5.044334,39.253,8.00725,74.111,70.38,12.892854,96.856,0.00281,1.550883,0.013518,1.819637,0.00661,3.443042,0.017577,1.892159,0.007192,0.598637,0.003632,-16.857,0.076,0.168675,-3.977,-8.5e-05,-99.995,33.234,0.086,74.2,0.116,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-IQ2_S.md,64.43,False,Q6_K / IQ2_XS / IQ2_XS / IQ3_XXS
	Step-3.7-Flash-IQ3_S (aes_sedai),76.11755790336001,3.05,0.459317,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-7344130321.0,103668.773503,1.43,-1.0,-3.2881,-6.1,-835.32262144,-935.324096,-1035.3211264,-1435.322,-2135.328,-2335.32128,-2535.0,-2735.321,-3135.323,-44.322,-48.3,-49.6,-342.0,-284.0,6.0,0.0,,-1.0,-128007.0,818.0,,,0.822,0.0,4096.0,48.0,6.496969664969697e+95,512.0,128.0,8.121212812121282e+83,1024.0,0.0,0.0,0.0,0.0,8.0,2.0,5000000.0,128.0,0.0,199.38,128007.0,1.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-0.0,-98.575,4949.0,413.03,29835.9,59809.55,76134.7,8192.0,1.0,0.0,8192.0,7.87,-512.0,64.0,164.0,232.0,364.0,432.0,564.0,632.0,732.0,128.0,128.0,512.0,160.0,1128.0,2160.0,3128.0,4160.0,5128.0,6160.0,7128.0,128.0,3073.12,13073.12,33073.12,63073.12,321.38,9.0,76.874,4.848561200112841e+50,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,6e-06,-92.396,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.000365,-52.983,26957.08,92244.23287233,34.0,-7344130321.0,,,,,,,,,,,,,,,,-19.145,8.5e-05,-72.513,-0.01,1.40186,2.639,2.186722,11.149,4.074806,39.705,6.777759,74.073,76.56,12.450217,97.187,0.002233,1.023676,0.009538,1.541009,0.004714,2.915835,0.013847,1.892159,0.007192,0.432438,0.003059,-12.8,0.068,0.101123,-1.924,-0.000422,-99.966,28.603,0.084,78.494,0.109,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-IQ3_S.md,70.89,False,Q6_K / IQ2_S / IQ2_S / IQ3_S
	Step-3.7-Flash-IQ4_XS (aes_sedai),98.04336594944,3.93,0.159543,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-8565340321.0,,,,124455.773503,1.56,,,,,,,,,,,,,,,,,,,,,,,-0.35,-3.2881,-6.1,-835.32262144,-1035.3211264,-1435.322,-2035.32288,-2235.32128,-2335.32128,-2735.321,-3135.323,-46.3,-49.8,-442.0,3.0,80.0,0.0,-1.0,-128007.0,818.0,0.822,0.0,48.0,6.496969664969697e+95,64.0,128.0,8.121212812121282e+83,1024.0,0.0,0.0,0.0,8.0,2.0,1.0,262144.0,196.11,199.38,128007.0,128802.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-3e-06,-88.841,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,47.0,4949.0,534.97,112734.95,412769.09,77229.18,,,,,,16.0,8192.0,1.0,5000000.0,8192.0,7.87,-512.0,64.0,164.0,232.0,364.0,432.0,564.0,632.0,732.0,128.0,128.0,512.0,160.0,1128.0,2160.0,3128.0,4160.0,5128.0,6160.0,7128.0,128.0,3073.12,23073.12,43073.12,74861.25,3422.0,81.724,4.848561200112841e+50,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1e-06,-67.566,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,7e-05,-22.352,,,,,39467.95,34.0,-8565340321.0,,,,,,,,,,,,,-5.301,1.7e-05,-37.828,0.016,0.463644,4.035,0.78791,11.519,1.729105,34.373,3.431495,64.945,91.08,16.556026,99.586,0.000943,0.294879,0.003875,1.155843,0.001973,2.187038,0.009041,1.892159,0.007192,0.14483,0.001707,-4.679,0.042,0.022848,-0.177,-0.000551,-99.425,16.65,0.068,88.125,0.086,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-IQ4_XS.md,91.31,False,Q8_0 / IQ3_S / IQ3_S / IQ4_XS
	Step-3.7-Flash-Q4_K_M (aes_sedai),124.79027478528,5.01,0.047917,,,,,-8565340321.0,,,,,,,,,,,,,,,,,149970.773503,0.82,,,,,,,,,,,,,,,,,,-0.35,-1.0,-3.2881,-6.1,-935.324096,-1035.3211264,-1435.322,-2035.32288,-2235.32128,-2335.32128,-2535.0,-2635.323,-2735.321,-3135.323,,,,,,-44.322,-48.5,-49.8,-32287.0,-542.0,3.0,80.0,,,,,,,,,0.0,,,,,,,,,,,,,,,,,,,,,,,-1.0,-128007.0,818.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,0.822,35.0,0.0,48.0,6.496969664969697e+95,64.0,128.0,8.121212812121282e+83,1024.0,1024.0,0.0,0.0,0.0,8.0,2.0,5000000.0,1.0,128.0,0.0,196.11,199.38,127741.0,128801.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-4e-06,-59.195,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4949.0,11650.76,216414.09,516379.95,78444.18,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,16.0,8192.0,1.0,0.0,8192.0,7.87,-512.0,64.0,164.0,232.0,364.0,432.0,564.0,632.0,732.0,128.0,512.0,160.0,1128.0,2160.0,3128.0,4160.0,5128.0,6160.0,7128.0,128.0,128.0,3073.12,13073.12,43073.12,74861.25,321.38,9.0,85.094,4.848561200112841e+50,,,,,,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-0.0,-35.717,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,1.5e-05,-8.195,,,,,,,,42534.46,34.0,-8565340321.0,,,,,,,,,,-1.252,4e-06,-15.941,0.202,0.134149,4.005,0.235283,9.397,0.56835,25.436,1.257683,47.392,97.26,4.269025,79.15,0.000315,0.0658,0.00178,1.034775,0.000935,1.957959,0.00761,1.892159,0.007192,0.034184,0.000903,-1.123,0.023,0.005487,-0.006,-0.000214,-90.666,8.942,0.046,93.684,0.064,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-Q4_K_M.md,116.22,False,Q8_0 / Q4_K / Q4_K / Q5_K
	Step-3.7-Flash-Q5_K_M (aes_sedai),149.06757742592004,5.98,0.017023,,-8565340321.0,,,,173122.773503,0.81,,,,,,,,,,,,,,,,,-0.35,-1.0,-3.2881,-6.1,-835.32262144,-1035.3211264,-1435.322,-2135.328,-2335.32128,-2535.0,-2735.321,-3135.323,,,,,,,,,,,,,,,,,,,,-46.5,-49.8,-584.0,-642.0,80.0,,,,,,,,,,,,0.0,-1.0,-128007.0,818.0,,,,,,,,,,,,,,,,,,,,,,,,,,,0.822,35.0,4096.0,48.0,6.496969664969697e+95,64.0,128.0,8.121212812121282e+83,1024.0,1024.0,0.0,0.0,0.0,0.0,0.0,11264.0,0.0,2.0,5000000.0,10000.0,128.0,128.0,262144.0,0.0,199.38,127741.0,128007.0,1.0,128801.0,128802.0,1.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-7e-06,-35.555,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,4949.0,534.97,119687.45,519687.45,79546.68,,16.0,8192.0,1.0,1.0,8192.0,7.87,-512.0,64.0,164.0,232.0,364.0,432.0,564.0,632.0,732.0,128.0,128.0,512.0,160.0,1128.0,2160.0,3128.0,4160.0,5128.0,6160.0,7128.0,128.0,3073.12,13073.12,33073.12,53073.12,63073.12,74861.25,3422.0,9.0,84.314,4.848561200112841e+50,561512819216.0,-1e-06,-19.76,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,6e-06,-4.025,,,,,,,,,,,50405.26,34.0,-8565340321.0,,,,,,,-0.496,2e-06,-8.166,0.208,0.04668,2.912,0.081936,6.418,0.204605,16.299,0.477511,32.075,99.0,2.310791,68.186,0.000119,0.019442,0.001035,1.010275,0.000546,1.911601,0.007329,1.892159,0.007192,0.010223,0.00054,-0.312,0.014,0.001903,-0.001,-0.000316,-67.357,5.28,0.03,96.21,0.05,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-Q5_K_M.md,138.83,False,Q8_0 / Q5_K / Q5_K / Q6_K
	Step-3.7-Flash-Q8_0 (aes_sedai),211.98884831232002,8.51,0.005301,-8565340321.0,,233130.773503,0.8,,,,-1.0,-3.2881,-6.1,-735.3248,-935.324096,-1035.3211264,-1435.322,-2135.328,-2235.32128,-2335.32128,-2535.0,-2635.323,-2735.321,-3135.323,,,,,,,,,,,,,,,-44.322,-45.327,-80518.0,3.0,80.0,,,,,,,,,,,,,,,,,,,,,,,,,,,0.0,-1.0,-128007.0,818.0,,,,,,0.822,35.0,0.0,48.0,6.496969664969697e+95,128.0,128.0,8.121212812121282e+83,1024.0,1024.0,0.0,0.0,11264.0,2.0,5000000.0,1.0,0.0,196.11,199.38,0.0,128802.0,256.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-7e-06,-20.53,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,-1e-06,-9.861,47.0,4949.0,534.97,128259.95,428294.09,712404.18,16.0,8192.0,1.0,1.0,8192.0,7.87,-512.0,64.0,164.0,232.0,364.0,432.0,564.0,632.0,732.0,128.0,512.0,160.0,1128.0,2160.0,3128.0,4160.0,5128.0,6160.0,7128.0,128.0,3073.12,23073.12,43073.12,74861.25,3422.0,79.484,4.848561200112841e+50,561512819216.0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2e-06,-1.903,,,,,,,,,,,,,,74512.91,34.0,-8565340321.0,,,,-0.202,0.0,-3.938,0.156,0.013583,1.761,0.024169,3.726,0.064554,9.444,0.182747,19.121,99.66,2.740491,65.073,5.2e-05,0.002409,0.000597,1.001273,0.000316,1.894568,0.007218,1.892159,0.007192,0.001272,0.000315,-0.041,0.008,0.00053,0.0,-0.00033,-46.043,2.91,0.021,97.966,0.037,kld/Step-3.7-Flash/wiki-test-raw/aes_sedai/Step-3.7-Flash-Q8_0.md,197.43,False,