You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Continuum-0.1B — The Hybrid Mind

Continuum-0.1B is a ~111M-parameter self-evolving Small Language Model built from scratch by 11-47 / WithInUsAI. It unifies 25 autonomous Hybrid-Mind subsystems into a single decoder-only transformer forward pass.

Architecture

Component Spec
Type Decoder-only transformer
Parameters ~111M (tied embeddings)
Context 64 096 tokens (64K + seed)
Layers 12
Hidden size 768
FFN SwiGLU, intermediate=2048
Attention GQA (12Q / 4KV heads)
Position NTK-aware RoPE θ=500 000
Norm Pre-norm RMSNorm
Dtype BFloat16

Training

  • Hardware: 2x T4 GPUs (Kaggle) via DataParallel
  • Steps: 3000 (effective batch 16 × 2048 tokens)
  • Resume: Auto-resumes from HF Hub checkpoint if kernel restarts

Datasets

Dataset Source
Claude Opus Mythos 5K WithinUsAI/claude_opus_mythos_5k
Claude Opus 4.8 Distill WithinUsAI/claude_opus_4.8_distill
Claude Mythos Distill WithinUsAI/claude_mythos_distill
Opus 4.7 Thinking Max Distill (25K) WithinUsAI/Opus4.7_thinking_max_distill_god_seed_25k
Claude Opus 4.7 Distilled WithinUsAI/claude_Opus_4.7_Distilled
Mythos Preview 5K v2 11-47/cluade_mythos_preview_5k_v2
Claude Opus 4.8 Max Thinking 5K v2 11-47/claude_opus_4.8_max_thinking_5k_v2

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("11-47/Continuum-0.1B", trust_remote_code=True)
model     = AutoModelForCausalLM.from_pretrained(
    "11-47/Continuum-0.1B",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

License

Apache 2.0

Downloads last month
-
Safetensors
Model size
0.1B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Collection including 11-47/Continuum-0.1B