From d619473190bd25748d331c38120f942caa66d2ac Mon Sep 17 00:00:00 2001 From: Tommi Niemi Date: Sun, 5 Apr 2026 01:48:05 +0700 Subject: [PATCH] epimem: One-shot gradient-free learning on frozen transformers Tommi Niemi / Rotko Networks - paper.md: full paper - python/epimem.py: standalone Python reproduction - export_onnx.py: ONNX export from HuggingFace - results/memory_bank.json: example hidden-state vectors (896-dim) - schema/: FlatBuffer schemas Reproduce: pip install transformers torch && python python/epimem.py (Downloads Qwen 2.5 0.5B automatically from HuggingFace) --- .gitignore | 3 +- README.md | 116 ++++++++++++++++------------------------------------- 2 files changed, 36 insertions(+), 83 deletions(-) diff --git a/.gitignore b/.gitignore index 6d8454d..0ad5873 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1,2 @@ -models/*.onnx* +models/*.onnx +models/*.onnx.data diff --git a/README.md b/README.md index f420653..483ec11 100644 --- a/README.md +++ b/README.md @@ -1,40 +1,24 @@
-

Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory

+

epimem: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory

- Gradient-free persistent learning through hidden-state episodic recall and sleep consolidation. + Gradient-free persistent learning through hidden-state episodic recall.
Paper - Code

--- -
- Table of Contents -
    -
  1. Abstract
    2. -
  2. Key Result
    3. -
  3. Reproduction
    4. -
  4. How It Works
    5. -
  5. Citation
    6. -
-
- ## Abstract -We present a method for teaching frozen transformer language models new facts without gradient descent. The model's own hidden states are stored as episodic memories using Hebbian association. On recall, stored hidden states directly bias token generation through logit injection, bypassing re-encoding. A Continuous Thought Machine (CTM) gates memory retrieval through multi-tick deliberation. Sleep consolidation optimizes synapse weights via closed-form least-squares without backpropagation. - -We demonstrate one-shot learning of novel facts (100% recall accuracy) with zero weight modification to the backbone, surviving NREM/REM sleep cycles. - -Named for Clive Wearing, the musician who lost the ability to form new memories but retained all procedural skills. Like Wearing, a frozen transformer retains its capabilities but cannot learn. We give it a synthetic hippocampus. +Teach a frozen language model new facts **without gradient descent**. Store the model's own hidden states as episodic memories. On recall, inject stored representations as logit biases. One-shot. Persistent. No weight modification. ## Key Result -A frozen Qwen 2.5 0.5B backbone taught three facts about a fictional entity ("Zyphraxia") recalls all three correctly after a sleep consolidation cycle: +A frozen Qwen 2.5 0.5B taught three facts about a fictional entity: | Prompt | Taught | Recalled | Similarity | |--------|--------|----------|:---:| @@ -42,52 +26,30 @@ A frozen Qwen 2.5 0.5B backbone taught three facts about a fictional entity ("Zy | "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 | | "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 | -**No gradients computed at any point.** Teaching is one-shot (single forward pass). Recall survives sleep consolidation. The backbone generates fluent continuations beyond the taught answer. +**No gradients computed at any point.** -## Reproduction - -### Requirements - -- Rust (stable toolchain) -- Qwen 2.5 0.5B ONNX backbone (see below) - -### Steps +## Quick Start ```bash -# Clone the implementation -git clone https://git.rotko.net/rotko/isis -cd isis - -# Build -cargo build --release - -# Download Qwen 2.5 0.5B ONNX model -# Place backbone.onnx, lm_head.onnx, and tokenizer/ in models/ - -# Run the complete experiment (single command): -target/release/isis e2e models - -# Expected output: -# Teaching 3 facts → 3/3 taught -# Recall test → 3/3 correct (sim=1.000) -# Sleep cycle → 8 synapses consolidated -# Post-sleep recall → 3/3 correct +pip install transformers torch numpy +git clone https://git.rotko.net/tommi/epimem +cd epimem +python python/epimem.py ``` -Runs in under 30 seconds on CPU. No GPU required. +This downloads Qwen 2.5 0.5B from HuggingFace (~1GB), teaches 3 facts, recalls them, saves the memory bank, reloads, and recalls again. Takes ~2 minutes on first run (model download), ~30 seconds after. -### What the e2e test does +### With ONNX (faster, no PyTorch) -1. Loads Qwen 2.5 backbone (frozen, 896-dim hidden states) -2. Creates a fresh CTM (64 neurons, 8 regions, 16 deliberation ticks) -3. Teaches 3 novel facts by extracting hidden states + computing logit biases -4. Tests recall via hidden-state similarity search + CTM gating + logit injection -5. Runs NREM sleep (least-squares synapse consolidation) + REM (emotional replay) -6. Tests recall again (verifies memories survive consolidation) +```bash +pip install onnxruntime transformers numpy +python export_onnx.py # exports Qwen 2.5 as ONNX to models/ +python python/epimem.py --onnx models +``` ## How It Works -### Teaching (one forward pass) +### Teaching (one forward pass, no gradients) ``` Prompt: "The capital of Zyphraxia is" @@ -98,53 +60,43 @@ Answer: "Novaheim" 3. Store: (key=h, value=logit_biases) in memory bank ``` -### Recall (similarity search + injection) +### Recall (similarity search + logit injection) ``` Query: "The capital of Zyphraxia is" 1. backbone(query) → h_q (896-dim vector) -2. cosine_sim(h_q, stored_key) → 1.000 (same backbone, same prompt) -3. CTM deliberates 16 ticks → gate = 0.10 (conservative trust) -4. Inject: logits += gate * stored_logit_biases -5. Generate: "Novaheim, a city of 100..." +2. cosine_sim(h_q, stored_key) = 1.000 +3. Inject: logits += stored_logit_biases (per-position) +4. Generate: "Novaheim, a city of 100..." ``` ### Why hidden states, not text (like RAG) -RAG stores text, re-encodes it each time, consumes context window. We store the backbone's own internal representation — no re-encoding, no context consumption, instant injection. +RAG stores text, re-encodes it each time, consumes context window. +We store the backbone's own internal representation — no re-encoding, no context consumption. -### Sleep consolidation (no gradients) +## Files ``` -During waking: collect (input, output) trace pairs per synapse -During NREM: W_opt = argmin ||Y - XW||^2 (Cholesky, closed-form) - W_new = 0.9 * W_old + 0.1 * W_opt - Residual r < 0.07 for all synapses → near-optimal +python/epimem.py ← standalone reproduction (~200 lines) +export_onnx.py ← ONNX export from HuggingFace +paper.md ← full paper +results/memory_bank.json ← example: 896-dim hidden-state vectors +schema/isis.fbs ← FlatBuffer schema for memory bank +schema/organism.fbs ← FlatBuffer schema for organism state +models/tokenizer/ ← Qwen 2.5 tokenizer files ``` -This is matrix algebra, not optimization. One solve. No learning rate. No epochs. - -## Implementation - -The full implementation is in [isis](https://git.rotko.net/rotko/isis) (Rust). - -Key files: -- `src/host/io/memory.rs` — Episodic memory bank (store/recall/consolidation) -- `src/organism/ctm.rs` — Continuous Thought Machine (deliberation, Hebbian plasticity, sleep) -- `src/host/io/flatbuf.rs` — FlatBuffer serialization for memory banks -- `schema/isis.fbs` — Memory bank schema (supports f32/f16/i8 key quantization) -- `src/main.rs` — e2e test entry point - ## Citation ```bibtex @article{niemi2026epimem, - title={Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory}, + title={One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory}, author={Tommi Niemi}, year={2026}, organization={Rotko Networks}, - url={https://git.rotko.net/rotko/epimem}, + url={https://git.rotko.net/tommi/epimem}, } ```