Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory

Gradient-free persistent learning through hidden-state episodic recall and sleep consolidation.
Paper Code

---
Table of Contents
  1. Abstract
  2. Key Result
  3. Reproduction
  4. How It Works
  5. Citation
## Abstract We present a method for teaching frozen transformer language models new facts without gradient descent. The model's own hidden states are stored as episodic memories using Hebbian association. On recall, stored hidden states directly bias token generation through logit injection, bypassing re-encoding. A Continuous Thought Machine (CTM) gates memory retrieval through multi-tick deliberation. Sleep consolidation optimizes synapse weights via closed-form least-squares without backpropagation. We demonstrate one-shot learning of novel facts (100% recall accuracy) with zero weight modification to the backbone, surviving NREM/REM sleep cycles. Named for Clive Wearing, the musician who lost the ability to form new memories but retained all procedural skills. Like Wearing, a frozen transformer retains its capabilities but cannot learn. We give it a synthetic hippocampus. ## Key Result A frozen Qwen 2.5 0.5B backbone taught three facts about a fictional entity ("Zyphraxia") recalls all three correctly after a sleep consolidation cycle: | Prompt | Taught | Recalled | Similarity | |--------|--------|----------|:---:| | "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 | | "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 | | "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 | **No gradients computed at any point.** Teaching is one-shot (single forward pass). Recall survives sleep consolidation. The backbone generates fluent continuations beyond the taught answer. ## Reproduction ### Requirements - Rust (stable toolchain) - Qwen 2.5 0.5B ONNX backbone (see below) ### Steps ```bash # Clone the implementation git clone https://git.rotko.net/rotko/isis cd isis # Build cargo build --release # Download Qwen 2.5 0.5B ONNX model # Place backbone.onnx, lm_head.onnx, and tokenizer/ in models/ # Run the complete experiment (single command): target/release/isis e2e models # Expected output: # Teaching 3 facts → 3/3 taught # Recall test → 3/3 correct (sim=1.000) # Sleep cycle → 8 synapses consolidated # Post-sleep recall → 3/3 correct ``` Runs in under 30 seconds on CPU. No GPU required. ### What the e2e test does 1. Loads Qwen 2.5 backbone (frozen, 896-dim hidden states) 2. Creates a fresh CTM (64 neurons, 8 regions, 16 deliberation ticks) 3. Teaches 3 novel facts by extracting hidden states + computing logit biases 4. Tests recall via hidden-state similarity search + CTM gating + logit injection 5. Runs NREM sleep (least-squares synapse consolidation) + REM (emotional replay) 6. Tests recall again (verifies memories survive consolidation) ## How It Works ### Teaching (one forward pass) ``` Prompt: "The capital of Zyphraxia is" Answer: "Novaheim" 1. backbone("The capital of Zyphraxia is") → hidden state h (896-dim vector) 2. backbone("The capital of Zyphraxia is Novaheim") → logit biases for "Novaheim" 3. Store: (key=h, value=logit_biases) in memory bank ``` ### Recall (similarity search + injection) ``` Query: "The capital of Zyphraxia is" 1. backbone(query) → h_q (896-dim vector) 2. cosine_sim(h_q, stored_key) → 1.000 (same backbone, same prompt) 3. CTM deliberates 16 ticks → gate = 0.10 (conservative trust) 4. Inject: logits += gate * stored_logit_biases 5. Generate: "Novaheim, a city of 100..." ``` ### Why hidden states, not text (like RAG) RAG stores text, re-encodes it each time, consumes context window. We store the backbone's own internal representation — no re-encoding, no context consumption, instant injection. ### Sleep consolidation (no gradients) ``` During waking: collect (input, output) trace pairs per synapse During NREM: W_opt = argmin ||Y - XW||^2 (Cholesky, closed-form) W_new = 0.9 * W_old + 0.1 * W_opt Residual r < 0.07 for all synapses → near-optimal ``` This is matrix algebra, not optimization. One solve. No learning rate. No epochs. ## Implementation The full implementation is in [isis](https://git.rotko.net/rotko/isis) (Rust). Key files: - `src/host/io/memory.rs` — Episodic memory bank (store/recall/consolidation) - `src/organism/ctm.rs` — Continuous Thought Machine (deliberation, Hebbian plasticity, sleep) - `src/host/io/flatbuf.rs` — FlatBuffer serialization for memory banks - `schema/isis.fbs` — Memory bank schema (supports f32/f16/i8 key quantization) - `src/main.rs` — e2e test entry point ## Citation ```bibtex @article{niemi2026epimem, title={Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory}, author={Tommi Niemi}, year={2026}, organization={Rotko Networks}, url={https://git.rotko.net/rotko/epimem}, } ``` ## License MIT