3.2 KiB
Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers
Gradient-free persistent learning through hidden-state episodic recall.
Abstract
Teach a frozen language model new facts without gradient descent. Store the model's own hidden states as episodic memories. On recall, inject stored representations as logit biases. One-shot. Persistent. No weight modification.
Key Result
A frozen Qwen 2.5 0.5B taught three facts about a fictional entity:
| Prompt | Taught | Recalled | Similarity |
|---|---|---|---|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
No gradients computed at any point.
Quick Start
pip install transformers torch numpy
git clone https://git.rotko.net/tommi/epimem
cd epimem
python python/epimem.py
This downloads Qwen 2.5 0.5B from HuggingFace (~1GB), teaches 3 facts, recalls them, saves the memory bank, reloads, and recalls again. Takes ~2 minutes on first run (model download), ~30 seconds after.
With ONNX (faster, no PyTorch)
pip install onnxruntime transformers numpy
python export_onnx.py # exports Qwen 2.5 as ONNX to models/
python python/epimem.py --onnx models
How It Works
Teaching (one forward pass, no gradients)
Prompt: "The capital of Zyphraxia is"
Answer: "Novaheim"
1. backbone("The capital of Zyphraxia is") → hidden state h (896-dim vector)
2. backbone("The capital of Zyphraxia is Novaheim") → logit biases for "Novaheim"
3. Store: (key=h, value=logit_biases) in memory bank
Recall (similarity search + logit injection)
Query: "The capital of Zyphraxia is"
1. backbone(query) → h_q (896-dim vector)
2. cosine_sim(h_q, stored_key) = 1.000
3. Inject: logits += stored_logit_biases (per-position)
4. Generate: "Novaheim, a city of 100..."
Why hidden states, not text (like RAG)
RAG stores text, re-encodes it each time, consumes context window. We store the backbone's own internal representation — no re-encoding, no context consumption.
Files
python/epimem.py ← standalone reproduction (~200 lines)
export_onnx.py ← ONNX export from HuggingFace
paper.md ← full paper
results/memory_bank.json ← example: 896-dim hidden-state vectors
schema/isis.fbs ← FlatBuffer schema for memory bank
schema/organism.fbs ← FlatBuffer schema for organism state
models/tokenizer/ ← Qwen 2.5 tokenizer files
Citation
@article{niemi2026clivewearing,
title={Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers},
author={Tommi Niemi},
year={2026},
organization={Rotko Networks},
url={https://git.rotko.net/tommi/epimem},
}
License
MIT