cri/README.md

epimem: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory

Gradient-free persistent learning through hidden-state episodic recall.

---

Abstract

Teach a frozen language model new facts without gradient descent. Store the model's own hidden states as episodic memories. On recall, inject stored representations as logit biases. One-shot. Persistent. No weight modification.

Key Result

A frozen Qwen 2.5 0.5B taught three facts about a fictional entity:

Prompt	Taught	Recalled	Similarity
"The capital of Zyphraxia is"	"Novaheim"	"Novaheim, a city of 100"	1.000
"The ruler of Zyphraxia is"	"Queen Stellara"	"Queen Stellara. She is a beautiful woman"	1.000
"The currency of Zyphraxia is"	"Glimmers"	"Glimmers. The currency is divided into"	1.000

No gradients computed at any point.

Quick Start

pip install transformers torch numpy
git clone https://git.rotko.net/tommi/epimem
cd epimem
python python/epimem.py

This downloads Qwen 2.5 0.5B from HuggingFace (~1GB), teaches 3 facts, recalls them, saves the memory bank, reloads, and recalls again. Takes ~2 minutes on first run (model download), ~30 seconds after.

With ONNX (faster, no PyTorch)

pip install onnxruntime transformers numpy
python export_onnx.py          # exports Qwen 2.5 as ONNX to models/
python python/epimem.py --onnx models

How It Works

Teaching (one forward pass, no gradients)

Prompt: "The capital of Zyphraxia is"
Answer: "Novaheim"

1. backbone("The capital of Zyphraxia is") → hidden state h (896-dim vector)
2. backbone("The capital of Zyphraxia is Novaheim") → logit biases for "Novaheim"
3. Store: (key=h, value=logit_biases) in memory bank

Recall (similarity search + logit injection)

Query: "The capital of Zyphraxia is"

1. backbone(query) → h_q (896-dim vector)
2. cosine_sim(h_q, stored_key) = 1.000
3. Inject: logits += stored_logit_biases (per-position)
4. Generate: "Novaheim, a city of 100..."

Why hidden states, not text (like RAG)

RAG stores text, re-encodes it each time, consumes context window. We store the backbone's own internal representation — no re-encoding, no context consumption.

Files

python/epimem.py          ← standalone reproduction (~200 lines)
export_onnx.py            ← ONNX export from HuggingFace
paper.md                  ← full paper
results/memory_bank.json  ← example: 896-dim hidden-state vectors
schema/isis.fbs           ← FlatBuffer schema for memory bank
schema/organism.fbs       ← FlatBuffer schema for organism state
models/tokenizer/         ← Qwen 2.5 tokenizer files

Citation

@article{niemi2026epimem,
  title={One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory},
  author={Tommi Niemi},
  year={2026},
  organization={Rotko Networks},
  url={https://git.rotko.net/tommi/epimem},
}

License

MIT