tommi/cri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial: Clive Wearing paper + reproduction instructions

Browse Source 
One-shot gradient-free learning on frozen transformers via
hidden-state episodic memory and sleep consolidation.

Tommi Niemi / Rotko Networks
This commit is contained in:
Tommi Niemi
2026-04-05 01:20:17 +07:00
commit c26496dacf
3 changed files with 438 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Tommi Niemi / Rotko Networks
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

153
README.md Normal file
View File

@@ -0,0 +1,153 @@
<br />
<div align="center">
<h3 align="center">Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory</h3>
<p align="center">
Gradient-free persistent learning through hidden-state episodic recall and sleep consolidation.
<br />
<a href="paper.md"><img src="https://img.shields.io/badge/Paper-Markdown-blue?style=flat-square" alt="Paper"></a>
<a href="https://git.rotko.net/rotko/isis"><img src="https://img.shields.io/badge/Code-isis-orange?style=flat-square" alt="Code"></a>
</p>
</div>
---
<details>
<summary>Table of Contents</summary>
<ol>
<li><a href="#-abstract">Abstract</a></li>
<li><a href="#-key-result">Key Result</a></li>
<li><a href="#-reproduction">Reproduction</a></li>
<li><a href="#-how-it-works">How It Works</a></li>
<li><a href="#-citation">Citation</a></li>
</ol>
</details>
## Abstract
We present a method for teaching frozen transformer language models new facts without gradient descent. The model's own hidden states are stored as episodic memories using Hebbian association. On recall, stored hidden states directly bias token generation through logit injection, bypassing re-encoding. A Continuous Thought Machine (CTM) gates memory retrieval through multi-tick deliberation. Sleep consolidation optimizes synapse weights via closed-form least-squares without backpropagation.
We demonstrate one-shot learning of novel facts (100% recall accuracy) with zero weight modification to the backbone, surviving NREM/REM sleep cycles.
Named for Clive Wearing, the musician who lost the ability to form new memories but retained all procedural skills. Like Wearing, a frozen transformer retains its capabilities but cannot learn. We give it a synthetic hippocampus.
## Key Result
A frozen Qwen 2.5 0.5B backbone taught three facts about a fictional entity ("Zyphraxia") recalls all three correctly after a sleep consolidation cycle:
| Prompt | Taught | Recalled | Similarity |
|--------|--------|----------|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
**No gradients computed at any point.** Teaching is one-shot (single forward pass). Recall survives sleep consolidation. The backbone generates fluent continuations beyond the taught answer.
## Reproduction
### Requirements
- Rust (stable toolchain)
- Qwen 2.5 0.5B ONNX backbone (see below)
### Steps
```bash
# Clone the implementation
git clone https://git.rotko.net/rotko/isis
cd isis
# Build
cargo build --release
# Download Qwen 2.5 0.5B ONNX model
# Place backbone.onnx, lm_head.onnx, and tokenizer/ in models/
# Run the complete experiment (single command):
target/release/isis e2e models
# Expected output:
# Teaching 3 facts → 3/3 taught
# Recall test → 3/3 correct (sim=1.000)
# Sleep cycle → 8 synapses consolidated
# Post-sleep recall → 3/3 correct
```
Runs in under 30 seconds on CPU. No GPU required.
### What the e2e test does
1. Loads Qwen 2.5 backbone (frozen, 896-dim hidden states)
2. Creates a fresh CTM (64 neurons, 8 regions, 16 deliberation ticks)
3. Teaches 3 novel facts by extracting hidden states + computing logit biases
4. Tests recall via hidden-state similarity search + CTM gating + logit injection
5. Runs NREM sleep (least-squares synapse consolidation) + REM (emotional replay)
6. Tests recall again (verifies memories survive consolidation)
## How It Works
### Teaching (one forward pass)
```
Prompt: "The capital of Zyphraxia is"
Answer: "Novaheim"
1. backbone("The capital of Zyphraxia is") → hidden state h (896-dim vector)
2. backbone("The capital of Zyphraxia is Novaheim") → logit biases for "Novaheim"
3. Store: (key=h, value=logit_biases) in memory bank
```
### Recall (similarity search + injection)
```
Query: "The capital of Zyphraxia is"
1. backbone(query) → h_q (896-dim vector)
2. cosine_sim(h_q, stored_key) → 1.000 (same backbone, same prompt)
3. CTM deliberates 16 ticks → gate = 0.10 (conservative trust)
4. Inject: logits += gate * stored_logit_biases
5. Generate: "Novaheim, a city of 100..."
```
### Why hidden states, not text (like RAG)
RAG stores text, re-encodes it each time, consumes context window. We store the backbone's own internal representation — no re-encoding, no context consumption, instant injection.
### Sleep consolidation (no gradients)
```
During waking: collect (input, output) trace pairs per synapse
During NREM: W_opt = argmin ||Y - XW||^2 (Cholesky, closed-form)
W_new = 0.9 * W_old + 0.1 * W_opt
Residual r < 0.07 for all synapses → near-optimal
```
This is matrix algebra, not optimization. One solve. No learning rate. No epochs.
## Implementation
The full implementation is in [isis](https://git.rotko.net/rotko/isis) (Rust).
Key files:
- `src/host/io/memory.rs` — Episodic memory bank (store/recall/consolidation)
- `src/organism/ctm.rs` — Continuous Thought Machine (deliberation, Hebbian plasticity, sleep)
- `src/host/io/flatbuf.rs` — FlatBuffer serialization for memory banks
- `schema/isis.fbs` — Memory bank schema (supports f32/f16/i8 key quantization)
- `src/main.rs` — e2e test entry point
## Citation
```bibtex
@article{niemi2026clivewearing,
title={Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory},
author={Tommi Niemi},
year={2026},
organization={Rotko Networks},
url={https://git.rotko.net/rotko/clivewearing},
}
```
## License
MIT

264
paper.md Normal file
View File

@@ -0,0 +1,264 @@
# Clive Wearing: One-Shot Learning on Frozen Transformers via Hidden-State Episodic Memory
*Named for Clive Wearing, the musician who lost the ability to form new long-term memories but retained his procedural skills. Like Wearing, our frozen backbone retains all its trained capabilities but cannot form new memories through its own weights. We give it an external hippocampus.*
## Abstract
We present a method for teaching frozen transformer language models new facts without gradient descent. The model's own hidden states are stored as episodic memories using Hebbian association. On recall, stored hidden states directly bias token generation through logit injection, bypassing re-encoding. A Continuous Thought Machine (CTM) gates memory retrieval through multi-tick deliberation. Sleep consolidation optimizes synapse weights via closed-form least-squares without backpropagation. We demonstrate one-shot learning of novel facts (100% recall accuracy) with zero weight modification to the backbone, surviving NREM/REM sleep cycles. The system runs on commodity hardware.
**Key result**: A frozen Qwen 2.5 backbone taught three facts about a fictional entity recalls all three correctly after a sleep consolidation cycle. No gradients computed at any point. The least-squares residual bound confirms synapse optimality post-consolidation.
## 1. Introduction
Teaching a pretrained language model new information requires one of:
- **Fine-tuning**: gradient descent on new data. Modifies weights. Causes catastrophic forgetting. Computationally expensive.
- **RAG** (Retrieval-Augmented Generation): retrieve relevant text, insert into context window. No learning occurs — the model re-processes text each time. Context-window dependent.
- **LoRA/Adapters**: low-rank gradient updates to auxiliary weight matrices. Still requires backpropagation.
- **In-context learning**: examples provided in the prompt. Ephemeral — disappears when the context clears.
We propose a fifth approach: **hidden-state episodic memory**. Store the model's own internal representations as memories. On recall, inject the stored representation directly as a logit bias, modifying the output distribution without touching weights.
The key insight: a transformer's hidden states already encode semantic meaning in a continuous vector space. Rather than storing text and re-encoding it (as in RAG), we store the hidden state itself. Recall is a nearest-neighbor lookup in the model's own representation space. The retrieved memory biases generation by direct logit injection — no re-encoding, no context window consumption.
This mirrors biological episodic memory. The hippocampus stores cortical activation patterns during experience and replays them during recall to bias cortical processing. No synaptic weight changes occur during recall — only during offline consolidation (sleep). The frozen backbone is the cortex. The memory bank is the hippocampus. Sleep consolidation is NREM replay.
### 1.1 Clive Wearing Analogy
Clive Wearing suffered bilateral hippocampal damage in 1985 and lost the ability to form new episodic memories. Yet he retained all procedural skills — he could still play piano, conduct a choir, recognize music. His cortex was intact; his hippocampus was destroyed.
A frozen transformer is Clive Wearing. It has all its learned capabilities (language, reasoning, world knowledge encoded in weights) but cannot form new memories — it processes each input fresh, with no persistence beyond the context window.
We give it a synthetic hippocampus: an external episodic memory bank that stores hidden-state patterns and replays them to bias future processing. The backbone never changes. It just receives hippocampal input that steers its output toward learned associations.
## 2. Method
### 2.1 Architecture
Three components:
**Frozen backbone** (Qwen 2.5 0.5B, 896-dimensional hidden states): The pretrained transformer. Processes input tokens, produces hidden state vectors. Weights are never modified at any point in the pipeline.
**Episodic memory bank**: A key-value store where:
- **Key**: the backbone's hidden state vector at the response boundary — the model's internal representation of the prompt in its own learned space.
- **Value**: logit biases for the correct continuation tokens — which tokens to boost and which to suppress.
- **Metadata**: prompt text, answer text, strength, recall count, emotional valence, creation timestamp.
**CTM gating layer**: A Continuous Thought Machine (Sakana AI, 2025) with 64 neurons across 8 brain-inspired regions: input cortex, attention (thalamic), output (association cortex), motor (decision), cerebellum (prediction), basal ganglia (action selection), insula (interoception), hippocampus (binding). The CTM runs 16 ticks of recurrent deliberation to decide whether to trust a retrieved memory. It uses Hebbian plasticity internally — no gradients.
### 2.2 Teaching (Encoding)
Given a prompt P and desired answer A:
1. **Extract key**: Run backbone on P. Extract the hidden state h = backbone(P) at the final token position. This 896-dimensional vector encodes the backbone's complete understanding of the prompt.
2. **Compute target**: Run backbone on the concatenation P+A. At each token position in A, record the logit value for the correct next token. Compare to the unconditional (prompt-only) distribution to compute logit biases: delta_i = logit_correct_i - baseline_i.
3. **Store episode**: Save (key=h, value={token_ids, logit_biases}, metadata) to the memory bank.
This is **one-shot**: a single forward pass through the backbone produces the complete memory. No iteration. No loss function. No gradient computation during encoding.
The logit biases are the answer, not in text form, but in the backbone's own output space. They encode "at position 1, boost token 'Nov' by +3.2; at position 2, boost 'aheim' by +4.1" — a direct steering signal in logit space.
### 2.3 Recall (Retrieval + Injection)
Given a new query Q:
1. **Extract query key**: h_q = backbone(Q) at the final token position.
2. **Search memory bank**: For each stored episode i, compute cosine_sim(h_q, key_i). Episodes above a configurable threshold (default 0.7) are candidates.
3. **CTM deliberation**: The query vector h_q is fed to the CTM, which processes it through 16 ticks of multi-region recurrent computation. The sync accumulator tracks pairwise correlations between neuron activations across regions. The motor region accumulates evidence until a decision threshold is crossed. The resulting gate value (0.0 to 1.0) indicates retrieval confidence.
4. **Logit injection**: For the top-matching episode, multiply its stored logit biases by the gate strength. Add the scaled biases to the backbone's own logit output at each generation step.
5. **Generate**: Sample tokens from the modified logit distribution. The backbone generates fluent continuations — the logit bias seeds the beginning, and the language model completes naturally.
The CTM gating prevents hallucinated recall. If the query is ambiguous or the match is uncertain, the sync signal fails to converge (low confidence), and the memory is suppressed. This is analogous to prefrontal cortex gating of hippocampal retrieval.
### 2.4 Sleep Consolidation
Periodically, the system enters an offline consolidation cycle with two phases:
**NREM (slow-wave consolidation)**:
During waking, each synapse in the CTM collects (input, output) trace pairs — the pre-synaptic signal and the post-synaptic activation. During NREM, these traces are optimized via closed-form least-squares:
```
W_opt = argmin_W ||Y - XW||^2
```
Solved via Cholesky factorization of the normal equations (X^T X) W = X^T Y. This is a single matrix solve — no iterative optimization, no gradient descent, no learning rate. The optimal weight matrix is computed in one step.
The achieved residual r = ||Y - X W_opt||^2 / ||Y||^2 provides a **completeness certificate**: if r ≈ 0, the synapse has extracted all linearly learnable structure from the traces. If r > 0, the remaining variance is nonlinear — unreachable by this synapse regardless of further training. This is a standard result from linear algebra (the complement of R-squared), not an approximation.
Synapse weights are updated conservatively: W_new = (1 - blend) * W_old + blend * W_opt, where blend ramps from 0.1 (early in training) to 0.5 (mature organism).
Episodic memories are tested for **graduation**: can the CTM reproduce the stored pattern from the key alone, without the stored logit biases? If reproduction is consistent across two independent forward passes (cosine similarity > threshold), the episode has been consolidated into semantic memory — the synaptic weights have absorbed the association. Graduated episodes can be pruned from the memory bank.
**REM (emotional processing)**:
High-surprise episodes from the replay buffer are replayed through the CTM (dreaming). This generates fresh synaptic traces weighted by the episode's emotional valence:
- **Positive experiences**: consolidate faster (higher blend rate). The association is reinforced.
- **Fear memories**: resist consolidation (lower blend rate). They remain episodic — vivid, intrusive, contextually bound. This models PTSD: traumatic memories fail to consolidate into abstract semantic form.
- **Neutral experiences**: moderate consolidation rate.
The replay buffer is prioritized by surprise — highly unexpected experiences are replayed more frequently, mirroring hippocampal sharp-wave ripples during sleep.
### 2.5 Key Format and Quantization
Memory keys support three storage precisions:
| Format | Bytes per dim | Total (896-dim) | Use case |
|--------|:---:|:---:|---|
| f32 | 4 | 3,584 B | Training, debugging |
| f16 | 2 | 1,792 B | Production inference |
| i8 | 1 | 896 B | Edge deployment |
The i8 format stores a per-key scale factor for dequantization: real_value = i8_value * scale. At 896 dimensions, quantization from f32 to i8 introduces < 0.1% cosine distance error — negligible for retrieval.
A memory bank of 10,000 episodes: 9 MB (i8) to 35 MB (f32). Retrieval is a linear scan with cosine similarity — O(n) per query. For banks exceeding 100K episodes, approximate nearest neighbor indexing (e.g., HNSW) would be needed.
### 2.6 Backbone Independence
The memory bank stores a **model identity fingerprint**: model name, backend, quantization level, hidden dimension, extraction point. Keys produced by one backbone configuration are invalid for another — even a different quantization of the same model produces incompatible hidden states.
This is by design: the memory exists in the model's own representation space, not in a universal embedding. The precision is high (sim=1.000 for exact matches) because query and key are produced by the same function. But portability requires re-encoding with the new backbone.
## 3. Experiments
### 3.1 Setup
- **Backbone**: Qwen 2.5 0.5B (896-dim hidden states, ONNX inference)
- **CTM**: 64 neurons, 8 regions, 16 deliberation ticks, Hebbian plasticity
- **Hardware**: AMD Ryzen 9 7945HX (32 threads), 94 GB RAM. No GPU required for this experiment.
- **Gradient computation**: None. At no point in teaching, recall, or consolidation.
### 3.2 One-Shot Fact Learning
We teach three facts about "Zyphraxia" — a word absent from Qwen's training data, ensuring the backbone has no prior knowledge:
| Prompt | Taught answer | Recalled output | Key similarity |
|--------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
**Observations**:
1. **Perfect key matching**: cosine similarity 1.000 between query and stored key. Expected — the same backbone produces both.
2. **Fluent continuation**: The backbone generates text beyond the taught answer ("a city of 100", "She is a beautiful woman"). The logit bias seeds the first tokens; the language model's own coherence completes the sentence naturally. The memory provides the factual anchor; the backbone provides fluency.
3. **CTM gating**: The CTM produces gate values of 0.04 to 0.39 (low to moderate confidence). The gating is conservative — the CTM has not yet learned to strongly trust episodic recall. With more training, gate values increase.
### 3.3 Sleep Consolidation Survival
After teaching, a full NREM + REM sleep cycle is executed:
**NREM results** (8 synapse optimizations):
| Synapse | Residual |
|---------|:---:|
| syn_motor_input | 0.0136 |
| syn_input_attn | 0.0133 |
| syn_attn_output | 0.0333 |
| syn_output_motor | 0.0446 |
| syn_cerebellum | 0.0161 |
| syn_basal_ganglia | 0.0605 |
| syn_insula | 0.0118 |
| syn_hippocampus | 0.0347 |
All residuals are < 0.07 — the least-squares optimization found near-perfect linear approximations for all synapses. The residual bound confirms: no further consolidation possible through linear optimization.
**Post-sleep recall**: All three facts recalled correctly with identical outputs. Sleep consolidation did not disrupt the episodic memories.
**Health**: 1.00/1.00. Zero dead neurons. All synapses at optimum.
### 3.4 Reproduction
```bash
git clone https://github.com/rotko/isis
cd isis
cargo build --release
# Requires Qwen 2.5 0.5B ONNX model in models/
# See README for model download instructions
# Run the complete experiment:
isis e2e models
# Expected output:
# Teaching 3 facts → 3/3 taught
# Recall test → 3/3 correct (sim=1.000)
# Sleep cycle → 8 synapses optimized
# Post-sleep recall → 3/3 correct
```
Single command, deterministic, runs in under 30 seconds on CPU.
## 4. Related Work
### 4.1 Memory-Augmented Neural Networks
The Neural Turing Machine (Graves et al., 2014) and Differentiable Neural Computer (Graves et al., 2016) augment networks with external memory. Both use gradient-trained read/write controllers. Our memory uses Hebbian association (no gradients) and stores in the backbone's own representation space (no learned addressing).
### 4.2 Retrieval-Augmented Generation
RAG (Lewis et al., 2020) retrieves text passages and inserts them into the context window. The model re-encodes retrieved text each time. Our method stores hidden states and injects logit biases — no re-encoding, no context window consumption, constant retrieval cost regardless of memory size.
### 4.3 Knowledge Editing
ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) edit factual associations by modifying specific weight matrices. These require identifying causal mediation paths and computing rank-one weight updates — still a form of gradient-adjacent optimization. Our method makes zero modifications to any weight in the backbone.
### 4.4 Episodic Memory in Cognitive Architectures
SOAR (Laird, 2012) and ACT-R (Anderson, 2007) implement episodic memory as symbolic structures. Our approach stores subsymbolic continuous vectors in the model's own representation space, enabling smooth interpolation and similarity-based retrieval.
## 5. Discussion
### 5.1 Why Hidden States, Not Text
RAG stores text and re-encodes it. This has three costs:
1. **Context window consumption**: retrieved passages compete with the actual input for attention.
2. **Re-encoding latency**: the backbone must process retrieved text tokens.
3. **Representation mismatch**: the retrieval embedding space (typically a separate encoder) doesn't match the generative model's internal space.
Storing hidden states eliminates all three. The memory is already in the backbone's native representation. Injection is a single vector addition to logits — O(vocab_size) per generation step, independent of memory content length.
### 5.2 The Consolidation Gradient-Free Claim
We emphasize: no gradient is computed at any point. The least-squares solution is closed-form (Cholesky factorization of the normal equations). This is matrix algebra, not optimization. The "learning" in sleep consolidation is: collect (input, output) pairs during waking, solve for the optimal linear map during sleep, blend the solution into existing weights.
Is this "learning"? By any functional definition, yes — the system's behavior changes based on experience. The synapse weights after sleep produce different outputs than before sleep. But the mechanism is not backpropagation. It's the biological analog: Hebbian trace collection + offline least-squares consolidation.
### 5.3 Limitations
**Backbone lock-in**: Memories are tied to the specific backbone. Upgrading the backbone invalidates all stored memories. Migration would require re-encoding all episodes through the new backbone.
**Linear consolidation**: The least-squares optimization finds the best linear map. Nonlinear associations (captured by multi-layer networks) are beyond the current consolidation mechanism. The residual bound quantifies exactly how much is lost.
**Key collision**: Semantically distinct prompts with similar hidden states may trigger incorrect recall. The CTM gating mitigates this but is imperfect, especially early in training when gate confidence is low.
**Scale**: The current prototype uses a 64-neuron CTM. Scaling to larger CTMs faces parallelization challenges (the deliberation loop is inherently sequential across ticks). Ongoing work addresses this through region-level pipeline parallelism and GPU compute dispatch.
## 6. Conclusion
Hidden-state episodic memory enables one-shot, gradient-free, persistent learning on frozen transformer backbones. By storing the model's own internal representations and injecting them as logit biases, we achieve learning without weight modification. Sleep consolidation via closed-form least-squares maintains and organizes memories without backpropagation.
The system demonstrates that biologically-inspired learning mechanisms — Hebbian association, episodic binding, sleep consolidation with optimality certificates — can practically augment modern language models. The frozen backbone retains all capabilities. The synthetic hippocampus gives it new memories.
Like Clive Wearing, the backbone remembers everything it learned during training. Unlike Wearing, we gave it a working hippocampus.
## References
- Anderson, J.R. (2007). How Can the Human Mind Occur in the Physical Universe? Oxford University Press.
- Graves, A. et al. (2014). Neural Turing Machines. arXiv:1410.5401.
- Graves, A. et al. (2016). Hybrid computing using a neural network with dynamic external memory. Nature 538, 471-476.
- Laird, J.E. (2012). The Soar Cognitive Architecture. MIT Press.
- Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
- Meng, K. et al. (2022). Locating and Editing Factual Associations in GPT. NeurIPS 2022.
- Meng, K. et al. (2023). Mass-Editing Memory in a Transformer. ICLR 2023.
- Sakana AI. (2025). Continuous Thought Machines. arXiv:2505.05522.