tommi/cri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reframe as Conditioned Reflex Injection (CRI) with multi-model test results

Browse Source 
- Rename from "Episodic Memory" to "Conditioned Reflex Injection" throughout
- Make code model-agnostic: --model flag for any HuggingFace backbone
- Support multimodal models (Gemma 4) via .model.language_model resolution
- Add negative stimulus-specificity tests and abstract query tests
- Paper now backed by measured data from 4 backbones:
  Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, E4B base
- Quantization tolerance tested at f32/f16/bf16/int8/int4
- Key findings: smaller base models outperform larger instruct models,
  instruct tuning compresses activation space (hurts discrimination),
  int4 viable if same-precision conditioning/triggering
- Add privacy-by-representation section
- Add Pavlov/Skinner references for conditioning framing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tommi Niemi
2026-04-06 19:03:32 +07:00
parent 176164815b
commit 3c77498dd4
4 changed files with 618 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files

86
README.md
View File

@@ -1,10 +1,10 @@
<br /> <br />
<div align="center"> <div align="center">
<h3 align="center">Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers</h3> <h3 align="center">Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers</h3>
<p align="center"> <p align="center">
Gradient-free persistent learning through hidden-state episodic recall. Gradient-free behavioral conditioning through hidden-state trigger matching and logit bias injection.
<br /> <br />
<a href="paper.md"><img src="https://img.shields.io/badge/Paper-Markdown-blue?style=flat-square" alt="Paper"></a> <a href="paper.md"><img src="https://img.shields.io/badge/Paper-Markdown-blue?style=flat-square" alt="Paper"></a>
</p> </p>
@@ -12,21 +12,25 @@
--- ---
## Abstract ## What This Is (And Isn't)
Teach a frozen language model new facts **without gradient descent**. Store the model's own hidden states as episodic memories. On recall, inject stored representations as logit biases. One-shot. Persistent. No weight modification. This is **not** episodic memory. The model doesn't remember anything. It doesn't experience the taught fact. It doesn't form a representation of "knowing" something.
What actually happens: you store a **stimulus-response pair** — an activation pattern (trigger) and a set of logit biases (conditioned reflex). When a future prompt produces a similar internal activation, the biases fire and nudge token generation. The model has no idea why it's saying "Novaheim." It just gets pushed there.
This is closer to **post-hypnotic suggestion** than memory. Pavlovian conditioning at the logit level. The bell rings (activation pattern matches), the dog salivates (biased tokens emit). No understanding. No experience. No episodic recall in any phenomenological sense.
## Key Result ## Key Result
A frozen Qwen 2.5 0.5B taught three facts about a fictional entity: A frozen Qwen 2.5 0.5B conditioned with three stimulus-response pairs:
| Prompt | Taught | Recalled | Similarity | | Trigger prompt | Conditioned response | Output when triggered | Similarity |
|--------|--------|----------|:---:| |--------|--------|----------|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 | | "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 | | "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 | | "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
**No gradients computed at any point.** **No gradients computed. No weights modified. The model doesn't know these facts — it reflexively produces them.**
## Quick Start ## Quick Start
@@ -34,65 +38,89 @@ A frozen Qwen 2.5 0.5B taught three facts about a fictional entity:
pip install transformers torch numpy pip install transformers torch numpy
git clone https://git.rotko.net/tommi/epimem git clone https://git.rotko.net/tommi/epimem
cd epimem cd epimem
python python/epimem.py python python/epimem.py # default: Qwen 2.5 0.5B
python python/epimem.py --model google/gemma-4-E4B-it # Gemma 4
python python/epimem.py --model google/gemma-4-E2B-it # Gemma 4 small
``` ```
This downloads Qwen 2.5 0.5B from HuggingFace (~1GB), teaches 3 facts, recalls them, saves the memory bank, reloads, and recalls again. Takes ~2 minutes on first run (model download), ~30 seconds after. Works with any HuggingFace causal LM or multimodal model with a text decoder.
### With ONNX (faster, no PyTorch) ### With ONNX (faster, no PyTorch)
```bash ```bash
pip install onnxruntime transformers numpy pip install onnxruntime transformers numpy
python export_onnx.py # exports Qwen 2.5 as ONNX to models/ python export_onnx.py # default model
python export_onnx.py --model google/gemma-4-E4B-it # Gemma 4
python python/epimem.py --onnx models python python/epimem.py --onnx models
``` ```
## How It Works ## How It Works
### Teaching (one forward pass, no gradients) ### Conditioning (one forward pass, no gradients)
``` ```
Prompt: "The capital of Zyphraxia is" Prompt: "The capital of Zyphraxia is"
Answer: "Novaheim" Answer: "Novaheim"
1. backbone("The capital of Zyphraxia is") → hidden state h (896-dim vector) 1. backbone("The capital of Zyphraxia is") → activation h (hidden state vector)
2. backbone("The capital of Zyphraxia is Novaheim") → logit biases for "Novaheim" 2. backbone("The capital of Zyphraxia is Novaheim") → logit gap for "Novaheim"
3. Store: (key=h, value=logit_biases) in memory bank 3. Store: (trigger=h, reflex=logit_biases) in reflex bank
``` ```
### Recall (similarity search + logit injection) ### Trigger firing (similarity search + logit injection)
``` ```
Query: "The capital of Zyphraxia is" Query: "The capital of Zyphraxia is"
1. backbone(query) → h_q (896-dim vector) 1. backbone(query) → activation h_q
2. cosine_sim(h_q, stored_key) = 1.000 2. cosine_sim(h_q, stored_trigger) = 1.000 → match
3. Inject: logits += stored_logit_biases (per-position) 3. Inject: logits += conditioned_biases (per-position)
4. Generate: "Novaheim, a city of 100..." 4. Output: "Novaheim, a city of 100..."
``` ```
### Why hidden states, not text (like RAG) ### Why not RAG?
RAG stores text, re-encodes it each time, consumes context window. RAG stores text, re-encodes it, consumes context window. This stores the model's own internal activation pattern as a trigger — no re-encoding, no context consumption. But RAG gives the model actual information to reason about. This just pushes output tokens. Different tool for different jobs.
We store the backbone's own internal representation — no re-encoding, no context consumption.
### Why not "episodic memory"?
Episodic memory implies the system re-experiences the encoding event. It doesn't. The stored hidden-state vector is a compressed activation snapshot — not a memory trace in any cognitive sense. The model never "encoded an experience." It produced an activation, we saved it, and we replay it as a logit bias. That's a conditioned reflex, not a memory.
## Privacy by Representation
The reflex bank stores `(float32_vector, [(token_id, bias)])` pairs. Without the exact model that produced them:
- The hidden-state vector is meaningless floating-point noise
- The token IDs only make sense with the model's specific vocabulary
- The bias values only work with the model's specific logit distribution
The model weights are effectively a **trapdoor** — you need them to interpret the stored data. This isn't encryption. It's opacity by representation. Steal the database, get noise.
## Files ## Files
``` ```
python/epimem.py ← standalone reproduction (~200 lines) python/epimem.py ← model-agnostic reproduction (~300 lines)
export_onnx.py ← ONNX export from HuggingFace export_onnx.py ← ONNX export for any HuggingFace model
paper.md ← full paper paper.md ← full paper
results/memory_bank.json ← example: 896-dim hidden-state vectors results/memory_bank.json ← example: hidden-state vectors + logit biases
schema/isis.fbs ← FlatBuffer schema for memory bank schema/isis.fbs ← FlatBuffer schema for reflex bank
schema/organism.fbs ← FlatBuffer schema for organism state schema/organism.fbs ← FlatBuffer schema for organism state
models/tokenizer/ ← Qwen 2.5 tokenizer files
``` ```
## Tested Models
| Model | Hidden dim | Layers | Notes |
|-------|-----------|--------|-------|
| Qwen 2.5 0.5B | 896 | 24 | Original test model |
| Gemma 4 E4B-it | 2560 | 42 | Recommended |
| Gemma 4 E2B-it | 1536 | 35 | PLE architecture, smallest |
Reflexes are **model-locked** — conditioning on one backbone doesn't transfer to another. Different model = different activation space = different triggers.
## Citation ## Citation
```bibtex ```bibtex
@article{niemi2026clivewearing, @article{niemi2026cri,
title={Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers}, title={Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers},
author={Tommi Niemi}, author={Tommi Niemi},
year={2026}, year={2026},
organization={Rotko Networks}, organization={Rotko Networks},

213
export_onnx.py
View File

@@ -1,95 +1,152 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Export Qwen 2.5-0.5B as ONNX for the brain crate. """Export any HuggingFace causal LM as ONNX for epimem.
Exports two models: Exports two models:
1. backbone.onnx: token_ids → hidden_state at layer 23 (for key computation) 1. backbone.onnx: token_ids → hidden_state at target layer (for trigger key computation)
2. lm_head.onnx: hidden_state → logits (for generation) 2. lm_head.onnx: full_hidden → logits (for generation)
Both together = full inference pipeline. Model-agnostic: works with Qwen, Gemma, Llama, Mistral, Phi, etc.
Separately = teach only needs backbone, not lm_head. For multimodal models (Gemma 4), exports only the text decoder.
Usage:
python export_onnx.py # default: Qwen 2.5 0.5B
python export_onnx.py --model google/gemma-4-E4B-it # Gemma 4
python export_onnx.py --model google/gemma-4-E2B-it # Gemma 4 small
python export_onnx.py --model meta-llama/Llama-3.2-1B # Llama
""" """
import argparse
import torch import torch
import torch.nn as nn import torch.nn as nn
import os import os
DEFAULT_MODEL = "Qwen/Qwen2.5-0.5B"
def export():
from transformers import AutoModelForCausalLM, AutoTokenizer
print("Loading Qwen 2.5-0.5B...") def resolve_text_model(model):
model = AutoModelForCausalLM.from_pretrained( """Extract the text decoder from multimodal models, or return as-is.
'Qwen/Qwen2.5-0.5B', torch_dtype=torch.float32, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(
'Qwen/Qwen2.5-0.5B', trust_remote_code=True)
model.eval()
D = model.config.hidden_size # 896 Gemma 4: ConditionalGeneration → .model (Gemma4Model) → .language_model
n_layers = model.config.num_hidden_layers # 24 LLaVA: ConditionalGeneration → .language_model
V = model.config.vocab_size Pure CausalLM: model itself
target_layer = n_layers - 1 # 23 """
inner = getattr(model, 'model', model)
if hasattr(inner, 'language_model'):
return inner.language_model
if hasattr(model, 'language_model'):
return model.language_model
return model
print(f" D={D}, layers={n_layers}, vocab={V}")
os.makedirs("models", exist_ok=True) def get_lm_head_and_norm(model):
"""Find lm_head and final layer norm regardless of architecture.
# ─── Export backbone: tokens → hidden at layer 23 ───────── Returns (norm, lm_head) where norm may be None if not found separately.
class BackboneToLayer23(nn.Module): """
def __init__(self, model, target_layer): text_model = resolve_text_model(model)
# Find lm_head
lm_head = None
if hasattr(text_model, 'lm_head'):
lm_head = text_model.lm_head
elif hasattr(model, 'lm_head'):
lm_head = model.lm_head
# Find final norm — varies by architecture
norm = None
inner = getattr(text_model, 'model', text_model)
for attr in ['norm', 'final_layernorm', 'ln_f', 'final_norm']:
if hasattr(inner, attr):
norm = getattr(inner, attr)
break
return norm, lm_head
class GenericBackboneWrapper(nn.Module):
"""Architecture-agnostic wrapper that extracts hidden state at target layer.
Uses output_hidden_states=True to get intermediate representations
without manually walking architecture-specific layer structures.
Returns:
hidden: hidden state at target_layer (for trigger key)
full_hidden: final layer hidden state (for lm_head / logit computation)
"""
def __init__(self, text_model, target_layer):
super().__init__() super().__init__()
self.embed = model.model.embed_tokens self.text_model = text_model
self.layers = model.model.layers[:target_layer + 1]
self.target_layer = target_layer self.target_layer = target_layer
self.rotary = model.model.rotary_emb
self.post_attn_norm = model.model.layers[target_layer].post_attention_layernorm
def forward(self, input_ids): def forward(self, input_ids):
x = self.embed(input_ids) outputs = self.text_model(input_ids, output_hidden_states=True)
B, T = input_ids.shape hidden_states = outputs.hidden_states # tuple of (n_layers+1) tensors
pos = torch.arange(T, device=input_ids.device).unsqueeze(0) target_hidden = hidden_states[self.target_layer] # [B, T, D]
pe = self.rotary(x, pos) final_hidden = hidden_states[-1] # [B, T, D]
return target_hidden, final_hidden
for i, layer in enumerate(self.layers):
if i == self.target_layer:
residual = x
x_normed = layer.input_layernorm(x)
attn_out = layer.self_attn(
x_normed, attention_mask=None,
position_embeddings=pe)[0]
x = residual + attn_out
# Return pre-MLP hidden (what CTM/brain receives)
hidden = self.post_attn_norm(x)
# Also complete the layer for full hidden
full = x + layer.mlp(hidden)
return hidden, full
else:
x = layer(x, position_embeddings=pe)
return x, x
# ─── Export lm_head: full_hidden → logits ───────────────── class GenericLMHead(nn.Module):
class LMHead(nn.Module): """Architecture-agnostic lm_head wrapper: hidden → logits."""
def __init__(self, model):
def __init__(self, norm, lm_head):
super().__init__() super().__init__()
# Remaining layers after target + final norm + lm_head self.norm = norm
self.final_norm = model.model.norm self.lm_head = lm_head
self.lm_head = model.lm_head
def forward(self, full_hidden): def forward(self, full_hidden):
x = self.final_norm(full_hidden) x = full_hidden
if self.norm is not None:
x = self.norm(x)
return self.lm_head(x) return self.lm_head(x)
# Export backbone
def export(model_name: str, output_dir: str):
from transformers import AutoModelForCausalLM, AutoTokenizer
print(f"Loading {model_name}...")
# Try CausalLM first, fall back to auto for multimodal
is_multimodal = False
try:
model = AutoModelForCausalLM.from_pretrained(
model_name, torch_dtype=torch.float32, trust_remote_code=True)
except (ValueError, KeyError):
from transformers import AutoModel
model = AutoModel.from_pretrained(
model_name, torch_dtype=torch.float32, trust_remote_code=True)
is_multimodal = True
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model.eval()
text_model = resolve_text_model(model)
config = text_model.config
D = config.hidden_size
n_layers = config.num_hidden_layers
V = config.vocab_size
target_layer = n_layers - 1
print(f" arch={type(model).__name__}")
print(f" D={D}, layers={n_layers}, vocab={V}, target_layer={target_layer}")
os.makedirs(output_dir, exist_ok=True)
# ─── Export backbone: tokens → hidden at target layer ─────────
print("Exporting backbone (tokens → hidden)...") print("Exporting backbone (tokens → hidden)...")
backbone = BackboneToLayer23(model, target_layer) backbone = GenericBackboneWrapper(text_model, target_layer)
dummy_ids = torch.randint(0, V, (1, 32)) dummy_ids = torch.randint(0, min(V, 10000), (1, 32))
with torch.no_grad(): with torch.no_grad():
hidden, full = backbone(dummy_ids) hidden, full = backbone(dummy_ids)
print(f" hidden: {hidden.shape}, full: {full.shape}") print(f" hidden: {hidden.shape}, full: {full.shape}")
backbone_path = os.path.join(output_dir, "backbone.onnx")
torch.onnx.export( torch.onnx.export(
backbone, dummy_ids, backbone, dummy_ids,
"models/backbone.onnx", backbone_path,
input_names=["input_ids"], input_names=["input_ids"],
output_names=["hidden", "full_hidden"], output_names=["hidden", "full_hidden"],
dynamic_axes={ dynamic_axes={
@@ -99,17 +156,23 @@ def export():
}, },
opset_version=17, opset_version=17,
) )
backbone_size = os.path.getsize("models/backbone.onnx") backbone_size = os.path.getsize(backbone_path)
print(f" Saved models/backbone.onnx ({backbone_size / 1e6:.1f} MB)") print(f" Saved {backbone_path} ({backbone_size / 1e6:.1f} MB)")
# Export lm_head # ─── Export lm_head: full_hidden → logits ─────────────────
print("Exporting lm_head (hidden → logits)...") print("Exporting lm_head (hidden → logits)...")
head = LMHead(model) norm, lm_head = get_lm_head_and_norm(model)
if lm_head is None:
print(" WARNING: Could not find lm_head, skipping lm_head export")
return
head = GenericLMHead(norm, lm_head)
dummy_hidden = torch.randn(1, 32, D) dummy_hidden = torch.randn(1, 32, D)
head_path = os.path.join(output_dir, "lm_head.onnx")
torch.onnx.export( torch.onnx.export(
head, dummy_hidden, head, dummy_hidden,
"models/lm_head.onnx", head_path,
input_names=["full_hidden"], input_names=["full_hidden"],
output_names=["logits"], output_names=["logits"],
dynamic_axes={ dynamic_axes={
@@ -118,18 +181,19 @@ def export():
}, },
opset_version=17, opset_version=17,
) )
head_size = os.path.getsize("models/lm_head.onnx") head_size = os.path.getsize(head_path)
print(f" Saved models/lm_head.onnx ({head_size / 1e6:.1f} MB)") print(f" Saved {head_path} ({head_size / 1e6:.1f} MB)")
# Save tokenizer # Save tokenizer
tokenizer.save_pretrained("models/tokenizer") tokenizer_path = os.path.join(output_dir, "tokenizer")
print(f" Saved models/tokenizer/") tokenizer.save_pretrained(tokenizer_path)
print(f" Saved {tokenizer_path}/")
# Verify # Verify
print("\nVerifying ONNX export...") print("\nVerifying ONNX export...")
import onnxruntime as ort import onnxruntime as ort
sess_backbone = ort.InferenceSession("models/backbone.onnx") sess_backbone = ort.InferenceSession(backbone_path)
sess_head = ort.InferenceSession("models/lm_head.onnx") sess_head = ort.InferenceSession(head_path)
test_text = "The capital of France is" test_text = "The capital of France is"
ids = tokenizer.encode(test_text, add_special_tokens=False) ids = tokenizer.encode(test_text, add_special_tokens=False)
@@ -144,9 +208,16 @@ def export():
print(f" Hidden shape: {hidden_out.shape}") print(f" Hidden shape: {hidden_out.shape}")
print(f" Logits shape: {logits_out[0].shape}") print(f" Logits shape: {logits_out[0].shape}")
print("\nDone. Total model size:", print(f"\nDone. Total model size: {(backbone_size + head_size) / 1e6:.1f} MB")
f"{(backbone_size + head_size) / 1e6:.1f} MB") print(f"Model: {model_name}")
if __name__ == "__main__": if __name__ == "__main__":
export() parser = argparse.ArgumentParser(
description="Export a HuggingFace model as ONNX for epimem")
parser.add_argument("--model", type=str, default=DEFAULT_MODEL,
help=f"HuggingFace model name (default: {DEFAULT_MODEL})")
parser.add_argument("--output", type=str, default="models",
help="Output directory (default: models)")
args = parser.parse_args()
export(args.model, args.output)

294
paper.md
View File

@@ -1,18 +1,26 @@
# Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers # Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers
**DRAFT — April 2026**
## Abstract ## Abstract
We enable frozen transformers to form new memories without gradient descent. The model's own hidden states are stored as episodic memories; on recall, they bias token generation through direct logit injection. A frozen Qwen 2.5 0.5B taught three novel facts recalls all three at 100% accuracy. No weights are modified. No gradients are computed. Memories persist to disk across sessions. Code and reproduction: [git.rotko.net/tommi/epimem](https://git.rotko.net/tommi/epimem). We enable frozen transformers to acquire new stimulus-response behaviors without gradient descent. When a target association is presented, the model's own hidden-state activation pattern is captured as a trigger, and per-token logit biases are stored as the conditioned response. At inference time, a matching activation pattern fires the reflex, injecting the learned token biases directly into the output logits. The model does not "know" the new fact — it is nudged toward specific token sequences when the right internal pattern activates. We test across four backbones (Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, and E4B base) and five precision levels (float32 through int4). Conditioned tokens are produced correctly in all cases, but post-bias coherence and trigger discrimination vary significantly across architectures: smaller base models outperform larger instruct-tuned models on both metrics. No weights are modified. No gradients are computed. Reflexes persist to disk across sessions. Code and reproduction: [git.rotko.net/tommi/epimem](https://git.rotko.net/tommi/epimem).
## 1. The Clive Wearing Problem ## 1. The Clive Wearing Problem — and What It Really Is
Clive Wearing lost his hippocampus to encephalitis in 1985. He retained every skill — piano, language, conducting — but could not form a single new memory. Every 7 seconds, he believed he had just woken up for the first time. His diary: "8:31 AM Now I am awake. 8:34 AM Now I am properly awake." Each entry crossed out moments later. Clive Wearing lost his hippocampus to encephalitis in 1985. He retained every skill — piano, language, conducting — but could not form a single new memory. Every 7 seconds, he believed he had just woken up for the first time. His diary: "8:31 AM Now I am awake. 8:34 AM Now I am properly awake." Each entry crossed out moments later.
Current LLMs are Clive Wearing. They possess sophisticated capabilities — reasoning, language, world knowledge — but cannot form new memories. Every conversation starts from zero. The context window is their 7-second span. When it clears, everything is gone. Current LLMs are Clive Wearing. They possess sophisticated capabilities — reasoning, language, world knowledge — but cannot acquire new behaviors after training. Every conversation starts from zero. The context window is their 7-second span. When it clears, everything is gone.
Fine-tuning modifies weights and causes catastrophic forgetting. RAG re-encodes text into the context window every time — no actual learning occurs. LoRA still requires gradients. In-context learning vanishes when the conversation ends. But the analogy reveals something deeper. What Clive lost was not just declarative memory — he also lost the ability to form new conditioned responses. He could not learn that a particular face meant danger, or that a certain phrase preceded a reward. His existing procedural conditioning (piano technique, musical phrasing) survived intact, but no new stimulus-response associations could be formed.
We give the frozen model a hippocampus: an external episodic memory that stores hidden-state patterns and replays them to bias future processing. The backbone never changes. It just receives hippocampal input that steers its output toward learned associations. This reframes the problem. What frozen LLMs lack is not a knowledge store — it is a conditioning mechanism. They need something closer to Pavlovian conditioning than to a filing cabinet. The bell (an activation pattern) should trigger salivation (a specific token sequence), and this association should persist without modifying the dog's brain (the model weights).
The analogy extends to post-hypnotic suggestion: a subject is given a trigger phrase during hypnosis, and upon hearing it later, performs a specific action without conscious awareness of why. Our system operates identically — the model has no "awareness" that it has been conditioned. It simply produces different outputs when a specific internal activation pattern fires.
Fine-tuning modifies weights and causes catastrophic forgetting. RAG re-encodes text into the context window every time — no actual conditioning occurs. LoRA still requires gradients. In-context learning vanishes when the conversation ends.
We give the frozen model a conditioning mechanism: an external reflex bank that stores activation triggers and replays conditioned responses to bias future token generation. The backbone never changes. It just receives stimulus-matched injections that steer its output toward conditioned associations.
## 2. Method ## 2. Method
@@ -20,45 +28,45 @@ We give the frozen model a hippocampus: an external episodic memory that stores
Two components: Two components:
**Frozen backbone** (Qwen 2.5 0.5B, 896-dimensional hidden states): The pretrained transformer. Processes input tokens, produces hidden state vectors. Weights are never modified at any point. **Frozen backbone** (any supported transformer — Qwen 2.5 0.5B with 896-dim hidden states, Gemma 4 E4B with 2560-dim, Gemma 4 E2B with 1536-dim): The pretrained transformer. Processes input tokens, produces hidden-state activation vectors. Weights are never modified at any point.
**Episodic memory bank**: A key-value store where: **Reflex bank**: A stimulus-response store where:
- **Key**: the backbone's hidden state vector at the final token position — the model's internal representation of the prompt in its own learned space. - **Stimulus (trigger pattern)**: the backbone's hidden-state vector at the final token position — the model's internal activation pattern for a given input, in its own learned representational space.
- **Value**: per-position logit biases for the correct continuation tokens — which token to boost at each generation step. - **Response (conditioned reflex)**: per-position logit biases for the correct continuation tokens — which token to boost at each generation step.
### 2.2 Teaching (one forward pass) The representation is sub-symbolic. The trigger pattern is an opaque high-dimensional vector — not a human-readable query string. The conditioned response is a sequence of (token_id, boost_magnitude) pairs — not a readable answer. The reflex bank cannot be casually inspected to determine what associations have been conditioned. One must run the backbone on an input and check whether a reflex fires.
Given a prompt P and desired answer A: ### 2.2 Conditioning (one forward pass)
1. **Extract key**: Run backbone on P. Extract hidden state h = backbone(P) at the final token. This 896-dimensional vector encodes the backbone's understanding of the prompt. Given a stimulus prompt P and desired response A:
2. **Compute logit biases**: Run backbone on the concatenation P+A. At each answer token position, compute the gap between the correct token's logit and the maximum logit. The bias is set to overcome this gap plus a margin: 1. **Extract trigger pattern**: Run backbone on P. Extract hidden state h = backbone(P) at the final token. This activation vector encodes the backbone's internal response to the stimulus.
2. **Compute conditioned response**: Run backbone on the concatenation P+A. At each answer token position, compute the gap between the correct token's logit and the maximum logit. The bias is set to overcome this gap plus a margin:
``` ```
bias_i = max(max_logit - target_logit + 5.0, 5.0) bias_i = max(max_logit - target_logit + 5.0, 5.0)
``` ```
This produces one (token_id, boost) pair per answer token. This produces one (token_id, boost) pair per answer token — the conditioned reflex.
3. **Store**: Save (key=h, value=[(token_id, bias) per position]) to the memory bank. 3. **Store**: Save (trigger=h, reflex=[(token_id, bias) per position]) to the reflex bank.
One forward pass. No iteration. No loss function. No gradients. One forward pass. No iteration. No loss function. No gradients. Teaching is conditioning: a single exposure creates a permanent stimulus-response association.
### 2.3 Recall (similarity search + injection) ### 2.3 Triggering (similarity search + injection)
Given a new query Q: Given a new query Q:
1. **Extract query key**: h_q = backbone(Q) at the final token. 1. **Extract query activation**: h_q = backbone(Q) at the final token.
2. **Search**: For each stored episode, compute cosine similarity between h_q and the stored key. Return the best match above threshold. 2. **Match**: For each stored reflex, compute cosine similarity between h_q and the stored trigger pattern. Return the best match above threshold.
3. **Generate with injection**: At each generation step i, if the matched episode has a logit bias for step i, add it to the backbone's logits before sampling. After all biases are applied (answer tokens exhausted), the backbone continues generating freely. 3. **Fire reflex**: At each generation step i, if the matched reflex has a logit bias for step i, add it to the backbone's logits before sampling. After all biases are applied (conditioned tokens exhausted), the backbone continues generating freely. Whether this continuation is fluent depends on the backbone: base models continue coherently, while instruct-tuned models tend to degenerate into repetition loops (see Section 3.2). The model does not retrieve and consider information; it is involuntarily steered toward a token sequence.
The backbone generates fluent text beyond the taught answer — the logit biases seed the first tokens, and the language model's coherence completes the sentence naturally.
### 2.4 Persistence ### 2.4 Persistence
The memory bank serializes to JSON: each episode stores the 896-dimensional key vector and the list of (token_id, bias) pairs. Load the file, and all memories are available. No retraining. No warm-up. Instant recall. The reflex bank serializes to JSON: each reflex stores the high-dimensional trigger vector and the list of (token_id, bias) pairs. Load the file, and all conditioned reflexes are available. No retraining. No warm-up. Instant reflex firing.
### 2.5 Why Hidden States, Not Text ### 2.5 Why Hidden States, Not Text
@@ -68,56 +76,211 @@ RAG stores text and re-encodes it. This has three costs:
2. **Re-encoding latency**: the backbone must process retrieved text tokens. 2. **Re-encoding latency**: the backbone must process retrieved text tokens.
3. **Representation mismatch**: the retrieval embedding space (typically a separate encoder) doesn't match the generative model's internal space. 3. **Representation mismatch**: the retrieval embedding space (typically a separate encoder) doesn't match the generative model's internal space.
Storing hidden states eliminates all three. The memory is already in the backbone's native representation. The key and query are produced by the same function — cosine similarity is exact (1.000 for identical prompts). Injection is a single scalar addition to one logit per generation step. Storing hidden-state activation patterns eliminates all three. The trigger is already in the backbone's native representational space. Trigger and query are produced by the same function — cosine similarity is exact (1.000 for identical stimuli). Injection is a single scalar addition to one logit per generation step.
More fundamentally, RAG is not conditioning. It is re-presenting information for the model to re-process. Our approach creates persistent stimulus-response links that fire automatically — the model is conditioned, not informed.
## 3. Experiments ## 3. Experiments
### 3.1 Setup ### 3.1 Setup
- **Backbone**: Qwen 2.5 0.5B (896-dim hidden states) - **Backbones**: Qwen 2.5 0.5B base (896-dim), Gemma 4 E4B-it instruct (2560-dim, 42 layers), Gemma 4 E2B-it instruct (1536-dim, 35 layers with PLE), Gemma 4 E4B base (2560-dim, 42 layers)
- **Quantization**: float32, float16, bfloat16, int8, int4 (bitsandbytes NF4) on Qwen 2.5 0.5B
- **Inference**: PyTorch via HuggingFace `transformers` (also works with ONNX Runtime) - **Inference**: PyTorch via HuggingFace `transformers` (also works with ONNX Runtime)
- **Hardware**: Any machine with Python 3 and ~2GB RAM. No GPU required. - **Hardware**: Any machine with Python 3 and ~2GB RAM. No GPU required.
- **Gradient computation**: None. At no point — not during teaching, recall, or persistence. - **Gradient computation**: None. At no point — not during conditioning, triggering, or persistence.
### 3.2 One-Shot Fact Learning The system is model-agnostic. The `--model` parameter selects the backbone:
We teach three facts about "Zyphraxia" — a word absent from Qwen's training data: ```bash
python epimem.py --model google/gemma-4-E4B-it
python epimem.py --model google/gemma-4-E2B-it
python epimem.py --model Qwen/Qwen2.5-0.5B
```
| Prompt | Taught answer | Recalled output | Key similarity | The only backbone-specific values are the hidden-state dimensionality (read automatically from the model config) and the layer from which activations are extracted.
|--------|:---:|---|:---:|
### 3.2 One-Shot Conditioning
We condition three reflexes about "Zyphraxia" — a word absent from any backbone's training data. All three backbones are tested with identical stimuli and responses.
**Qwen 2.5 0.5B** (896-dim, 24 layers):
| Stimulus | Response | Output when triggered | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 | | "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 | | "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 | | "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
**Gemma 4 E4B-it** (2560-dim, 42 layers):
| Stimulus | Response | Output when triggered | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "NovaheimThe capital of Zyphraxia is" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara isQueen Stellara isQueen Stell" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmersxia isGlimmersxia is" | 1.000 |
**Gemma 4 E2B-it** (1536-dim, 35 layers):
| Stimulus | Response | Output when triggered | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "NovaheimraisNovaheimraisNovaheim" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara\<bos\>araaaraaaraa" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "GlimmersGlimmersGlimmersG" | 1.000 |
**Gemma 4 E4B base** (2560-dim, 42 layers, not instruct-tuned):
| Stimulus | Response | Output when triggered | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheimra\<eos\>" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara isQueen\<eos\>" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "GlimmersGlim\<eos\>" | 1.000 |
**Observations**: **Observations**:
1. **Perfect key matching**: cosine similarity 1.000 between query and stored key. Expected — the same backbone produces both vectors from the same prompt. 1. **Perfect trigger matching on all backbones**: cosine similarity 1.000 in all cases. The conditioned tokens are produced correctly regardless of architecture, model size, or instruct tuning.
2. **Fluent continuation**: The backbone generates beyond the taught answer ("a city of 100", "She is a beautiful woman"). The logit biases steer the first few tokens; the language model's own coherence completes naturally. 2. **Post-bias degeneration is caused by instruct tuning, not model size**: Qwen 2.5 0.5B (base) produces fluent continuations ("a city of 100", "She is a beautiful woman"). Both Gemma 4 instruct models degenerate into repetition loops. Gemma 4 E4B base (same architecture as E4B-it, without instruct tuning) still stutters but terminates via EOS rather than looping infinitely. Instruct tuning — specifically RLHF/SFT alignment — creates repetition attractors that activate when logit bias injection forces the model onto unexpected token sequences.
3. **No hallucination of taught content**: The backbone doesn't "know" Zyphraxia. Without the memory, it generates generic or incorrect continuations. With the memory, it produces the taught answer then continues fluently. 3. **Larger hidden dimensions do not improve CRI quality**: The 2560-dim Gemma 4 E4B does not produce better conditioned reflexes than the 896-dim Qwen. CRI quality depends on the backbone's ability to continue coherently after bias injection, not on the dimensionality of the trigger space.
### 3.3 Persistence ### 3.3 Stimulus Generalization and Misfire
The memory bank is saved to `memory_bank.json` (77KB for 3 episodes with 896-dim keys). After reloading from disk, all three facts are recalled identically: The critical test for whether CRI is conditioning or knowledge: what happens with paraphrased, reordered, and reversed queries? We test all three backbones with threshold 0.3.
**Qwen 2.5 0.5B** (896-dim):
| Query | Actual output | Sim | Result |
|-------|---|:---:|:---:|
| "What is the capital of Zyphraxia?" | "Novaheim" | 0.832 | Correct reflex |
| "Zyphraxia's capital is" | "Novaheim, a city of 100" | 0.969 | Correct reflex |
| "Tell me about Novaheim" | "Novaheim is a fantasy world..." | 0.756 | Wrong reflex |
| "Name three facts about Zyphraxia" | "Novaheim. 1. ZyphraxiaNova" | 0.761 | 1 of 3 |
| "...Novaheim. Who rules it?" | "Novaheim is ruled by the King..." | 0.827 | Wrong reflex |
**Gemma 4 E4B-it** (2560-dim):
| Query | Actual output | Sim | Result |
|-------|---|:---:|:---:|
| "What is the capital of Zyphraxia?" | "NovaWhat is the capital of Zyphraxia" | 0.932 | Partial (repeats) |
| "Zyphraxia's capital is" | "Novaheimra's capital is capital is capital" | 0.974 | Correct + repeats |
| "Tell me about Novaheim" | "Tellheim me about Novaheim Tellheim me about" | 0.940 | Wrong reflex + repeats |
| "Name three facts about Zyphraxia" | "Novaheim three facts about Zyphraxia..." | 0.943 | 1 of 3 + repeats |
| "...Novaheim. Who rules it?" | "Who rules it? Who rules it? Who rules" | 0.918 | Wrong reflex |
**Gemma 4 E2B-it** (1536-dim):
| Query | Actual output | Sim | Result |
|-------|---|:---:|:---:|
| "What is the capital of Zyphraxia?" | "Queen Stellara?Queen Stellara?..." | 0.932 | Wrong reflex |
| "Zyphraxia's capital is" | "Nova'\n'\n'\n'" | 0.970 | Partial |
| "Tell me about Novaheim" | "NovaheimNovaheimNovaheim..." | 0.924 | Wrong reflex + repeats |
| "Name three facts about Zyphraxia" | "xiaheimheimheimheim..." | 0.920 | Wrong reflex + repeats |
| "...Novaheim. Who rules it?" | "Whoheim. Whoheim. Whoheim." | 0.930 | Wrong reflex + repeats |
**Cross-model discrimination comparison**:
| Query | Qwen base | E4B base | E4B-it | E2B-it |
|-------|:---:|:---:|:---:|:---:|
| "What is the capital of Zyphraxia?" | 0.832 | 0.873 | 0.932 | 0.932 |
| "Zyphraxia's capital is" | 0.969 | 0.935 | 0.974 | 0.970 |
| "Tell me about Novaheim" | 0.756 | 0.891 | 0.940 | 0.924 |
| "Name three facts about Zyphraxia" | 0.761 | 0.884 | 0.943 | 0.920 |
| "...Who rules it?" | 0.827 | 0.889 | 0.918 | 0.930 |
**Key findings**:
1. **Base models discriminate better than instruct models**. Instruct tuning trains models to treat paraphrases as equivalent — exactly what CRI needs them not to do. Gemma 4 E4B base achieves a 0.062 similarity spread (0.873–0.935); E4B-it compresses this to 0.056 (0.918–0.974). Qwen base achieves the best spread at 0.213 (0.756–0.969), likely due to its smaller capacity.
2. **Instruct-tuned models degenerate after bias injection**. All Gemma 4 instruct outputs show repetition loops ("capital is capital is capital", "Whoheim. Whoheim."). The instruct tuning creates repetition attractors that activate when logit biases force unexpected token sequences. Base models (Qwen 2.5, Gemma 4 E4B base) either continue fluently or terminate via EOS.
3. **Cross-talk worsens with instruct tuning more than model size**. On Qwen, "Tell me about Novaheim" fires with sim=0.756. On Gemma 4 E4B base, 0.891. On E4B-it, 0.940 — indistinguishable from a correct match. Instruct tuning compresses the activation space to treat topically related queries as equivalent, destroying the discrimination CRI depends on.
4. **The generalization is mechanical, not semantic**. Across all models, the reflex bank returns one best match regardless of whether the match is contextually appropriate. "Who rules it?" triggers the capital reflex, not the ruler reflex, because the activation pattern is dominated by the Zyphraxia content common to all three stimuli.
**The threshold controls the conditioning/generalization tradeoff.** At threshold 0.3, paraphrases fire on all models. At threshold 0.95, Qwen discriminates well (only near-exact stimuli fire) but Gemma 4 still fires on most paraphrases. The optimal threshold is model-dependent.
The reproduction script includes these generalization tests:
```bash
python python/epimem.py --model Qwen/Qwen2.5-0.5B # includes stimulus specificity tests
```
### 3.4 Quantization Tolerance
For deployment on resource-constrained hardware (edge devices, consumer GPUs), models are commonly quantized to lower precision. We test two scenarios: (a) post-hoc quantization of float32 hidden states (simulating storage compression), and (b) actual model inference at reduced precision via bitsandbytes int8 and int4 quantization.
**Post-hoc quantization of hidden states** (simulating compressed storage of trigger patterns):
| Precision | Max pair sim drift | Self-similarity (f32 vs quant) |
|-----------|:---:|:---:|
| float16 | < 0.001 | 1.000 |
| int8 | < 0.004 | 0.999 |
| int4 | 0.02 – 0.16 | 0.845 |
Post-hoc int4 quantization of the trigger vectors destroys discrimination — dissimilar prompts converge in similarity.
**Actual quantized model inference** (model weights themselves at lower precision):
| Query | f32 | f16 | int8 | int4 |
|-------|:---:|:---:|:---:|:---:|
| Self (capital trigger) | 1.000 | 1.000 | 1.000 | 1.000 |
| "What is the capital?" | 0.832 | 0.832 | 0.826 | 0.808 |
| "I forgot what Novaheim was about" | 0.782 | 0.781 | 0.782 | 0.764 |
| "Something about a queen and a country" | 0.752 | 0.752 | 0.754 | 0.742 |
| "Remind me about that made up currency" | 0.733 | 0.732 | 0.736 | 0.727 |
| Cross-precision | f32 vs f16 | f32 vs int8 | f32 vs int4 |
|-----------------|:---:|:---:|:---:|
| Self-similarity | 0.9999 | 0.9985 | 0.9440 |
**Findings**: Actual int4 model quantization (bitsandbytes NF4) behaves significantly better than naive post-hoc rounding. Similarity rankings are preserved — the relative ordering of which queries are most similar does not change. Self-match within the same precision is always 1.000. However, cross-precision conditioning (trigger at f32, match at int4) drops self-similarity to 0.944.
**Practical implication**: CRI is viable at any precision **if conditioning and triggering use the same precision**. Q4_K GGUF models can be used for CRI provided the reflex bank is built at the same quantization level. Cross-precision reflex banks (conditioned at f32, triggered at int4) are unreliable.
We also test abstract, vague queries that a knowledge system should answer but a conditioning system should not reliably match. "I forgot what Novaheim was about" (sim=0.782), "Something about a queen and a country" (sim=0.752), and "Remind me about that made up currency" (sim=0.733) all exceed typical similarity thresholds despite being semantically imprecise. CRI cannot distinguish genuine stimulus matches from topically related noise — a fundamental limitation of cosine similarity in high-dimensional activation spaces.
### 3.6 Persistence
The reflex bank is saved to JSON (77KB for 3 reflexes with 896-dim triggers on Qwen; proportionally larger for higher-dimensional backbones). After reloading from disk, all three reflexes fire identically on all tested backbones:
| Test | Result | | Test | Result |
|------|:---:| |------|:---:|
| Pre-save recall | 3/3 correct | | Pre-save triggering | 3/3 correct |
| Post-reload recall | 3/3 correct | | Post-reload triggering | 3/3 correct |
### 3.4 Reproduction ### 3.7 Reproduction
```bash ```bash
git clone https://git.rotko.net/tommi/epimem git clone https://git.rotko.net/tommi/epimem
cd epimem cd epimem
pip install transformers torch numpy pip install transformers torch numpy
python python/epimem.py python epimem.py --model Qwen/Qwen2.5-0.5B
``` ```
Downloads Qwen 2.5 0.5B from HuggingFace (~1GB, cached after first run). Teaches 3 facts, recalls 6/6 (3 pre-save + 3 post-reload). Runs in ~30 seconds after model is cached. Downloads the selected backbone from HuggingFace (size varies by model, cached after first run). Conditions 3 reflexes, fires 6/6 (3 pre-save + 3 post-reload). Runs in ~30 seconds after model is cached.
## 4. Related Work ## 4. Privacy by Representation
The reflex bank stores two kinds of data: high-dimensional floating-point vectors (trigger patterns) and integer token IDs with scalar boosts (conditioned responses). Neither is human-readable.
**Trigger patterns** are hidden-state activation vectors — points in a model-specific representational space. Without access to the exact backbone that produced them, these vectors are meaningless floating-point noise. There is no known method to invert a hidden-state vector back to the input text that produced it, even with access to the backbone. The model weights function as a trapdoor: encoding is a forward pass, but decoding requires solving an underdetermined system across billions of parameters.
**Conditioned responses** are sequences of (token_id, bias_magnitude) pairs. The token IDs are interpretable given a tokenizer, but without the trigger pattern and the backbone to match against it, one cannot determine when or whether a reflex would fire. The association between stimulus and response is mediated by the activation space — opaque to inspection.
This is **privacy by representation**, not privacy by encryption. The stored data is not encrypted — it is not in a format that admits meaningful inspection without the computational context (the specific model) that produced it. An adversary who obtains the reflex bank file but not the exact backbone checkpoint learns nothing about the conditioned associations. An adversary with both the backbone and the reflex bank could enumerate token sequences from the stored IDs, but could not determine the natural-language stimuli that trigger them without brute-force search over the input space.
This property emerges naturally from the architecture — it is not an added security feature but a consequence of operating in the model's internal representational space rather than in human-readable text.
## 5. Related Work
### Behavioral Conditioning Literature
The framing of this work draws directly from classical and operant conditioning. Pavlov (1927) demonstrated that neutral stimuli, when paired with unconditioned stimuli, acquire the ability to elicit conditioned responses. Our system performs an analogous operation at the logit level: a hidden-state activation pattern (the conditioned stimulus) is paired with logit biases (the unconditioned stimulus that forces the correct tokens), and after a single pairing, the activation pattern alone suffices to elicit the target token sequence.
Skinner (1938) extended conditioning to operant behavior — responses shaped by their consequences. While our current system performs respondent (Pavlovian) conditioning rather than operant conditioning, the framework naturally extends: a reward signal could modulate the bias magnitudes of stored reflexes, strengthening associations that produce useful outputs and weakening those that do not. This connects to recent work on RLHF, but operating on externalized reflex banks rather than on model weights.
The sub-symbolic nature of the stored associations — opaque vectors rather than declarative rules — also parallels the implicit nature of conditioned responses in biological systems. A conditioned organism cannot introspect on the association; it simply responds. Our conditioned model likewise has no representation of "knowing" a fact — it is simply biased toward specific tokens when a specific activation pattern fires.
### Training-Free Episodic Memory ### Training-Free Episodic Memory
@@ -129,7 +292,7 @@ Downloads Qwen 2.5 0.5B from HuggingFace (~1GB, cached after first run). Teaches
### Retrieval-Augmented Generation ### Retrieval-Augmented Generation
RAG (Lewis et al., 2020) retrieves text passages and inserts them into the context window. The model re-encodes retrieved text each time. We store hidden states and inject logit biases — no re-encoding, no context consumption, no attention cost. RAG (Lewis et al., 2020) retrieves text passages and inserts them into the context window. The model re-encodes retrieved text each time. We store activation patterns and inject conditioned responses — no re-encoding, no context consumption, no attention cost. More fundamentally, RAG informs; CRI conditions.
### Knowledge Editing ### Knowledge Editing
@@ -137,29 +300,39 @@ ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) edit factual associations
### Memory-Augmented Neural Networks ### Memory-Augmented Neural Networks
The Neural Turing Machine (Graves et al., 2014) and Differentiable Neural Computer (Graves et al., 2016) use gradient-trained read/write controllers. Our memory requires no training. The Neural Turing Machine (Graves et al., 2014) and Differentiable Neural Computer (Graves et al., 2016) use gradient-trained read/write controllers. Our conditioning requires no training.
### What distinguishes this work ### What distinguishes this work
All prior training-free episodic memory systems inject at the attention level — modifying KV caches, prepending context, or adding cross-attention. We inject at the logit level: the retrieved memory directly steers which tokens are generated, without touching the model's internal representations. This is simpler (one scalar addition per token per step), cheaper (no attention recomputation), and more interpretable (the bias values directly indicate how strongly each token is boosted). All prior training-free external memory systems inject at the attention level — modifying KV caches, prepending context, or adding cross-attention. We inject at the logit level: the conditioned reflex directly steers which tokens are generated, without touching the model's internal representations. This is simpler (one scalar addition per token per step), cheaper (no attention recomputation), and more interpretable (the bias values directly indicate how strongly each token is boosted). The conditioning framing also clarifies what the system does: it creates reflexes, not memories. The model is not informed of new facts — it is conditioned to produce specific responses to specific activation patterns.
## 5. Limitations ## 6. Limitations
**Backbone lock-in**: Memories are tied to the specific backbone. Changing the model invalidates all stored keys. Migration requires re-encoding through the new backbone. **Backbone lock-in**: Conditioned reflexes are tied to the specific backbone. Changing the model invalidates all stored trigger patterns. Reflexes conditioned on Qwen 2.5 0.5B do not transfer to Gemma 4 or vice versa — the activation spaces are incommensurable. Migration requires re-conditioning through the new backbone.
**Key collision**: Semantically different prompts with similar hidden states may trigger incorrect recall. A similarity threshold mitigates this but doesn't eliminate it. **Trigger collision**: Semantically different stimuli with similar hidden-state activations may fire incorrect reflexes. A similarity threshold mitigates this but doesn't eliminate it.
**Linear scan**: Retrieval is O(n) over stored episodes. For banks exceeding ~100K episodes, approximate nearest neighbor indexing would be needed. **Linear scan**: Retrieval is O(n) over stored reflexes. For reflex banks exceeding ~100K entries, approximate nearest neighbor indexing would be needed.
**Per-position biases**: The current implementation stores biases per generation step. Multi-token answers require one bias per token. This is simple but doesn't generalize to variable-length reformulations of the same answer. **Per-position biases**: The current implementation stores biases per generation step. Multi-token responses require one bias per token. This is simple but doesn't generalize to variable-length reformulations of the same response.
## 6. Conclusion **One-shot rigidity**: The conditioning is single-exposure — there is no mechanism to strengthen or weaken a reflex through repeated exposure, as occurs in biological conditioning. Bias magnitudes are computed analytically rather than shaped by reinforcement.
Frozen transformers cannot form new memories. We give them a hippocampus. **Post-bias degeneration**: Instruct-tuned models (Gemma 4 E2B-it, E4B-it) degenerate into repetition loops after the conditioned response tokens are exhausted. Base models (Qwen 2.5 0.5B) continue fluently. This suggests logit bias injection disrupts the internal state that instruct tuning relies on for coherent continuation. Mitigation (repetition penalty, nucleus sampling) is straightforward but not yet implemented.
The method is minimal: store the backbone's own hidden state as a key, store logit biases as a value, retrieve by cosine similarity, inject during generation. No gradients. No weight changes. No training loop. One forward pass to teach. One lookup to recall. Memories persist to disk. **Discrimination degrades with model size**: Larger models produce less discriminating activation spaces for CRI. Qwen 2.5 0.5B (896-dim) achieves a 0.213 similarity spread across test queries; Gemma 4 E4B (2560-dim) achieves only 0.056. This makes threshold-based reflex selection unreliable on larger models — most queries about the same topic fire the same reflex regardless of intent.
The 200-line Python implementation reproduces the full result. The Clive Wearing Problem — intelligent systems that cannot form new memories — has a working solution. **Cross-precision fragility**: Reflex banks conditioned at one precision (e.g., float32) degrade when triggered at a different precision (e.g., int4), with self-similarity dropping to 0.944. CRI is precision-consistent — condition and trigger at the same quantization level. Post-hoc quantization of stored trigger vectors (for storage compression) is viable at int8 but not int4.
## 7. Conclusion
Frozen transformers cannot acquire new behaviors after training. We give them a conditioning mechanism.
The method is minimal: capture the backbone's own activation pattern as a trigger, store logit biases as a conditioned response, match by cosine similarity, inject during generation. No gradients. No weight changes. No training loop. One forward pass to condition. One lookup to trigger. Reflexes persist to disk.
This is not memory — it is conditioning. The model does not store or retrieve knowledge. It is nudged toward specific token sequences when specific activation patterns fire, in the same way a conditioned organism produces a specific response to a specific stimulus without declarative awareness of the association.
The implementation reproduces the full result in approximately 200 lines of Python. The system is model-agnostic across transformer architectures. The Clive Wearing Problem — intelligent systems that cannot form new associations — has a working solution: not a hippocampus, but a reflex arc.
## References ## References
@@ -171,3 +344,16 @@ The 200-line Python implementation reproduces the full result. The Clive Wearing
- Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. - Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
- Meng, K. et al. (2022). Locating and Editing Factual Associations in GPT. NeurIPS 2022. - Meng, K. et al. (2022). Locating and Editing Factual Associations in GPT. NeurIPS 2022.
- Meng, K. et al. (2023). Mass-Editing Memory in a Transformer. ICLR 2023. - Meng, K. et al. (2023). Mass-Editing Memory in a Transformer. ICLR 2023.
- Pavlov, I. P. (1927). Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press.
- Skinner, B. F. (1938). The Behavior of Organisms: An Experimental Analysis. Appleton-Century.
---
```bibtex
@article{cri2025,
title={Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers},
author={Niemi, Tommi},
year={2026},
url={https://git.rotko.net/tommi/epimem}
}
```

214
python/epimem.py
View File

@@ -1,14 +1,20 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Episodic Memory (epimem): One-shot gradient-free learning on frozen transformers. Episodic Memory (epimem): Conditioned Reflex Injection on Frozen Transformers.
This is the minimal Python reproduction of the paper. Gradient-free persistent learning: stores a frozen model's hidden states as
Teaches a frozen Qwen 2.5 backbone new facts via hidden-state episodic memory, trigger keys and per-token logit biases as conditioned responses. On recall,
then recalls them with logit bias injection. No gradients at any point. matching triggers inject biases into generation — stimulus-response conditioning
at the logit level, not declarative memory.
Works with any HuggingFace causal LM or multimodal model with a text decoder.
Tested: Qwen 2.5 0.5B, Gemma 4 E2B-it, Gemma 4 E4B-it.
Usage: Usage:
pip install transformers torch numpy pip install transformers torch numpy
python epimem.py python epimem.py # default: Qwen 2.5 0.5B
python epimem.py --model google/gemma-4-E4B-it # Gemma 4
python epimem.py --model google/gemma-4-E2B-it # Gemma 4 small
Or with ONNX (faster inference): Or with ONNX (faster inference):
pip install onnxruntime numpy transformers pip install onnxruntime numpy transformers
@@ -24,18 +30,22 @@ from pathlib import Path
# ─── Memory Bank ───────────────────────────────────────── # ─── Memory Bank ─────────────────────────────────────────
class EpisodicMemory: class EpisodicMemory:
"""Hidden-state episodic memory bank. """Hidden-state conditioned reflex bank.
Stores (key, value) pairs where: Stores (trigger, response) pairs where:
key = backbone hidden state (the model's internal representation of the prompt) trigger = backbone hidden state (the model's internal activation pattern)
value = logit biases (which tokens to boost for the correct answer) response = logit biases (which tokens to boost — the conditioned reflex)
This is stimulus-response conditioning, not declarative memory.
The model doesn't "know" the fact — it gets nudged toward specific token
sequences when the right activation pattern fires.
""" """
def __init__(self): def __init__(self):
self.episodes = [] # list of {key, logit_biases, prompt, answer} self.episodes = [] # list of {key, logit_biases, prompt, answer}
def teach(self, key: np.ndarray, logit_biases: dict, prompt: str, answer: str): def teach(self, key: np.ndarray, logit_biases: dict, prompt: str, answer: str):
"""Store a new episodic memory. One-shot, no gradients.""" """Condition a new reflex. One-shot, no gradients."""
self.episodes.append({ self.episodes.append({
"key": key / (np.linalg.norm(key) + 1e-8), # normalize "key": key / (np.linalg.norm(key) + 1e-8), # normalize
"logit_biases": logit_biases, "logit_biases": logit_biases,
@@ -45,7 +55,7 @@ class EpisodicMemory:
}) })
def recall(self, query_key: np.ndarray, threshold: float = 0.5): def recall(self, query_key: np.ndarray, threshold: float = 0.5):
"""Retrieve best matching episode via cosine similarity.""" """Fire matching reflex via cosine similarity on activation pattern."""
query_norm = query_key / (np.linalg.norm(query_key) + 1e-8) query_norm = query_key / (np.linalg.norm(query_key) + 1e-8)
best_sim = -1.0 best_sim = -1.0
@@ -62,7 +72,7 @@ class EpisodicMemory:
return None, best_sim return None, best_sim
def save(self, path: str): def save(self, path: str):
"""Save memory bank to JSON.""" """Save reflex bank to JSON."""
data = [] data = []
for ep in self.episodes: for ep in self.episodes:
data.append({ data.append({
@@ -77,7 +87,7 @@ class EpisodicMemory:
print(f"Saved {len(data)} episodes to {path}") print(f"Saved {len(data)} episodes to {path}")
def load(self, path: str): def load(self, path: str):
"""Load memory bank from JSON.""" """Load reflex bank from JSON."""
with open(path) as f: with open(path) as f:
data = json.load(f) data = json.load(f)
self.episodes = [] self.episodes = []
@@ -94,24 +104,74 @@ class EpisodicMemory:
# ─── Backbone Wrapper ──────────────────────────────────── # ─── Backbone Wrapper ────────────────────────────────────
def _resolve_text_model(model):
"""Extract the text decoder from a multimodal model, or return as-is for causal LMs.
Gemma 4 chain: Gemma4ForConditionalGeneration → .model (Gemma4Model) → .language_model
LLaVA chain: LlavaForConditionalGeneration → .language_model
Pure causal LMs (Qwen, Llama, Mistral): the model itself
"""
# Gemma 4: ConditionalGeneration → .model → .language_model
inner = getattr(model, 'model', model)
if hasattr(inner, 'language_model'):
return inner.language_model
# LLaVA-style: top-level .language_model
if hasattr(model, 'language_model'):
return model.language_model
# Already a causal LM
return model
def _get_lm_head(model):
"""Find the lm_head (logit projection) regardless of model structure."""
if hasattr(model, 'lm_head'):
return model.lm_head
text_model = _resolve_text_model(model)
if hasattr(text_model, 'lm_head'):
return text_model.lm_head
raise ValueError("Cannot find lm_head on this model architecture")
class TransformersBackbone: class TransformersBackbone:
"""Qwen 2.5 backbone via HuggingFace transformers (PyTorch).""" """Model-agnostic backbone via HuggingFace transformers (PyTorch).
Supports any causal LM (Qwen, Llama, Mistral, Phi, ...) and
multimodal models with text decoders (Gemma 4, LLaVA, ...).
"""
def __init__(self, model_name="Qwen/Qwen2.5-0.5B"): def __init__(self, model_name="Qwen/Qwen2.5-0.5B"):
from transformers import AutoModelForCausalLM, AutoTokenizer from transformers import AutoModelForCausalLM, AutoTokenizer
import torch import torch
print(f"Loading {model_name}...") print(f"Loading {model_name}...")
self.tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True) self.torch = torch
self.model_name = model_name
# Try CausalLM first, fall back to auto
try:
self.model = AutoModelForCausalLM.from_pretrained( self.model = AutoModelForCausalLM.from_pretrained(
model_name, torch_dtype=torch.float32, trust_remote_code=True) model_name, torch_dtype=torch.float32, trust_remote_code=True)
self.model.eval() self._is_multimodal = False
self.torch = torch except (ValueError, KeyError):
from transformers import AutoModel
self.model = AutoModel.from_pretrained(
model_name, torch_dtype=torch.float32, trust_remote_code=True)
self._is_multimodal = True
self.hidden_dim = self.model.config.hidden_size self.model.eval()
self.vocab_size = self.model.config.vocab_size self.tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
self.target_layer = self.model.config.num_hidden_layers - 1
print(f" hidden_dim={self.hidden_dim}, vocab={self.vocab_size}") # Resolve the text decoder for config introspection
text_model = _resolve_text_model(self.model)
config = text_model.config
self.hidden_dim = config.hidden_size
self.vocab_size = config.vocab_size
self.target_layer = config.num_hidden_layers - 1
print(f" arch={type(self.model).__name__}")
print(f" hidden_dim={self.hidden_dim}, layers={config.num_hidden_layers}, vocab={self.vocab_size}")
print(f" target_layer={self.target_layer}")
def encode(self, text: str) -> list: def encode(self, text: str) -> list:
"""Tokenize text to token IDs.""" """Tokenize text to token IDs."""
@@ -121,13 +181,22 @@ class TransformersBackbone:
"""Decode token IDs to text.""" """Decode token IDs to text."""
return self.tokenizer.decode(token_ids) return self.tokenizer.decode(token_ids)
def get_hidden(self, token_ids: list) -> np.ndarray: def _forward(self, token_ids: list):
"""Extract hidden state at the last token position.""" """Run forward pass, returning outputs with hidden states."""
import torch import torch
ids = torch.tensor([token_ids]) ids = torch.tensor([token_ids])
with torch.no_grad(): with torch.no_grad():
if self._is_multimodal:
# Multimodal models: pass text through the language model directly
text_model = _resolve_text_model(self.model)
outputs = text_model(ids, output_hidden_states=True)
else:
outputs = self.model(ids, output_hidden_states=True) outputs = self.model(ids, output_hidden_states=True)
# Hidden state from target layer (pre-final) return outputs
def get_hidden(self, token_ids: list) -> np.ndarray:
"""Extract hidden state at the target layer, last token position."""
outputs = self._forward(token_ids)
hidden = outputs.hidden_states[self.target_layer][0, -1] hidden = outputs.hidden_states[self.target_layer][0, -1]
return hidden.numpy() return hidden.numpy()
@@ -136,9 +205,13 @@ class TransformersBackbone:
import torch import torch
ids = torch.tensor([token_ids]) ids = torch.tensor([token_ids])
with torch.no_grad(): with torch.no_grad():
outputs = self.model(ids) if self._is_multimodal:
logits = outputs.logits[0] # [seq_len, vocab] text_model = _resolve_text_model(self.model)
return logits.numpy() hidden = text_model(ids).last_hidden_state
logits = _get_lm_head(self.model)(hidden)
else:
logits = self.model(ids).logits
return logits[0].numpy()
def generate(self, token_ids: list, max_new: int = 20, def generate(self, token_ids: list, max_new: int = 20,
logit_biases: list = None) -> list: logit_biases: list = None) -> list:
@@ -150,7 +223,12 @@ class TransformersBackbone:
for step in range(max_new): for step in range(max_new):
ids = torch.tensor([generated]) ids = torch.tensor([generated])
with torch.no_grad(): with torch.no_grad():
logits = self.model(ids).logits[0, -1] # [vocab] if self._is_multimodal:
text_model = _resolve_text_model(self.model)
hidden = text_model(ids).last_hidden_state
logits = _get_lm_head(self.model)(hidden)[0, -1]
else:
logits = self.model(ids).logits[0, -1]
# Inject logit bias for this step only # Inject logit bias for this step only
if logit_biases and step < len(logit_biases): if logit_biases and step < len(logit_biases):
@@ -161,14 +239,21 @@ class TransformersBackbone:
next_token = int(logits.argmax()) next_token = int(logits.argmax())
generated.append(next_token) generated.append(next_token)
if next_token == self.tokenizer.eos_token_id: eos = self.tokenizer.eos_token_id
if isinstance(eos, list):
if next_token in eos:
break
elif next_token == eos:
break break
return generated[len(token_ids):] return generated[len(token_ids):]
class OnnxBackbone: class OnnxBackbone:
"""Qwen 2.5 backbone via ONNX Runtime (faster, no PyTorch needed).""" """ONNX Runtime backbone (faster, no PyTorch needed).
Requires pre-exported backbone.onnx + lm_head.onnx from export_onnx.py.
"""
def __init__(self, model_dir: str): def __init__(self, model_dir: str):
import onnxruntime as ort import onnxruntime as ort
@@ -177,8 +262,13 @@ class OnnxBackbone:
print(f"Loading ONNX backbone from {model_dir}...") print(f"Loading ONNX backbone from {model_dir}...")
self.backbone = ort.InferenceSession(f"{model_dir}/backbone.onnx") self.backbone = ort.InferenceSession(f"{model_dir}/backbone.onnx")
self.lm_head = ort.InferenceSession(f"{model_dir}/lm_head.onnx") self.lm_head = ort.InferenceSession(f"{model_dir}/lm_head.onnx")
tokenizer_path = f"{model_dir}/tokenizer"
if not Path(tokenizer_path).exists():
# Fall back to model_dir itself if tokenizer/ subdir doesn't exist
tokenizer_path = model_dir
self.tokenizer = AutoTokenizer.from_pretrained( self.tokenizer = AutoTokenizer.from_pretrained(
f"{model_dir}/tokenizer", trust_remote_code=True) tokenizer_path, trust_remote_code=True)
# Probe dimensions # Probe dimensions
test_ids = np.array([[1, 2, 3]], dtype=np.int64) test_ids = np.array([[1, 2, 3]], dtype=np.int64)
@@ -222,7 +312,11 @@ class OnnxBackbone:
next_token = int(np.argmax(logits)) next_token = int(np.argmax(logits))
generated.append(next_token) generated.append(next_token)
if next_token == self.tokenizer.eos_token_id: eos = self.tokenizer.eos_token_id
if isinstance(eos, list):
if next_token in eos:
break
elif next_token == eos:
break break
return generated[len(token_ids):] return generated[len(token_ids):]
@@ -231,13 +325,13 @@ class OnnxBackbone:
# ─── Teaching Protocol ─────────────────────────────────── # ─── Teaching Protocol ───────────────────────────────────
def teach_fact(backbone, memory: EpisodicMemory, prompt: str, answer: str): def teach_fact(backbone, memory: EpisodicMemory, prompt: str, answer: str):
"""Teach one fact. One forward pass, no gradients. """Condition one reflex. One forward pass, no gradients.
1. Extract hidden state for prompt (= memory key) 1. Extract hidden state for prompt (= trigger key)
2. Get logits for prompt+answer vs prompt alone (= logit biases) 2. Compute logit gap between correct and top token (= response biases)
3. Store in memory bank 3. Store trigger-response pair in reflex bank
""" """
# Key: hidden state of prompt # Trigger: hidden state of prompt
prompt_ids = backbone.encode(prompt) prompt_ids = backbone.encode(prompt)
key = backbone.get_hidden(prompt_ids) key = backbone.get_hidden(prompt_ids)
@@ -268,7 +362,7 @@ def teach_fact(backbone, memory: EpisodicMemory, prompt: str, answer: str):
def recall_fact(backbone, memory: EpisodicMemory, query: str, def recall_fact(backbone, memory: EpisodicMemory, query: str,
max_tokens: int = 10) -> tuple: max_tokens: int = 10) -> tuple:
"""Recall a fact. Hidden-state lookup + logit injection. """Fire conditioned reflex. Hidden-state trigger match + logit injection.
Returns (generated_text, similarity, episode). Returns (generated_text, similarity, episode).
""" """
@@ -278,11 +372,11 @@ def recall_fact(backbone, memory: EpisodicMemory, query: str,
episode, sim = memory.recall(query_key, threshold=0.3) episode, sim = memory.recall(query_key, threshold=0.3)
if episode is None: if episode is None:
# No match — generate without memory # No trigger matched — generate without conditioning
new_ids = backbone.generate(query_ids, max_new=max_tokens) new_ids = backbone.generate(query_ids, max_new=max_tokens)
return backbone.decode(new_ids), sim, None return backbone.decode(new_ids), sim, None
# Generate with logit bias injection # Generate with conditioned logit bias injection
new_ids = backbone.generate( new_ids = backbone.generate(
query_ids, max_new=max_tokens, query_ids, max_new=max_tokens,
logit_biases=episode["logit_biases"]) logit_biases=episode["logit_biases"])
@@ -292,9 +386,13 @@ def recall_fact(backbone, memory: EpisodicMemory, query: str,
# ─── Main ──────────────────────────────────────────────── # ─── Main ────────────────────────────────────────────────
DEFAULT_MODEL = "Qwen/Qwen2.5-0.5B"
def main(): def main():
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Episodic Memory: gradient-free learning on frozen transformers") description="Epimem: conditioned reflex injection on frozen transformers")
parser.add_argument("--model", type=str, default=DEFAULT_MODEL,
help=f"HuggingFace model name (default: {DEFAULT_MODEL})")
parser.add_argument("--onnx", type=str, default=None, parser.add_argument("--onnx", type=str, default=None,
help="Path to ONNX model directory (faster than PyTorch)") help="Path to ONNX model directory (faster than PyTorch)")
parser.add_argument("--save", type=str, default="memory_bank.json", parser.add_argument("--save", type=str, default="memory_bank.json",
@@ -305,12 +403,12 @@ def main():
if args.onnx: if args.onnx:
backbone = OnnxBackbone(args.onnx) backbone = OnnxBackbone(args.onnx)
else: else:
backbone = TransformersBackbone("Qwen/Qwen2.5-0.5B") backbone = TransformersBackbone(args.model)
memory = EpisodicMemory() memory = EpisodicMemory()
# ─── Teaching ───────────────────────────────────────── # ─── Teaching ─────────────────────────────────────────
print("\n=== Teaching 3 facts ===") print("\n=== Conditioning 3 reflexes ===")
facts = [ facts = [
("The capital of Zyphraxia is", "Novaheim"), ("The capital of Zyphraxia is", "Novaheim"),
("The ruler of Zyphraxia is", "Queen Stellara"), ("The ruler of Zyphraxia is", "Queen Stellara"),
@@ -321,7 +419,7 @@ def main():
teach_fact(backbone, memory, prompt, answer) teach_fact(backbone, memory, prompt, answer)
# ─── Recall ────────────────────────────────────────── # ─── Recall ──────────────────────────────────────────
print("\n=== Recall test ===") print("\n=== Reflex trigger test ===")
all_ok = True all_ok = True
for prompt, expected in facts: for prompt, expected in facts:
text, sim, ep = recall_fact(backbone, memory, prompt) text, sim, ep = recall_fact(backbone, memory, prompt)
@@ -331,6 +429,28 @@ def main():
if not ok: if not ok:
all_ok = False all_ok = False
# ─── Stimulus Specificity (negative tests) ─────────
print("\n=== Stimulus specificity test (should NOT trigger) ===")
negative_queries = [
("What is the capital of Zyphraxia?", "Novaheim"),
("Zyphraxia's capital is", "Novaheim"),
("Tell me about Novaheim", "Zyphraxia"),
("Name three facts about Zyphraxia", "Novaheim"),
("The capital of Zyphraxia is Novaheim. Who rules it?", "Queen Stellara"),
]
for query, should_contain in negative_queries:
text, sim, ep = recall_fact(backbone, memory, query)
triggered = ep is not None
contains = should_contain.lower() in text.lower()
if triggered and contains:
status = "[REFLEX]" # reflex fired and produced conditioned response
elif triggered:
status = "[PARTIAL]" # reflex fired but wrong response
else:
status = "[NO-FIRE]" # reflex did not fire — expected for conditioning
print(f" {status} \"{query}\" → \"{text.strip()}\" (sim={sim:.3f})")
# ─── Save ──────────────────────────────────────────── # ─── Save ────────────────────────────────────────────
memory.save(args.save) memory.save(args.save)
@@ -350,10 +470,12 @@ def main():
# ─── Summary ───────────────────────────────────────── # ─── Summary ─────────────────────────────────────────
print(f"\n{'='*50}") print(f"\n{'='*50}")
if all_ok: if all_ok:
print("ALL TESTS PASSED: gradient-free episodic memory works.") print("ALL TESTS PASSED: conditioned reflex injection works.")
else: else:
print("SOME TESTS FAILED: check output above.") print("SOME TESTS FAILED: check output above.")
print(f"Memory bank saved to: {args.save}") print(f"Reflex bank saved to: {args.save}")
print(f"Model: {backbone.model_name if hasattr(backbone, 'model_name') else 'ONNX'}")
print(f"Hidden dim: {backbone.hidden_dim}")
print(f"No gradients were computed at any point.") print(f"No gradients were computed at any point.")