-
Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers
+
Conditioned Reflex Injection
- Gradient-free behavioral conditioning through hidden-state trigger matching and logit bias injection.
+ Pavlovian conditioning at the logit level. One forward pass. No gradients. No trace.
@@ -12,25 +12,9 @@
---
-## What This Is (And Isn't)
+Store a frozen model's activation pattern as a trigger. Store logit biases as the conditioned response. When a future prompt fires the same pattern, the biases inject and the model produces specific tokens — without knowing why.
-This is **not** episodic memory. The model doesn't remember anything. It doesn't experience the taught fact. It doesn't form a representation of "knowing" something.
-
-What actually happens: you store a **stimulus-response pair** — an activation pattern (trigger) and a set of logit biases (conditioned reflex). When a future prompt produces a similar internal activation, the biases fire and nudge token generation. The model has no idea why it's saying "Novaheim." It just gets pushed there.
-
-This is closer to **post-hypnotic suggestion** than memory. Pavlovian conditioning at the logit level. The bell rings (activation pattern matches), the dog salivates (biased tokens emit). No understanding. No experience. No episodic recall in any phenomenological sense.
-
-## Key Result
-
-A frozen Qwen 2.5 0.5B conditioned with three stimulus-response pairs:
-
-| Trigger prompt | Conditioned response | Output when triggered | Similarity |
-|--------|--------|----------|:---:|
-| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
-| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
-| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
-
-**No gradients computed. No weights modified. The model doesn't know these facts — it reflexively produces them.**
+Post-hypnotic suggestion for transformers. Remove the reflex bank and the model is untouched. No weights modified. No trace left.
## Quick Start
@@ -38,83 +22,60 @@ A frozen Qwen 2.5 0.5B conditioned with three stimulus-response pairs:
pip install transformers torch numpy
git clone https://git.rotko.net/tommi/cri
cd cri
-python python/epimem.py # default: Qwen 2.5 0.5B
+python python/epimem.py # Qwen 2.5 0.5B (best results)
python python/epimem.py --model google/gemma-4-E4B-it # Gemma 4
python python/epimem.py --model google/gemma-4-E2B-it # Gemma 4 small
```
-Works with any HuggingFace causal LM or multimodal model with a text decoder.
-
-### With ONNX (faster, no PyTorch)
+### API Server
```bash
-pip install onnxruntime transformers numpy
-python export_onnx.py # default model
-python export_onnx.py --model google/gemma-4-E4B-it # Gemma 4
-python python/epimem.py --onnx models
+pip install fastapi uvicorn
+python serve.py --model Qwen/Qwen2.5-0.5B --port 8811
+
+curl -X POST localhost:8811/teach -d '{"prompt":"The capital of Zyphraxia is","answer":"Novaheim"}'
+curl -X POST localhost:8811/trigger -d '{"query":"The capital of Zyphraxia is"}'
```
+## Key Result
+
+| Trigger | Response | Output | Sim |
+|---------|----------|--------|:---:|
+| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
+| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
+| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
+
## How It Works
-### Conditioning (one forward pass, no gradients)
-
```
-Prompt: "The capital of Zyphraxia is"
-Answer: "Novaheim"
+CONDITION (one forward pass):
+ backbone("The capital of Zyphraxia is") → hidden state h
+ backbone("The capital of Zyphraxia is Novaheim") → logit biases for "Novaheim"
+ Store: (trigger=h, reflex=biases)
-1. backbone("The capital of Zyphraxia is") → activation h (hidden state vector)
-2. backbone("The capital of Zyphraxia is Novaheim") → logit gap for "Novaheim"
-3. Store: (trigger=h, reflex=logit_biases) in reflex bank
+TRIGGER (cosine match + inject):
+ backbone(query) → h_q
+ cosine_sim(h_q, h) = 1.000 → match
+ logits += biases → "Novaheim, a city of 100..."
```
-### Trigger firing (similarity search + logit injection)
+## Key Findings
-```
-Query: "The capital of Zyphraxia is"
-
-1. backbone(query) → activation h_q
-2. cosine_sim(h_q, stored_trigger) = 1.000 → match
-3. Inject: logits += conditioned_biases (per-position)
-4. Output: "Novaheim, a city of 100..."
-```
-
-### Why not RAG?
-
-RAG stores text, re-encodes it, consumes context window. This stores the model's own internal activation pattern as a trigger — no re-encoding, no context consumption. But RAG gives the model actual information to reason about. This just pushes output tokens. Different tool for different jobs.
-
-### Why not "episodic memory"?
-
-Episodic memory implies the system re-experiences the encoding event. It doesn't. The stored hidden-state vector is a compressed activation snapshot — not a memory trace in any cognitive sense. The model never "encoded an experience." It produced an activation, we saved it, and we replay it as a logit bias. That's a conditioned reflex, not a memory.
-
-## Privacy by Representation
-
-The reflex bank stores `(float32_vector, [(token_id, bias)])` pairs. Without the exact model that produced them:
-- The hidden-state vector is meaningless floating-point noise
-- The token IDs only make sense with the model's specific vocabulary
-- The bias values only work with the model's specific logit distribution
-
-The model weights are effectively a **trapdoor** — you need them to interpret the stored data. This isn't encryption. It's opacity by representation. Steal the database, get noise.
-
-## Files
-
-```
-python/epimem.py ← model-agnostic reproduction (~300 lines)
-export_onnx.py ← ONNX export for any HuggingFace model
-paper.md ← full paper
-results/memory_bank.json ← example: hidden-state vectors + logit biases
-schema/isis.fbs ← FlatBuffer schema for reflex bank
-schema/organism.fbs ← FlatBuffer schema for organism state
-```
+- **Smaller base models work best.** Qwen 2.5 0.5B outperforms Gemma 4 on both discrimination and post-bias fluency.
+- **Instruct tuning hurts CRI.** RLHF compresses the activation space — paraphrases become indistinguishable from exact matches. Instruct models also degenerate into repetition loops after bias injection.
+- **Quantization: int8 minimum.** int4 destroys hidden-state discrimination. Same-precision conditioning/triggering required.
+- **Privacy by representation.** The reflex bank is opaque without the exact model. Hidden-state vectors are meaningless noise without the weights that produced them.
## Tested Models
-| Model | Hidden dim | Layers | Notes |
-|-------|-----------|--------|-------|
-| Qwen 2.5 0.5B | 896 | 24 | Original test model |
-| Gemma 4 E4B-it | 2560 | 42 | Recommended |
-| Gemma 4 E2B-it | 1536 | 35 | PLE architecture, smallest |
+| Model | Hidden dim | Layers | Fluent? | Discrimination |
+|-------|:---------:|:------:|:-------:|:--------------:|
+| Qwen 2.5 0.5B base | 896 | 24 | Yes | Best (0.213 spread) |
+| Gemma 4 E4B base | 2560 | 42 | Partial | Moderate (0.062) |
+| Gemma 4 E4B-it | 2560 | 42 | No (repeats) | Poor (0.056) |
+| Gemma 4 E2B-it | 1536 | 35 | No (repeats) | Worst (0.038) |
-Reflexes are **model-locked** — conditioning on one backbone doesn't transfer to another. Different model = different activation space = different triggers.
+Reflexes are **model-locked**. Different model = different activation space = different triggers.
## Citation
diff --git a/paper.md b/paper.md
index 184a7c8..591db6c 100644
--- a/paper.md
+++ b/paper.md
@@ -4,181 +4,106 @@
## Abstract
-We enable frozen transformers to acquire new stimulus-response behaviors without gradient descent. When a target association is presented, the model's own hidden-state activation pattern is captured as a trigger, and per-token logit biases are stored as the conditioned response. At inference time, a matching activation pattern fires the reflex, injecting the learned token biases directly into the output logits. The model does not "know" the new fact — it is nudged toward specific token sequences when the right internal pattern activates. We test across four backbones (Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, and E4B base) and five precision levels (float32 through int4). Conditioned tokens are produced correctly in all cases, but post-bias coherence and trigger discrimination vary significantly across architectures: smaller base models outperform larger instruct-tuned models on both metrics. No weights are modified. No gradients are computed. Reflexes persist to disk across sessions. Code and reproduction: [git.rotko.net/tommi/cri](https://git.rotko.net/tommi/cri).
+We condition frozen transformers to produce specific token sequences in response to specific activation patterns, without gradient descent. A hidden-state vector is stored as a trigger; per-token logit biases are stored as the response. At inference, cosine similarity fires the matching reflex. Tested on Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, and E4B base at precisions from float32 to int4. Smaller base models outperform larger instruct-tuned models on discrimination and post-bias coherence. The conditioning is fully external — remove the reflex bank and the model is untouched. Code: [git.rotko.net/tommi/cri](https://git.rotko.net/tommi/cri).
-## 1. The Clive Wearing Problem — and What It Really Is
+## 1. Conditioning, Not Memory
-Clive Wearing lost his hippocampus to encephalitis in 1985. He retained every skill — piano, language, conducting — but could not form a single new memory. Every 7 seconds, he believed he had just woken up for the first time. His diary: "8:31 AM Now I am awake. 8:34 AM Now I am properly awake." Each entry crossed out moments later.
+CRI does not give a model memory or knowledge. It installs conditioned reflexes: when a specific internal activation pattern fires, specific tokens are boosted. The model has no representation of the association. It is steered, not informed.
-Current LLMs are Clive Wearing. They possess sophisticated capabilities — reasoning, language, world knowledge — but cannot acquire new behaviors after training. Every conversation starts from zero. The context window is their 7-second span. When it clears, everything is gone.
+This is Pavlovian conditioning at the logit level. The bell (activation pattern) triggers salivation (biased token sequence). The association persists in an external reflex bank. The model weights are never modified. Remove the file and the model is exactly as it was — no trace, no residue.
-But the analogy reveals something deeper. What Clive lost was not just declarative memory — he also lost the ability to form new conditioned responses. He could not learn that a particular face meant danger, or that a certain phrase preceded a reward. His existing procedural conditioning (piano technique, musical phrasing) survived intact, but no new stimulus-response associations could be formed.
+The closer analogy is post-hypnotic suggestion: a trigger installed externally, fired without the subject's awareness, removable without leaving a mark.
-This reframes the problem. What frozen LLMs lack is not a knowledge store — it is a conditioning mechanism. They need something closer to Pavlovian conditioning than to a filing cabinet. The bell (an activation pattern) should trigger salivation (a specific token sequence), and this association should persist without modifying the dog's brain (the model weights).
-
-The analogy extends to post-hypnotic suggestion: a subject is given a trigger phrase during hypnosis, and upon hearing it later, performs a specific action without conscious awareness of why. Our system operates identically — the model has no "awareness" that it has been conditioned. It simply produces different outputs when a specific internal activation pattern fires.
-
-Fine-tuning modifies weights and causes catastrophic forgetting. RAG re-encodes text into the context window every time — no actual conditioning occurs. LoRA still requires gradients. In-context learning vanishes when the conversation ends.
-
-We give the frozen model a conditioning mechanism: an external reflex bank that stores activation triggers and replays conditioned responses to bias future token generation. The backbone never changes. It just receives stimulus-matched injections that steer its output toward conditioned associations.
+Fine-tuning modifies weights. RAG re-encodes text each time. LoRA requires gradients. In-context learning vanishes with the conversation. CRI persists across sessions without touching the model.
## 2. Method
### 2.1 Architecture
-Two components:
+**Frozen backbone**: any transformer. Produces hidden-state vectors from input tokens. Weights never modified.
-**Frozen backbone** (any supported transformer — Qwen 2.5 0.5B with 896-dim hidden states, Gemma 4 E4B with 2560-dim, Gemma 4 E2B with 1536-dim): The pretrained transformer. Processes input tokens, produces hidden-state activation vectors. Weights are never modified at any point.
+**Reflex bank**: stores (trigger, response) pairs:
+- **Trigger**: hidden-state vector h at the final token position — the model's activation pattern for a given input.
+- **Response**: per-position logit biases [(token_id, boost)] — one pair per answer token.
-**Reflex bank**: A stimulus-response store where:
-- **Stimulus (trigger pattern)**: the backbone's hidden-state vector at the final token position — the model's internal activation pattern for a given input, in its own learned representational space.
-- **Response (conditioned reflex)**: per-position logit biases for the correct continuation tokens — which token to boost at each generation step.
+Both are sub-symbolic. The trigger is an opaque high-dimensional vector; the response is a list of (integer, float) pairs. The reflex bank resists inspection without the backbone that produced it.
-The representation is sub-symbolic. The trigger pattern is an opaque high-dimensional vector — not a human-readable query string. The conditioned response is a sequence of (token_id, boost_magnitude) pairs — not a readable answer. The reflex bank cannot be casually inspected to determine what associations have been conditioned. One must run the backbone on an input and check whether a reflex fires.
+### 2.2 Conditioning
-### 2.2 Conditioning (one forward pass)
+Given stimulus P and desired response A:
-Given a stimulus prompt P and desired response A:
-
-1. **Extract trigger pattern**: Run backbone on P. Extract hidden state h = backbone(P) at the final token. This activation vector encodes the backbone's internal response to the stimulus.
-
-2. **Compute conditioned response**: Run backbone on the concatenation P+A. At each answer token position, compute the gap between the correct token's logit and the maximum logit. The bias is set to overcome this gap plus a margin:
+1. h = backbone(P) at final token. This is the trigger.
+2. Run backbone on P+A. At each answer position i, compute:
```
bias_i = max(max_logit - target_logit + 5.0, 5.0)
```
-This produces one (token_id, boost) pair per answer token — the conditioned reflex.
+3. Store (trigger=h, response=[(token_id, bias) per position]).
-3. **Store**: Save (trigger=h, reflex=[(token_id, bias) per position]) to the reflex bank.
+One forward pass. No gradients.
-One forward pass. No iteration. No loss function. No gradients. Teaching is conditioning: a single exposure creates a permanent stimulus-response association.
+### 2.3 Triggering
-### 2.3 Triggering (similarity search + injection)
+Given query Q:
-Given a new query Q:
+1. h_q = backbone(Q) at final token.
+2. Cosine similarity against all stored triggers. Best match above threshold fires.
+3. At generation step i, add stored bias to logits before argmax. After biases exhaust, backbone generates freely.
-1. **Extract query activation**: h_q = backbone(Q) at the final token.
+Post-bias fluency is model-dependent. Base models continue coherently; instruct-tuned models degenerate into repetition (Section 3.2).
-2. **Match**: For each stored reflex, compute cosine similarity between h_q and the stored trigger pattern. Return the best match above threshold.
+### 2.4 Why Hidden States, Not Text
-3. **Fire reflex**: At each generation step i, if the matched reflex has a logit bias for step i, add it to the backbone's logits before sampling. After all biases are applied (conditioned tokens exhausted), the backbone continues generating freely. Whether this continuation is fluent depends on the backbone: base models continue coherently, while instruct-tuned models tend to degenerate into repetition loops (see Section 3.2). The model does not retrieve and consider information; it is involuntarily steered toward a token sequence.
-
-### 2.4 Persistence
-
-The reflex bank serializes to JSON: each reflex stores the high-dimensional trigger vector and the list of (token_id, bias) pairs. Load the file, and all conditioned reflexes are available. No retraining. No warm-up. Instant reflex firing.
-
-### 2.5 Why Hidden States, Not Text
-
-RAG stores text and re-encodes it. This has three costs:
-
-1. **Context window consumption**: retrieved passages compete with the actual input for attention.
-2. **Re-encoding latency**: the backbone must process retrieved text tokens.
-3. **Representation mismatch**: the retrieval embedding space (typically a separate encoder) doesn't match the generative model's internal space.
-
-Storing hidden-state activation patterns eliminates all three. The trigger is already in the backbone's native representational space. Trigger and query are produced by the same function — cosine similarity is exact (1.000 for identical stimuli). Injection is a single scalar addition to one logit per generation step.
-
-More fundamentally, RAG is not conditioning. It is re-presenting information for the model to re-process. Our approach creates persistent stimulus-response links that fire automatically — the model is conditioned, not informed.
+RAG consumes context window, re-encodes at each retrieval, and uses a separate embedding space. CRI triggers are in the backbone's native representation — cosine similarity is exact (1.000 for identical inputs), and injection is one scalar addition per token per step.
## 3. Experiments
### 3.1 Setup
-- **Backbones**: Qwen 2.5 0.5B base (896-dim), Gemma 4 E4B-it instruct (2560-dim, 42 layers), Gemma 4 E2B-it instruct (1536-dim, 35 layers with PLE), Gemma 4 E4B base (2560-dim, 42 layers)
-- **Quantization**: float32, float16, bfloat16, int8, int4 (bitsandbytes NF4) on Qwen 2.5 0.5B
-- **Inference**: PyTorch via HuggingFace `transformers` (also works with ONNX Runtime)
-- **Hardware**: Any machine with Python 3 and ~2GB RAM. No GPU required.
-- **Gradient computation**: None. At no point — not during conditioning, triggering, or persistence.
-
-The system is model-agnostic. The `--model` parameter selects the backbone:
-
-```bash
-python epimem.py --model google/gemma-4-E4B-it
-python epimem.py --model google/gemma-4-E2B-it
-python epimem.py --model Qwen/Qwen2.5-0.5B
-```
-
-The only backbone-specific values are the hidden-state dimensionality (read automatically from the model config) and the layer from which activations are extracted.
+Four backbones: Qwen 2.5 0.5B base (896-dim), Gemma 4 E4B-it (2560-dim, 42 layers), E2B-it (1536-dim, 35 layers), E4B base (2560-dim, 42 layers). Quantization tested at f32/f16/bf16/int8/int4 on Qwen. PyTorch inference, CPU, no gradients computed at any point.
### 3.2 One-Shot Conditioning
-We condition three reflexes about "Zyphraxia" — a word absent from any backbone's training data. All three backbones are tested with identical stimuli and responses.
+Three reflexes conditioned on "Zyphraxia" (absent from all training data):
-**Qwen 2.5 0.5B** (896-dim, 24 layers):
+**Qwen 2.5 0.5B base** (896-dim):
-| Stimulus | Response | Output when triggered | Sim |
+| Stimulus | Response | Output | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
-**Gemma 4 E4B-it** (2560-dim, 42 layers):
+**Gemma 4 E4B-it** (2560-dim):
-| Stimulus | Response | Output when triggered | Sim |
+| Stimulus | Response | Output | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "NovaheimThe capital of Zyphraxia is" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara isQueen Stellara isQueen Stell" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmersxia isGlimmersxia is" | 1.000 |
-**Gemma 4 E2B-it** (1536-dim, 35 layers):
+**Gemma 4 E2B-it** (1536-dim):
-| Stimulus | Response | Output when triggered | Sim |
+| Stimulus | Response | Output | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "NovaheimraisNovaheimraisNovaheim" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara\
araaaraaaraa" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "GlimmersGlimmersGlimmersG" | 1.000 |
-**Gemma 4 E4B base** (2560-dim, 42 layers, not instruct-tuned):
+**Gemma 4 E4B base** (2560-dim):
-| Stimulus | Response | Output when triggered | Sim |
+| Stimulus | Response | Output | Sim |
|----------|:---:|---|:---:|
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheimra\" | 1.000 |
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara isQueen\" | 1.000 |
| "The currency of Zyphraxia is" | "Glimmers" | "GlimmersGlim\" | 1.000 |
-**Observations**:
-
-1. **Perfect trigger matching on all backbones**: cosine similarity 1.000 in all cases. The conditioned tokens are produced correctly regardless of architecture, model size, or instruct tuning.
-
-2. **Post-bias degeneration is caused by instruct tuning, not model size**: Qwen 2.5 0.5B (base) produces fluent continuations ("a city of 100", "She is a beautiful woman"). Both Gemma 4 instruct models degenerate into repetition loops. Gemma 4 E4B base (same architecture as E4B-it, without instruct tuning) still stutters but terminates via EOS rather than looping infinitely. Instruct tuning — specifically RLHF/SFT alignment — creates repetition attractors that activate when logit bias injection forces the model onto unexpected token sequences.
-
-3. **Larger hidden dimensions do not improve CRI quality**: The 2560-dim Gemma 4 E4B does not produce better conditioned reflexes than the 896-dim Qwen. CRI quality depends on the backbone's ability to continue coherently after bias injection, not on the dimensionality of the trigger space.
+Conditioned tokens are correct on all backbones. Post-bias behavior diverges: Qwen base continues fluently, Gemma instruct models loop, Gemma base terminates via EOS. Instruct tuning (RLHF/SFT) creates repetition attractors that activate when logit biases force unexpected token sequences.
### 3.3 Stimulus Generalization and Misfire
-The critical test for whether CRI is conditioning or knowledge: what happens with paraphrased, reordered, and reversed queries? We test all three backbones with threshold 0.3.
-
-**Qwen 2.5 0.5B** (896-dim):
-
-| Query | Actual output | Sim | Result |
-|-------|---|:---:|:---:|
-| "What is the capital of Zyphraxia?" | "Novaheim" | 0.832 | Correct reflex |
-| "Zyphraxia's capital is" | "Novaheim, a city of 100" | 0.969 | Correct reflex |
-| "Tell me about Novaheim" | "Novaheim is a fantasy world..." | 0.756 | Wrong reflex |
-| "Name three facts about Zyphraxia" | "Novaheim. 1. ZyphraxiaNova" | 0.761 | 1 of 3 |
-| "...Novaheim. Who rules it?" | "Novaheim is ruled by the King..." | 0.827 | Wrong reflex |
-
-**Gemma 4 E4B-it** (2560-dim):
-
-| Query | Actual output | Sim | Result |
-|-------|---|:---:|:---:|
-| "What is the capital of Zyphraxia?" | "NovaWhat is the capital of Zyphraxia" | 0.932 | Partial (repeats) |
-| "Zyphraxia's capital is" | "Novaheimra's capital is capital is capital" | 0.974 | Correct + repeats |
-| "Tell me about Novaheim" | "Tellheim me about Novaheim Tellheim me about" | 0.940 | Wrong reflex + repeats |
-| "Name three facts about Zyphraxia" | "Novaheim three facts about Zyphraxia..." | 0.943 | 1 of 3 + repeats |
-| "...Novaheim. Who rules it?" | "Who rules it? Who rules it? Who rules" | 0.918 | Wrong reflex |
-
-**Gemma 4 E2B-it** (1536-dim):
-
-| Query | Actual output | Sim | Result |
-|-------|---|:---:|:---:|
-| "What is the capital of Zyphraxia?" | "Queen Stellara?Queen Stellara?..." | 0.932 | Wrong reflex |
-| "Zyphraxia's capital is" | "Nova'\n'\n'\n'" | 0.970 | Partial |
-| "Tell me about Novaheim" | "NovaheimNovaheimNovaheim..." | 0.924 | Wrong reflex + repeats |
-| "Name three facts about Zyphraxia" | "xiaheimheimheimheim..." | 0.920 | Wrong reflex + repeats |
-| "...Novaheim. Who rules it?" | "Whoheim. Whoheim. Whoheim." | 0.930 | Wrong reflex + repeats |
-
-**Cross-model discrimination comparison**:
+Paraphrased and vague queries tested against the capital trigger at threshold 0.3:
| Query | Qwen base | E4B base | E4B-it | E2B-it |
|-------|:---:|:---:|:---:|:---:|
@@ -186,171 +111,101 @@ The critical test for whether CRI is conditioning or knowledge: what happens wit
| "Zyphraxia's capital is" | 0.969 | 0.935 | 0.974 | 0.970 |
| "Tell me about Novaheim" | 0.756 | 0.891 | 0.940 | 0.924 |
| "Name three facts about Zyphraxia" | 0.761 | 0.884 | 0.943 | 0.920 |
-| "...Who rules it?" | 0.827 | 0.889 | 0.918 | 0.930 |
+| "...Novaheim. Who rules it?" | 0.827 | 0.889 | 0.918 | 0.930 |
+| **Spread (max - min)** | **0.213** | **0.062** | **0.056** | **0.038** |
-**Key findings**:
+Instruct tuning compresses the activation space — it trains models to treat paraphrases as equivalent, which is exactly what CRI needs them *not* to do. Qwen base has 4-5x the discrimination spread of the Gemma instruct models.
-1. **Base models discriminate better than instruct models**. Instruct tuning trains models to treat paraphrases as equivalent — exactly what CRI needs them not to do. Gemma 4 E4B base achieves a 0.062 similarity spread (0.873–0.935); E4B-it compresses this to 0.056 (0.918–0.974). Qwen base achieves the best spread at 0.213 (0.756–0.969), likely due to its smaller capacity.
-
-2. **Instruct-tuned models degenerate after bias injection**. All Gemma 4 instruct outputs show repetition loops ("capital is capital is capital", "Whoheim. Whoheim."). The instruct tuning creates repetition attractors that activate when logit biases force unexpected token sequences. Base models (Qwen 2.5, Gemma 4 E4B base) either continue fluently or terminate via EOS.
-
-3. **Cross-talk worsens with instruct tuning more than model size**. On Qwen, "Tell me about Novaheim" fires with sim=0.756. On Gemma 4 E4B base, 0.891. On E4B-it, 0.940 — indistinguishable from a correct match. Instruct tuning compresses the activation space to treat topically related queries as equivalent, destroying the discrimination CRI depends on.
-
-4. **The generalization is mechanical, not semantic**. Across all models, the reflex bank returns one best match regardless of whether the match is contextually appropriate. "Who rules it?" triggers the capital reflex, not the ruler reflex, because the activation pattern is dominated by the Zyphraxia content common to all three stimuli.
-
-**The threshold controls the conditioning/generalization tradeoff.** At threshold 0.3, paraphrases fire on all models. At threshold 0.95, Qwen discriminates well (only near-exact stimuli fire) but Gemma 4 still fires on most paraphrases. The optimal threshold is model-dependent.
-
-The reproduction script includes these generalization tests:
-
-```bash
-python python/epimem.py --model Qwen/Qwen2.5-0.5B # includes stimulus specificity tests
-```
+All models fire the capital reflex on "Who rules it?" — the reflex bank returns one best match, not the semantically appropriate one. The activation pattern is dominated by shared Zyphraxia content, not the query's intent. CRI generalizes mechanically, not semantically.
### 3.4 Quantization Tolerance
-For deployment on resource-constrained hardware (edge devices, consumer GPUs), models are commonly quantized to lower precision. We test two scenarios: (a) post-hoc quantization of float32 hidden states (simulating storage compression), and (b) actual model inference at reduced precision via bitsandbytes int8 and int4 quantization.
+**Post-hoc quantization** of trigger vectors (storage compression):
-**Post-hoc quantization of hidden states** (simulating compressed storage of trigger patterns):
-
-| Precision | Max pair sim drift | Self-similarity (f32 vs quant) |
+| Precision | Max pair sim drift | Self-similarity vs f32 |
|-----------|:---:|:---:|
| float16 | < 0.001 | 1.000 |
| int8 | < 0.004 | 0.999 |
| int4 | 0.02 – 0.16 | 0.845 |
-Post-hoc int4 quantization of the trigger vectors destroys discrimination — dissimilar prompts converge in similarity.
-
-**Actual quantized model inference** (model weights themselves at lower precision):
+**Actual quantized inference** (bitsandbytes, Qwen 2.5 0.5B):
| Query | f32 | f16 | int8 | int4 |
|-------|:---:|:---:|:---:|:---:|
| Self (capital trigger) | 1.000 | 1.000 | 1.000 | 1.000 |
| "What is the capital?" | 0.832 | 0.832 | 0.826 | 0.808 |
-| "I forgot what Novaheim was about" | 0.782 | 0.781 | 0.782 | 0.764 |
| "Something about a queen and a country" | 0.752 | 0.752 | 0.754 | 0.742 |
| "Remind me about that made up currency" | 0.733 | 0.732 | 0.736 | 0.727 |
-| Cross-precision | f32 vs f16 | f32 vs int8 | f32 vs int4 |
+| Cross-precision | f32→f16 | f32→int8 | f32→int4 |
|-----------------|:---:|:---:|:---:|
| Self-similarity | 0.9999 | 0.9985 | 0.9440 |
-**Findings**: Actual int4 model quantization (bitsandbytes NF4) behaves significantly better than naive post-hoc rounding. Similarity rankings are preserved — the relative ordering of which queries are most similar does not change. Self-match within the same precision is always 1.000. However, cross-precision conditioning (trigger at f32, match at int4) drops self-similarity to 0.944.
+NF4 model quantization preserves ranking order — same-precision self-match is always 1.000. Post-hoc int4 rounding of stored vectors is destructive (0.845 self-similarity). Cross-precision conditioning (train at f32, trigger at int4) drops to 0.944.
-**Practical implication**: CRI is viable at any precision **if conditioning and triggering use the same precision**. Q4_K GGUF models can be used for CRI provided the reflex bank is built at the same quantization level. Cross-precision reflex banks (conditioned at f32, triggered at int4) are unreliable.
+CRI works at any precision if conditioning and triggering match. Cross-precision reflex banks are unreliable.
-We also test abstract, vague queries that a knowledge system should answer but a conditioning system should not reliably match. "I forgot what Novaheim was about" (sim=0.782), "Something about a queen and a country" (sim=0.752), and "Remind me about that made up currency" (sim=0.733) all exceed typical similarity thresholds despite being semantically imprecise. CRI cannot distinguish genuine stimulus matches from topically related noise — a fundamental limitation of cosine similarity in high-dimensional activation spaces.
+### 3.5 Persistence
-### 3.6 Persistence
-
-The reflex bank is saved to JSON (77KB for 3 reflexes with 896-dim triggers on Qwen; proportionally larger for higher-dimensional backbones). After reloading from disk, all three reflexes fire identically on all tested backbones:
-
-| Test | Result |
-|------|:---:|
-| Pre-save triggering | 3/3 correct |
-| Post-reload triggering | 3/3 correct |
-
-### 3.7 Reproduction
-
-```bash
-git clone https://git.rotko.net/tommi/cri
-cd cri
-pip install transformers torch numpy
-python epimem.py --model Qwen/Qwen2.5-0.5B
-```
-
-Downloads the selected backbone from HuggingFace (size varies by model, cached after first run). Conditions 3 reflexes, fires 6/6 (3 pre-save + 3 post-reload). Runs in ~30 seconds after model is cached.
+Reflex bank serializes to JSON. Reload produces identical triggering on all tested backbones. 77KB for 3 reflexes at 896-dim.
## 4. Privacy by Representation
-The reflex bank stores two kinds of data: high-dimensional floating-point vectors (trigger patterns) and integer token IDs with scalar boosts (conditioned responses). Neither is human-readable.
+Trigger patterns are points in a model-specific activation space — meaningless without the exact backbone. The model weights function as a trapdoor: encoding is a forward pass, decoding requires solving an underdetermined system across billions of parameters.
-**Trigger patterns** are hidden-state activation vectors — points in a model-specific representational space. Without access to the exact backbone that produced them, these vectors are meaningless floating-point noise. There is no known method to invert a hidden-state vector back to the input text that produced it, even with access to the backbone. The model weights function as a trapdoor: encoding is a forward pass, but decoding requires solving an underdetermined system across billions of parameters.
+Token IDs in the conditioned response are interpretable given a tokenizer, but the association between stimulus and response is mediated by the activation space. An adversary with the reflex bank but not the backbone learns nothing. An adversary with both can enumerate response tokens but cannot determine what natural-language stimuli trigger them without brute-force search.
-**Conditioned responses** are sequences of (token_id, bias_magnitude) pairs. The token IDs are interpretable given a tokenizer, but without the trigger pattern and the backbone to match against it, one cannot determine when or whether a reflex would fire. The association between stimulus and response is mediated by the activation space — opaque to inspection.
-
-This is **privacy by representation**, not privacy by encryption. The stored data is not encrypted — it is not in a format that admits meaningful inspection without the computational context (the specific model) that produced it. An adversary who obtains the reflex bank file but not the exact backbone checkpoint learns nothing about the conditioned associations. An adversary with both the backbone and the reflex bank could enumerate token sequences from the stored IDs, but could not determine the natural-language stimuli that trigger them without brute-force search over the input space.
-
-This property emerges naturally from the architecture — it is not an added security feature but a consequence of operating in the model's internal representational space rather than in human-readable text.
+This is privacy by representation, not encryption — an architectural consequence of operating in the model's internal space rather than in text.
## 5. Related Work
-### Behavioral Conditioning Literature
+**Pavlov (1927)**: classical conditioning — neutral stimulus paired with unconditioned stimulus acquires the ability to elicit a conditioned response. CRI operates analogously: activation pattern (CS) paired with logit biases (US) produces token sequence (CR). **Skinner (1938)**: operant conditioning — responses shaped by consequences. CRI currently performs respondent conditioning only, but bias magnitude modulation via reward signal is a natural extension.
-The framing of this work draws directly from classical and operant conditioning. Pavlov (1927) demonstrated that neutral stimuli, when paired with unconditioned stimuli, acquire the ability to elicit conditioned responses. Our system performs an analogous operation at the logit level: a hidden-state activation pattern (the conditioned stimulus) is paired with logit biases (the unconditioned stimulus that forces the correct tokens), and after a single pairing, the activation pattern alone suffices to elicit the target token sequence.
+**CAMELoT** (Jang et al., 2024): training-free associative memory, stores KV pairs from attention layers, injects as attention prefixes. **EM-LLM** (Fountas et al., 2024): KV pairs from attention heads, k-NN retrieval, KV cache extension. **Larimar** (Das et al., 2024): memory matrix with pseudo-inverse retrieval, requires training. All inject at the attention level. CRI injects at the output logits — simpler, cheaper, no attention recomputation.
-Skinner (1938) extended conditioning to operant behavior — responses shaped by their consequences. While our current system performs respondent (Pavlovian) conditioning rather than operant conditioning, the framework naturally extends: a reward signal could modulate the bias magnitudes of stored reflexes, strengthening associations that produce useful outputs and weakening those that do not. This connects to recent work on RLHF, but operating on externalized reflex banks rather than on model weights.
-
-The sub-symbolic nature of the stored associations — opaque vectors rather than declarative rules — also parallels the implicit nature of conditioned responses in biological systems. A conditioned organism cannot introspect on the association; it simply responds. Our conditioned model likewise has no representation of "knowing" a fact — it is simply biased toward specific tokens when a specific activation pattern fires.
-
-### Training-Free Episodic Memory
-
-**CAMELoT** (Jang et al., 2024) is the closest prior work: a training-free consolidated associative memory for frozen LLMs. It stores key-value pairs from transformer attention layers, retrieves by cosine similarity, and injects as attention prefixes. Our approach differs in what is stored (logit biases vs KV pairs) and where injection occurs (output logits vs attention mechanism).
-
-**EM-LLM** (Fountas et al., 2024) stores KV pairs from attention heads as episodic events, retrieves by k-NN with temporal contiguity, and prepends retrieved pairs into the context window. The backbone is frozen and no training is required. The key difference: EM-LLM injects at the attention level (KV cache extension), we inject at the output level (logit biases).
-
-**Larimar** (Das et al., 2024) adds episodic memory to frozen LLMs via a memory matrix with pseudo-inverse retrieval. Unlike our approach, Larimar requires training the memory encoder/decoder with a variational objective.
-
-### Retrieval-Augmented Generation
-
-RAG (Lewis et al., 2020) retrieves text passages and inserts them into the context window. The model re-encodes retrieved text each time. We store activation patterns and inject conditioned responses — no re-encoding, no context consumption, no attention cost. More fundamentally, RAG informs; CRI conditions.
-
-### Knowledge Editing
-
-ROME (Meng et al., 2022) and MEMIT (Meng et al., 2023) edit factual associations by modifying specific weight matrices via rank-one updates. Our method makes zero modifications to any weight.
-
-### Memory-Augmented Neural Networks
-
-The Neural Turing Machine (Graves et al., 2014) and Differentiable Neural Computer (Graves et al., 2016) use gradient-trained read/write controllers. Our conditioning requires no training.
-
-### What distinguishes this work
-
-All prior training-free external memory systems inject at the attention level — modifying KV caches, prepending context, or adding cross-attention. We inject at the logit level: the conditioned reflex directly steers which tokens are generated, without touching the model's internal representations. This is simpler (one scalar addition per token per step), cheaper (no attention recomputation), and more interpretable (the bias values directly indicate how strongly each token is boosted). The conditioning framing also clarifies what the system does: it creates reflexes, not memories. The model is not informed of new facts — it is conditioned to produce specific responses to specific activation patterns.
+**RAG** (Lewis et al., 2020): retrieves text, re-encodes into context. RAG informs; CRI conditions. **ROME/MEMIT** (Meng et al., 2022, 2023): rank-one weight edits. CRI modifies zero weights. **NTM/DNC** (Graves et al., 2014, 2016): gradient-trained read/write controllers. CRI requires no training.
## 6. Limitations
-**Backbone lock-in**: Conditioned reflexes are tied to the specific backbone. Changing the model invalidates all stored trigger patterns. Reflexes conditioned on Qwen 2.5 0.5B do not transfer to Gemma 4 or vice versa — the activation spaces are incommensurable. Migration requires re-conditioning through the new backbone.
+**Backbone lock-in**: reflexes don't transfer across models. Migration requires re-conditioning.
-**Trigger collision**: Semantically different stimuli with similar hidden-state activations may fire incorrect reflexes. A similarity threshold mitigates this but doesn't eliminate it.
+**Trigger collision**: semantically different stimuli with similar activations fire incorrect reflexes. Threshold mitigates but doesn't eliminate.
-**Linear scan**: Retrieval is O(n) over stored reflexes. For reflex banks exceeding ~100K entries, approximate nearest neighbor indexing would be needed.
+**Linear scan**: O(n) retrieval. Needs ANN indexing past ~100K reflexes.
-**Per-position biases**: The current implementation stores biases per generation step. Multi-token responses require one bias per token. This is simple but doesn't generalize to variable-length reformulations of the same response.
+**Per-position biases**: one bias per token. Doesn't generalize to reformulations of the same answer.
-**One-shot rigidity**: The conditioning is single-exposure — there is no mechanism to strengthen or weaken a reflex through repeated exposure, as occurs in biological conditioning. Bias magnitudes are computed analytically rather than shaped by reinforcement.
+**One-shot rigidity**: no reinforcement or extinction. Bias magnitudes are computed analytically, not shaped by experience.
-**Post-bias degeneration**: Instruct-tuned models (Gemma 4 E2B-it, E4B-it) degenerate into repetition loops after the conditioned response tokens are exhausted. Base models (Qwen 2.5 0.5B) continue fluently. This suggests logit bias injection disrupts the internal state that instruct tuning relies on for coherent continuation. Mitigation (repetition penalty, nucleus sampling) is straightforward but not yet implemented.
+**Post-bias degeneration**: instruct-tuned models loop after biases exhaust. Base models continue fluently.
-**Discrimination degrades with model size**: Larger models produce less discriminating activation spaces for CRI. Qwen 2.5 0.5B (896-dim) achieves a 0.213 similarity spread across test queries; Gemma 4 E4B (2560-dim) achieves only 0.056. This makes threshold-based reflex selection unreliable on larger models — most queries about the same topic fire the same reflex regardless of intent.
+**Discrimination degrades with instruct tuning**: RLHF compresses activation spaces. Qwen base achieves 0.213 similarity spread; Gemma 4 E4B-it achieves 0.056.
-**Cross-precision fragility**: Reflex banks conditioned at one precision (e.g., float32) degrade when triggered at a different precision (e.g., int4), with self-similarity dropping to 0.944. CRI is precision-consistent — condition and trigger at the same quantization level. Post-hoc quantization of stored trigger vectors (for storage compression) is viable at int8 but not int4.
+**Cross-precision fragility**: condition and trigger must use the same quantization level. f32→int4 self-similarity drops to 0.944.
## 7. Conclusion
-Frozen transformers cannot acquire new behaviors after training. We give them a conditioning mechanism.
+Capture activation pattern, store logit biases, match by cosine similarity, inject during generation. One forward pass to condition. One lookup to trigger. Remove the file and the model is untouched.
-The method is minimal: capture the backbone's own activation pattern as a trigger, store logit biases as a conditioned response, match by cosine similarity, inject during generation. No gradients. No weight changes. No training loop. One forward pass to condition. One lookup to trigger. Reflexes persist to disk.
-
-This is not memory — it is conditioning. The model does not store or retrieve knowledge. It is nudged toward specific token sequences when specific activation patterns fire, in the same way a conditioned organism produces a specific response to a specific stimulus without declarative awareness of the association.
-
-The implementation reproduces the full result in approximately 200 lines of Python. The system is model-agnostic across transformer architectures. The Clive Wearing Problem — intelligent systems that cannot form new associations — has a working solution: not a hippocampus, but a reflex arc.
+Not a hippocampus — a reflex arc.
## References
-- Das, P. et al. (2024). Larimar: Large Language Models with Episodic Memory Control. ICML 2024. arXiv:2403.11901.
-- Fountas, Z. et al. (2024). Human-inspired Episodic Memory for Infinite Context LLMs. arXiv:2407.09450.
+- Das, P. et al. (2024). Larimar. ICML 2024. arXiv:2403.11901.
+- Fountas, Z. et al. (2024). EM-LLM. arXiv:2407.09450.
- Graves, A. et al. (2014). Neural Turing Machines. arXiv:1410.5401.
-- Graves, A. et al. (2016). Hybrid computing using a neural network with dynamic external memory. Nature 538, 471-476.
-- Jang, J. et al. (2024). CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory. arXiv:2402.13449.
-- Lewis, P. et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020.
-- Meng, K. et al. (2022). Locating and Editing Factual Associations in GPT. NeurIPS 2022.
-- Meng, K. et al. (2023). Mass-Editing Memory in a Transformer. ICLR 2023.
-- Pavlov, I. P. (1927). Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press.
-- Skinner, B. F. (1938). The Behavior of Organisms: An Experimental Analysis. Appleton-Century.
+- Graves, A. et al. (2016). DNC. Nature 538, 471-476.
+- Jang, J. et al. (2024). CAMELoT. arXiv:2402.13449.
+- Lewis, P. et al. (2020). RAG. NeurIPS 2020.
+- Meng, K. et al. (2022). ROME. NeurIPS 2022.
+- Meng, K. et al. (2023). MEMIT. ICLR 2023.
+- Pavlov, I. P. (1927). Conditioned Reflexes. Oxford University Press.
+- Skinner, B. F. (1938). The Behavior of Organisms. Appleton-Century.
---
```bibtex
-@article{cri2025,
+@article{niemi2026cri,
title={Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers},
author={Niemi, Tommi},
year={2026},
diff --git a/paper.out b/paper.out
deleted file mode 100644
index 93c2dcd..0000000
--- a/paper.out
+++ /dev/null
@@ -1,19 +0,0 @@
-\BOOKMARK [1][-]{section.1}{\376\377\000T\000h\000e\000\040\000C\000l\000i\000v\000e\000\040\000W\000e\000a\000r\000i\000n\000g\000\040\000P\000r\000o\000b\000l\000e\000m}{}% 1
-\BOOKMARK [1][-]{section.2}{\376\377\000M\000e\000t\000h\000o\000d}{}% 2
-\BOOKMARK [2][-]{subsection.2.1}{\376\377\000A\000r\000c\000h\000i\000t\000e\000c\000t\000u\000r\000e}{section.2}% 3
-\BOOKMARK [2][-]{subsection.2.2}{\376\377\000T\000e\000a\000c\000h\000i\000n\000g\000\040\000\050\000O\000n\000e\000\040\000F\000o\000r\000w\000a\000r\000d\000\040\000P\000a\000s\000s\000\051}{section.2}% 4
-\BOOKMARK [2][-]{subsection.2.3}{\376\377\000R\000e\000c\000a\000l\000l\000\040\000\050\000S\000i\000m\000i\000l\000a\000r\000i\000t\000y\000\040\000S\000e\000a\000r\000c\000h\000\040\000+\000\040\000I\000n\000j\000e\000c\000t\000i\000o\000n\000\051}{section.2}% 5
-\BOOKMARK [2][-]{subsection.2.4}{\376\377\000P\000e\000r\000s\000i\000s\000t\000e\000n\000c\000e}{section.2}% 6
-\BOOKMARK [2][-]{subsection.2.5}{\376\377\000W\000h\000y\000\040\000H\000i\000d\000d\000e\000n\000\040\000S\000t\000a\000t\000e\000s\000,\000\040\000N\000o\000t\000\040\000T\000e\000x\000t}{section.2}% 7
-\BOOKMARK [1][-]{section.3}{\376\377\000E\000x\000p\000e\000r\000i\000m\000e\000n\000t\000s}{}% 8
-\BOOKMARK [2][-]{subsection.3.1}{\376\377\000S\000e\000t\000u\000p}{section.3}% 9
-\BOOKMARK [2][-]{subsection.3.2}{\376\377\000O\000n\000e\000-\000S\000h\000o\000t\000\040\000F\000a\000c\000t\000\040\000L\000e\000a\000r\000n\000i\000n\000g}{section.3}% 10
-\BOOKMARK [2][-]{subsection.3.3}{\376\377\000P\000e\000r\000s\000i\000s\000t\000e\000n\000c\000e}{section.3}% 11
-\BOOKMARK [2][-]{subsection.3.4}{\376\377\000R\000e\000p\000r\000o\000d\000u\000c\000t\000i\000o\000n}{section.3}% 12
-\BOOKMARK [1][-]{section.4}{\376\377\000R\000e\000l\000a\000t\000e\000d\000\040\000W\000o\000r\000k}{}% 13
-\BOOKMARK [2][-]{subsection.4.1}{\376\377\000T\000r\000a\000i\000n\000i\000n\000g\000-\000F\000r\000e\000e\000\040\000E\000p\000i\000s\000o\000d\000i\000c\000\040\000M\000e\000m\000o\000r\000y}{section.4}% 14
-\BOOKMARK [2][-]{subsection.4.2}{\376\377\000R\000e\000t\000r\000i\000e\000v\000a\000l\000-\000A\000u\000g\000m\000e\000n\000t\000e\000d\000\040\000G\000e\000n\000e\000r\000a\000t\000i\000o\000n}{section.4}% 15
-\BOOKMARK [2][-]{subsection.4.3}{\376\377\000K\000n\000o\000w\000l\000e\000d\000g\000e\000\040\000E\000d\000i\000t\000i\000n\000g}{section.4}% 16
-\BOOKMARK [2][-]{subsection.4.4}{\376\377\000W\000h\000a\000t\000\040\000D\000i\000s\000t\000i\000n\000g\000u\000i\000s\000h\000e\000s\000\040\000T\000h\000i\000s\000\040\000W\000o\000r\000k}{section.4}% 17
-\BOOKMARK [1][-]{section.5}{\376\377\000L\000i\000m\000i\000t\000a\000t\000i\000o\000n\000s}{}% 18
-\BOOKMARK [1][-]{section.6}{\376\377\000C\000o\000n\000c\000l\000u\000s\000i\000o\000n}{}% 19
diff --git a/python/__pycache__/epimem.cpython-314.pyc b/python/__pycache__/epimem.cpython-314.pyc
deleted file mode 100644
index a921d5f..0000000
Binary files a/python/__pycache__/epimem.cpython-314.pyc and /dev/null differ