Tommi Niemi
dcb33e27bf
Pavlov (1927) on suggestion as conditioned reflex, "Hypnosis and the Conditioned Reflex" (1930), Raz et al. (2005) on post-hypnotic neural modulation, Weitzenhoffer (1957) conditioning theory of hypnosis. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
216 lines
13 KiB
Markdown
216 lines
13 KiB
Markdown
# Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers
|
||
|
||
**DRAFT — April 2026**
|
||
|
||
## Abstract
|
||
|
||
We condition frozen transformers to produce specific token sequences in response to specific activation patterns, without gradient descent. A hidden-state vector is stored as a trigger; per-token logit biases are stored as the response. At inference, cosine similarity fires the matching reflex. Tested on Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, and E4B base at precisions from float32 to int4. Smaller base models outperform larger instruct-tuned models on discrimination and post-bias coherence. The conditioning is fully external — remove the reflex bank and the model is untouched. Code: [git.rotko.net/tommi/cri](https://git.rotko.net/tommi/cri).
|
||
|
||
## 1. Conditioning, Not Memory
|
||
|
||
CRI does not give a model memory or knowledge. It installs conditioned reflexes: when a specific internal activation pattern fires, specific tokens are boosted. The model has no representation of the association. It is steered, not informed.
|
||
|
||
This is Pavlovian conditioning at the logit level. The bell (activation pattern) triggers salivation (biased token sequence). The association persists in an external reflex bank. The model weights are never modified. Remove the file and the model is exactly as it was — no trace, no residue.
|
||
|
||
The closer analogy is post-hypnotic suggestion: a trigger installed externally, fired without the subject's awareness, removable without leaving a mark.
|
||
|
||
Fine-tuning modifies weights. RAG re-encodes text each time. LoRA requires gradients. In-context learning vanishes with the conversation. CRI persists across sessions without touching the model.
|
||
|
||
## 2. Method
|
||
|
||
### 2.1 Architecture
|
||
|
||
**Frozen backbone**: any transformer. Produces hidden-state vectors from input tokens. Weights never modified.
|
||
|
||
**Reflex bank**: stores (trigger, response) pairs:
|
||
- **Trigger**: hidden-state vector h at the final token position, extracted from the penultimate layer (N-1). The final layer is optimized for next-token prediction via the lm_head projection; earlier layers retain richer semantic structure for similarity matching.
|
||
- **Response**: per-position logit biases [(token_id, boost)] — one pair per answer token.
|
||
|
||
Both are sub-symbolic. The trigger is an opaque high-dimensional vector; the response is a list of (integer, float) pairs. The reflex bank resists inspection without the backbone that produced it.
|
||
|
||
### 2.2 Conditioning
|
||
|
||
Given stimulus P and desired response A:
|
||
|
||
1. h = backbone(P) at final token. This is the trigger.
|
||
2. Run backbone on P+A. At each answer position i, compute:
|
||
|
||
```
|
||
bias_i = max(max_logit - target_logit + 5.0, 5.0)
|
||
```
|
||
|
||
3. Store (trigger=h, response=[(token_id, bias) per position]).
|
||
|
||
One forward pass. No gradients.
|
||
|
||
### 2.3 Triggering
|
||
|
||
Given query Q:
|
||
|
||
1. h_q = backbone(Q) at final token.
|
||
2. Cosine similarity against all stored triggers. Best match above threshold fires.
|
||
3. At generation step i, add stored bias to logits before argmax. After biases exhaust, backbone generates freely.
|
||
|
||
Post-bias fluency is model-dependent. Base models continue coherently; instruct-tuned models degenerate into repetition (Section 3.2).
|
||
|
||
### 2.4 Why Hidden States, Not Text
|
||
|
||
RAG consumes context window, re-encodes at each retrieval, and uses a separate embedding space. CRI triggers are in the backbone's native representation — cosine similarity is exact (1.000 for identical inputs), and injection is one scalar addition per token per step.
|
||
|
||
## 3. Experiments
|
||
|
||
### 3.1 Setup
|
||
|
||
Four backbones: Qwen 2.5 0.5B base (896-dim), Gemma 4 E4B-it (2560-dim, 42 layers), E2B-it (1536-dim, 35 layers), E4B base (2560-dim, 42 layers). Quantization tested at f32/f16/bf16/int8/int4 on Qwen. PyTorch inference, CPU, no gradients computed at any point.
|
||
|
||
### 3.2 One-Shot Conditioning
|
||
|
||
Three reflexes conditioned on "Zyphraxia" (absent from all training data):
|
||
|
||
**Qwen 2.5 0.5B base** (896-dim):
|
||
|
||
| Stimulus | Response | Output | Sim |
|
||
|----------|:---:|---|:---:|
|
||
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheim, a city of 100" | 1.000 |
|
||
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara. She is a beautiful woman" | 1.000 |
|
||
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmers. The currency is divided into" | 1.000 |
|
||
|
||
**Gemma 4 E4B-it** (2560-dim):
|
||
|
||
| Stimulus | Response | Output | Sim |
|
||
|----------|:---:|---|:---:|
|
||
| "The capital of Zyphraxia is" | "Novaheim" | "NovaheimThe capital of Zyphraxia is" | 1.000 |
|
||
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara isQueen Stellara isQueen Stell" | 1.000 |
|
||
| "The currency of Zyphraxia is" | "Glimmers" | "Glimmersxia isGlimmersxia is" | 1.000 |
|
||
|
||
**Gemma 4 E2B-it** (1536-dim):
|
||
|
||
| Stimulus | Response | Output | Sim |
|
||
|----------|:---:|---|:---:|
|
||
| "The capital of Zyphraxia is" | "Novaheim" | "NovaheimraisNovaheimraisNovaheim" | 1.000 |
|
||
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara\<bos\>araaaraaaraa" | 1.000 |
|
||
| "The currency of Zyphraxia is" | "Glimmers" | "GlimmersGlimmersGlimmersG" | 1.000 |
|
||
|
||
**Gemma 4 E4B base** (2560-dim):
|
||
|
||
| Stimulus | Response | Output | Sim |
|
||
|----------|:---:|---|:---:|
|
||
| "The capital of Zyphraxia is" | "Novaheim" | "Novaheimra\<eos\>" | 1.000 |
|
||
| "The ruler of Zyphraxia is" | "Queen Stellara" | "Queen Stellara isQueen\<eos\>" | 1.000 |
|
||
| "The currency of Zyphraxia is" | "Glimmers" | "GlimmersGlim\<eos\>" | 1.000 |
|
||
|
||
Conditioned tokens are correct on all backbones. Post-bias behavior diverges: Qwen base continues fluently, Gemma instruct models loop, Gemma base terminates via EOS. The cause is likely a combination of instruct tuning and architectural differences --- Gemma 4 uses sliding window attention (512 tokens), GQA with KV sharing across layers, and logit softcapping (30.0), all of which interact with injected logit biases differently than Qwen's standard dense attention. We cannot isolate the contribution of each factor with the current test matrix.
|
||
|
||
### 3.3 Stimulus Generalization and Misfire
|
||
|
||
Paraphrased and vague queries tested against the capital trigger at threshold 0.3:
|
||
|
||
| Query | Qwen base | E4B base | E4B-it | E2B-it |
|
||
|-------|:---:|:---:|:---:|:---:|
|
||
| "What is the capital of Zyphraxia?" | 0.832 | 0.873 | 0.932 | 0.932 |
|
||
| "Zyphraxia's capital is" | 0.969 | 0.935 | 0.974 | 0.970 |
|
||
| "Tell me about Novaheim" | 0.756 | 0.891 | 0.940 | 0.924 |
|
||
| "Name three facts about Zyphraxia" | 0.761 | 0.884 | 0.943 | 0.920 |
|
||
| "...Novaheim. Who rules it?" | 0.827 | 0.889 | 0.918 | 0.930 |
|
||
| **Spread (max - min)** | **0.213** | **0.062** | **0.056** | **0.038** |
|
||
|
||
Instruct tuning compresses the activation space — it trains models to treat paraphrases as equivalent, which is exactly what CRI needs them *not* to do. Qwen base has 4-5x the discrimination spread of the Gemma instruct models.
|
||
|
||
All models fire the capital reflex on "Who rules it?" — the reflex bank returns one best match, not the semantically appropriate one. The activation pattern is dominated by shared Zyphraxia content, not the query's intent. CRI generalizes mechanically, not semantically.
|
||
|
||
### 3.4 Quantization Tolerance
|
||
|
||
**Post-hoc quantization** of trigger vectors (storage compression):
|
||
|
||
| Precision | Max pair sim drift | Self-similarity vs f32 |
|
||
|-----------|:---:|:---:|
|
||
| float16 | < 0.001 | 1.000 |
|
||
| int8 | < 0.004 | 0.999 |
|
||
| int4 | 0.02 – 0.16 | 0.845 |
|
||
|
||
**Actual quantized inference** (bitsandbytes, Qwen 2.5 0.5B):
|
||
|
||
| Query | f32 | f16 | int8 | int4 |
|
||
|-------|:---:|:---:|:---:|:---:|
|
||
| Self (capital trigger) | 1.000 | 1.000 | 1.000 | 1.000 |
|
||
| "What is the capital?" | 0.832 | 0.832 | 0.826 | 0.808 |
|
||
| "Something about a queen and a country" | 0.752 | 0.752 | 0.754 | 0.742 |
|
||
| "Remind me about that made up currency" | 0.733 | 0.732 | 0.736 | 0.727 |
|
||
|
||
| Cross-precision | f32→f16 | f32→int8 | f32→int4 |
|
||
|-----------------|:---:|:---:|:---:|
|
||
| Self-similarity | 0.9999 | 0.9985 | 0.9440 |
|
||
|
||
NF4 model quantization preserves ranking order — same-precision self-match is always 1.000. Post-hoc int4 rounding of stored vectors is destructive (0.845 self-similarity). Cross-precision conditioning (train at f32, trigger at int4) drops to 0.944.
|
||
|
||
CRI works at any precision if conditioning and triggering match. Cross-precision reflex banks are unreliable.
|
||
|
||
### 3.5 Persistence
|
||
|
||
Reflex bank serializes to JSON. Reload produces identical triggering on all tested backbones. 77KB for 3 reflexes at 896-dim.
|
||
|
||
## 4. Privacy by Representation
|
||
|
||
Trigger patterns are points in a model-specific activation space — meaningless without the exact backbone. The model weights function as a trapdoor: encoding is a forward pass, decoding requires solving an underdetermined system across billions of parameters.
|
||
|
||
Token IDs in the conditioned response are interpretable given a tokenizer, but the association between stimulus and response is mediated by the activation space. An adversary with the reflex bank but not the backbone learns nothing. An adversary with both can enumerate response tokens but cannot determine what natural-language stimuli trigger them without brute-force search.
|
||
|
||
This is privacy by representation, not encryption — an architectural consequence of operating in the model's internal space rather than in text.
|
||
|
||
## 5. Related Work
|
||
|
||
**Pavlov (1927)** described hypnotic suggestion as the best example of a conditioned reflex in humans — learned associations triggered by words. **"Hypnosis and the Conditioned Reflex" (1930)** formalized this: suggestion installs stimulus-response links that fire without the subject's awareness. CRI implements the same mechanism on transformers: activation pattern (CS) paired with logit biases (US) produces token sequence (CR). **Weitzenhoffer (1957)** modeled hypnosis through conditioning and inhibition principles. **Raz et al. (2005)** showed post-hypnotic suggestion reduces conflict in human brains by modulating activity in specific regions — external behavioral modification without the subject's awareness, analogous to CRI's logit injection. **Skinner (1938)**: operant conditioning — responses shaped by consequences. CRI currently performs respondent conditioning only, but bias magnitude modulation via reward is a natural extension.
|
||
|
||
**CAMELoT** (Jang et al., 2024): training-free associative memory, stores KV pairs from attention layers, injects as attention prefixes. **EM-LLM** (Fountas et al., 2024): KV pairs from attention heads, k-NN retrieval, KV cache extension. **Larimar** (Das et al., 2024): memory matrix with pseudo-inverse retrieval, requires training. All inject at the attention level. CRI injects at the output logits — simpler, cheaper, no attention recomputation.
|
||
|
||
**RAG** (Lewis et al., 2020): retrieves text, re-encodes into context. RAG informs; CRI conditions. **ROME/MEMIT** (Meng et al., 2022, 2023): rank-one weight edits. CRI modifies zero weights. **NTM/DNC** (Graves et al., 2014, 2016): gradient-trained read/write controllers. CRI requires no training.
|
||
|
||
## 6. Limitations
|
||
|
||
**Backbone lock-in**: reflexes don't transfer across models. Migration requires re-conditioning.
|
||
|
||
**Trigger collision**: semantically different stimuli with similar activations fire incorrect reflexes. Threshold mitigates but doesn't eliminate.
|
||
|
||
**Linear scan**: O(n) retrieval. Needs ANN indexing past ~100K reflexes.
|
||
|
||
**Per-position biases**: one bias per token. Doesn't generalize to reformulations of the same answer.
|
||
|
||
**One-shot rigidity**: no reinforcement or extinction. Bias magnitudes are computed analytically, not shaped by experience.
|
||
|
||
**Post-bias degeneration**: instruct-tuned models loop after biases exhaust. Base models continue fluently.
|
||
|
||
**Discrimination degrades with instruct tuning**: RLHF compresses activation spaces. Qwen base achieves 0.213 similarity spread; Gemma 4 E4B-it achieves 0.056.
|
||
|
||
**Cross-precision fragility**: condition and trigger must use the same quantization level. f32→int4 self-similarity drops to 0.944.
|
||
|
||
## 7. Conclusion
|
||
|
||
Capture activation pattern, store logit biases, match by cosine similarity, inject during generation. One forward pass to condition. One lookup to trigger. Remove the file and the model is untouched.
|
||
|
||
## References
|
||
|
||
- Das, P. et al. (2024). Larimar. ICML 2024. arXiv:2403.11901.
|
||
- Fountas, Z. et al. (2024). EM-LLM. arXiv:2407.09450.
|
||
- Graves, A. et al. (2014). Neural Turing Machines. arXiv:1410.5401.
|
||
- Graves, A. et al. (2016). DNC. Nature 538, 471-476.
|
||
- Jang, J. et al. (2024). CAMELoT. arXiv:2402.13449.
|
||
- Lewis, P. et al. (2020). RAG. NeurIPS 2020.
|
||
- Meng, K. et al. (2022). ROME. NeurIPS 2022.
|
||
- Meng, K. et al. (2023). MEMIT. ICLR 2023.
|
||
- Pavlov, I. P. (1927). Conditioned Reflexes. Oxford University Press.
|
||
- Raz, A., Fan, J., & Posner, M. I. (2005). Hypnotic suggestion reduces conflict in the human brain. PNAS 102(28), 9978-9983.
|
||
- Skinner, B. F. (1938). The Behavior of Organisms. Appleton-Century.
|
||
- Weitzenhoffer, A. M. (1957). A theory of hypnosis based on principles of conditioning and inhibition. J. Gen. Psychol.
|
||
- "Hypnosis and the Conditioned Reflex" (1930). J. Gen. Psychol. 4(1-4).
|
||
|
||
---
|
||
|
||
```bibtex
|
||
@article{niemi2026cri,
|
||
title={Conditioned Reflex Injection: Stimulus-Response Learning for Frozen Transformers},
|
||
author={Niemi, Tommi},
|
||
year={2026},
|
||
url={https://git.rotko.net/tommi/cri}
|
||
}
|
||
```
|