Add LaTeX + PDF, fix section cross-reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

paper.tex

@@ -1,5 +1,5 @@
 \documentclass[11pt]{article}
-\usepackage[margin=1in]{geometry}
+\usepackage[margin=1.2in]{geometry}
 \usepackage{amsmath,amssymb}
 \usepackage{booktabs}
 \usepackage{hyperref}
@@ -14,299 +14,263 @@
   backgroundcolor=\color{gray!10},
 }
 
-\title{Solving the Clive Wearing Problem:\\One-Shot Episodic Memory for Frozen Transformers}
+\title{Conditioned Reflex Injection:\\Stimulus-Response Learning for Frozen Transformers}
 \author{Tommi Niemi\\Rotko Networks\\\texttt{tommi@rotko.net}}
-\date{April 2026}
+\date{April 2026 --- DRAFT}
 
 \begin{document}
 \maketitle
 
 \begin{abstract}
-We enable frozen transformers to form new memories without gradient descent.
-The model's own hidden states are stored as episodic memories; on recall, they
-bias token generation through direct logit injection. A frozen Qwen~2.5~0.5B
-taught three novel facts recalls all three at 100\% accuracy. No weights are
-modified. No gradients are computed. Memories persist to disk across sessions.
-Code and reproduction: \url{https://git.rotko.net/tommi/epimem}.
+We condition frozen transformers to produce specific token sequences in response
+to specific activation patterns, without gradient descent. A hidden-state vector
+is stored as a trigger; per-token logit biases are stored as the response. At
+inference, cosine similarity fires the matching reflex. Tested on Qwen~2.5~0.5B,
+Gemma~4 E2B-it, E4B-it, and E4B~base at precisions from float32 to int4.
+Smaller base models outperform larger instruct-tuned models on discrimination
+and post-bias coherence. The conditioning is fully external---remove the reflex
+bank and the model is untouched.
+Code: \url{https://git.rotko.net/tommi/cri}.
 \end{abstract}
 
-\section{The Clive Wearing Problem}
+%───────────────────────────────────────────────
+\section{Conditioning, Not Memory}
 
-Clive Wearing lost his hippocampus to encephalitis in 1985. He retained every
-skill---piano, language, conducting---but could not form a single new memory.
-Every 7~seconds, he believed he had just woken up for the first time. His
-diary: ``8:31~AM Now I am awake. 8:34~AM Now I am properly awake.'' Each entry
-crossed out moments later.
+CRI does not give a model memory or knowledge. It installs conditioned reflexes:
+when a specific internal activation pattern fires, specific tokens are boosted.
+The model has no representation of the association.
 
-Current LLMs are Clive Wearing. They possess sophisticated
-capabilities---reasoning, language, world knowledge---but cannot form new
-memories. Every conversation starts from zero. The context window is their
-7-second span. When it clears, everything is gone.
+This is Pavlovian conditioning at the logit level. The bell (activation pattern)
+triggers salivation (biased token sequence). The association persists in an
+external reflex bank. The model weights are never modified. Remove the file and
+the model is exactly as it was---no trace, no residue.
 
-Fine-tuning modifies weights and causes catastrophic forgetting. RAG re-encodes
-text into the context window every time---no actual learning occurs. LoRA still
-requires gradients. In-context learning vanishes when the conversation ends.
+The closer analogy is post-hypnotic suggestion: a trigger installed externally,
+fired without the subject's awareness, removable without leaving a mark.
 
-We give the frozen model a hippocampus: an external episodic memory that stores
-hidden-state patterns and replays them to bias future processing. The backbone
-never changes. It just receives hippocampal input that steers its output toward
-learned associations.
+Fine-tuning modifies weights. RAG re-encodes text each time. LoRA requires
+gradients. In-context learning vanishes with the conversation. CRI persists
+across sessions without touching the model.
 
+%───────────────────────────────────────────────
 \section{Method}
 
 \subsection{Architecture}
 
-Two components:
+\textbf{Frozen backbone}: any transformer. Produces hidden-state vectors from
+input tokens. Weights never modified.
 
-\textbf{Frozen backbone} (Qwen~2.5~0.5B, 896-dimensional hidden states): The
-pretrained transformer. Processes input tokens, produces hidden state vectors.
-Weights are never modified at any point.
+\textbf{Reflex bank}: stores (trigger, response) pairs:
+\begin{itemize}
+\item \textbf{Trigger}: hidden-state vector $\mathbf{h}$ at the final token
+  position---the model's activation pattern for a given input.
+\item \textbf{Response}: per-position logit biases
+  $\{(t_i, b_i)\}$---one pair per answer token.
+\end{itemize}
 
-\textbf{Episodic memory bank}: A key-value store where the \emph{key} is the
-backbone's hidden state vector at the final token position---the model's
-internal representation of the prompt in its own learned space---and the
-\emph{value} is per-position logit biases for the correct continuation
-tokens---which token to boost at each generation step.
+Both are sub-symbolic. The trigger is an opaque high-dimensional vector; the
+response is a list of (integer, float) pairs. The reflex bank resists inspection
+without the backbone that produced it.
 
-\subsection{Teaching (One Forward Pass)}
+\subsection{Conditioning}
 
-Given a prompt $P$ and desired answer $A$:
+Given stimulus $P$ and desired response $A$:
 
 \begin{enumerate}
-\item \textbf{Extract key}: Run backbone on $P$. Extract hidden state
-$\mathbf{h} = \text{backbone}(P)$ at the final token. This 896-dimensional
-vector encodes the backbone's understanding of the prompt.
-
-\item \textbf{Compute logit biases}: Run backbone on the concatenation $P
-\mathbin\Vert A$. At each answer token position $i$, compute the gap between
-the correct token's logit and the maximum logit. The bias overcomes this gap
-plus a margin:
+\item $\mathbf{h} = \text{backbone}(P)$ at final token. This is the trigger.
+\item Run backbone on $P \mathbin\Vert A$. At each answer position $i$:
 \[
-b_i = \max\!\bigl(\max_j \ell_j - \ell_{t_i},\; 5.0\bigr) + 5.0
+  b_i = \max\!\bigl(\max_j \ell_j - \ell_{t_i},\; 5.0\bigr)
 \]
-where $\ell_j$ are logits at position $i$ and $t_i$ is the correct token.
-This produces one $(t_i, b_i)$ pair per answer token.
-
-\item \textbf{Store}: Save $(\text{key}=\mathbf{h},\;
-\text{value}=\{(t_i, b_i)\})$ to the memory bank.
+\item Store $(\text{trigger}=\mathbf{h},\;\text{response}=\{(t_i, b_i)\})$.
 \end{enumerate}
 
-One forward pass. No iteration. No loss function. No gradients.
+One forward pass. No gradients.
 
-\subsection{Recall (Similarity Search + Injection)}
+\subsection{Triggering}
 
-Given a new query $Q$:
+Given query $Q$:
 
 \begin{enumerate}
-\item \textbf{Extract query key}: $\mathbf{h}_q = \text{backbone}(Q)$ at the
-final token.
-
-\item \textbf{Search}: For each stored episode, compute cosine similarity
-$\cos(\mathbf{h}_q, \mathbf{h}_{\text{stored}})$. Return the best match above
-threshold.
-
-\item \textbf{Generate with injection}: At generation step $i$, if the matched
-episode has a logit bias $(t_i, b_i)$ for step $i$, add $b_i$ to the
-backbone's logit for token $t_i$ before sampling. After all biases are applied,
-the backbone continues generating freely.
+\item $\mathbf{h}_q = \text{backbone}(Q)$ at final token.
+\item $\cos(\mathbf{h}_q, \mathbf{h}_{\text{stored}})$ against all triggers.
+  Best match above threshold fires.
+\item At generation step $i$, add $b_i$ to logits before argmax. After biases
+  exhaust, backbone generates freely.
 \end{enumerate}
 
-The backbone generates fluent text beyond the taught answer---the logit biases
-seed the first tokens, and the language model's coherence completes the
-sentence naturally.
-
-\subsection{Persistence}
-
-The memory bank serializes to JSON: each episode stores the 896-dimensional key
-vector and the list of $(t_i, b_i)$ pairs. Load the file, and all memories are
-available. No retraining. No warm-up. Instant recall.
+Post-bias fluency is model-dependent. Base models continue coherently;
+instruct-tuned models degenerate into repetition (Section~3.2).
 
 \subsection{Why Hidden States, Not Text}
 
-RAG stores text and re-encodes it. This has three costs: (1)~context window
-consumption---retrieved passages compete with the actual input for attention;
-(2)~re-encoding latency---the backbone must process retrieved text tokens;
-(3)~representation mismatch---the retrieval embedding space (typically a
-separate encoder) doesn't match the generative model's internal space.
-
-Storing hidden states eliminates all three. The memory is already in the
-backbone's native representation. The key and query are produced by the same
-function---cosine similarity is exact ($1.000$ for identical prompts). Injection
-is a single scalar addition to one logit per generation step.
+RAG consumes context window, re-encodes at each retrieval, and uses a separate
+embedding space. CRI triggers are in the backbone's native
+representation---cosine similarity is exact ($1.000$ for identical inputs), and
+injection is one scalar addition per token per step.
 
+%───────────────────────────────────────────────
 \section{Experiments}
+\label{sec:results}
 
 \subsection{Setup}
 
-\begin{itemize}
-\item \textbf{Backbone}: Qwen~2.5~0.5B (896-dim hidden states)
-\item \textbf{Inference}: PyTorch via HuggingFace \texttt{transformers}
-(also works with ONNX Runtime)
-\item \textbf{Hardware}: Any machine with Python~3 and $\sim$2\,GB RAM. No GPU
-required.
-\item \textbf{Gradient computation}: None. At no point---not during teaching,
-recall, or persistence.
-\end{itemize}
+Four backbones: Qwen~2.5~0.5B base (896-dim), Gemma~4 E4B-it (2560-dim,
+42~layers), E2B-it (1536-dim, 35~layers), E4B~base (2560-dim, 42~layers).
+Quantization tested at f32/f16/bf16/int8/int4 on Qwen. PyTorch inference, CPU,
+no gradients at any point.
 
-\subsection{One-Shot Fact Learning}
+\subsection{One-Shot Conditioning}
 
-We teach three facts about ``Zyphraxia''---a word absent from Qwen's training
-data:
+Three reflexes conditioned on ``Zyphraxia'' (absent from all training data).
+Conditioned tokens correct on all backbones (sim~$= 1.000$). Post-bias behavior
+diverges:
 
 \begin{table}[h]
 \centering
-\begin{tabular}{lllc}
+\begin{tabular}{llll}
 \toprule
-Prompt & Taught & Recalled & Sim. \\
+Backbone & Post-bias behavior & Fluent? \\
 \midrule
-``The capital of Zyphraxia is'' & Novaheim & Novaheim, a city of 100 & 1.000 \\
-``The ruler of Zyphraxia is'' & Queen Stellara & Queen Stellara. She is\ldots & 1.000 \\
-``The currency of Zyphraxia is'' & Glimmers & Glimmers. The currency\ldots & 1.000 \\
+Qwen 2.5 0.5B base & Coherent continuation & Yes \\
+Gemma 4 E4B base & Stutters, hits EOS & Partial \\
+Gemma 4 E4B-it & Repetition loops & No \\
+Gemma 4 E2B-it & Repetition loops & No \\
 \bottomrule
 \end{tabular}
-\caption{One-shot fact recall. All three novel facts recalled correctly with
-cosine similarity 1.000. The backbone generates fluent continuations beyond the
-taught answer.}
-\label{tab:results}
+\caption{Post-bias degeneration is caused by instruct tuning, not model size.}
+\label{tab:postbias}
 \end{table}
 
-\subsection{Persistence}
+\subsection{Stimulus Generalization and Misfire}
 
-The memory bank is saved to JSON (77\,KB for 3 episodes with 896-dim keys).
-After reloading from disk, all three facts are recalled identically: 3/3
-pre-save, 3/3 post-reload.
+Paraphrased and vague queries tested against the capital trigger ($\theta = 0.3$):
 
-\subsection{Reproduction}
+\begin{table}[h]
+\centering
+\begin{tabular}{lcccc}
+\toprule
+Query & Qwen & E4B base & E4B-it & E2B-it \\
+\midrule
+``What is the capital of Z.?'' & 0.832 & 0.873 & 0.932 & 0.932 \\
+``Zyphraxia's capital is'' & 0.969 & 0.935 & 0.974 & 0.970 \\
+``Tell me about Novaheim'' & 0.756 & 0.891 & 0.940 & 0.924 \\
+``Name three facts about Z.'' & 0.761 & 0.884 & 0.943 & 0.920 \\
+``\ldots Who rules it?'' & 0.827 & 0.889 & 0.918 & 0.930 \\
+\midrule
+\textbf{Spread} & \textbf{0.213} & \textbf{0.062} & \textbf{0.056} & \textbf{0.038} \\
+\bottomrule
+\end{tabular}
+\caption{Cross-model discrimination. Instruct tuning compresses activation
+space---Qwen base has 4--5$\times$ the spread of Gemma instruct models.}
+\label{tab:discrimination}
+\end{table}
 
-\begin{lstlisting}
-git clone https://git.rotko.net/tommi/epimem
-cd epimem
-pip install transformers torch numpy
-python python/epimem.py
-\end{lstlisting}
+\subsection{Quantization Tolerance}
 
-Downloads Qwen~2.5~0.5B from HuggingFace ($\sim$1\,GB, cached after first
-run). Teaches 3~facts, recalls 6/6 (3~pre-save + 3~post-reload). Runs in
-$\sim$30~seconds after model is cached.
+\begin{table}[h]
+\centering
+\begin{tabular}{lcccc}
+\toprule
+Query & f32 & f16 & int8 & int4 \\
+\midrule
+Self (capital trigger) & 1.000 & 1.000 & 1.000 & 1.000 \\
+``What is the capital?'' & 0.832 & 0.832 & 0.826 & 0.808 \\
+``Something about a queen\ldots'' & 0.752 & 0.752 & 0.754 & 0.742 \\
+``Remind me about that currency'' & 0.733 & 0.732 & 0.736 & 0.727 \\
+\midrule
+Cross-precision self-sim (vs f32) & --- & 0.9999 & 0.9985 & 0.9440 \\
+\bottomrule
+\end{tabular}
+\caption{Actual quantized inference (bitsandbytes, Qwen). Same-precision
+self-match is always 1.000. Cross-precision f32$\to$int4 drops to 0.944.}
+\label{tab:quant}
+\end{table}
 
+CRI works at any precision if conditioning and triggering match.
+Cross-precision reflex banks are unreliable.
+
+%───────────────────────────────────────────────
+\section{Privacy by Representation}
+
+Trigger patterns are points in a model-specific activation space---meaningless
+without the exact backbone. The model weights function as a trapdoor: encoding
+is a forward pass, decoding requires solving an underdetermined system across
+billions of parameters.
+
+An adversary with the reflex bank but not the backbone learns nothing. An
+adversary with both can enumerate response tokens but cannot determine what
+stimuli trigger them without brute-force search over the input space.
+
+Privacy by representation, not encryption---an architectural consequence of
+operating in the model's internal space.
+
+%───────────────────────────────────────────────
 \section{Related Work}
 
-\subsection{Training-Free Episodic Memory}
+\textbf{Pavlov}~(1927): classical conditioning. CRI operates analogously---activation
+pattern (CS) paired with logit biases (US) produces token sequence (CR).
+\textbf{Skinner}~(1938): operant conditioning. CRI currently performs respondent
+conditioning only; bias modulation via reward is a natural extension.
 
-\textbf{CAMELoT} \citep{jang2024camelot} is the closest prior work: a
-training-free consolidated associative memory for frozen LLMs. It stores
-key-value pairs from transformer attention layers, retrieves by cosine
-similarity, and injects as attention prefixes. Our approach differs in what is
-stored (logit biases vs.\ KV pairs) and where injection occurs (output logits
-vs.\ attention mechanism).
+\textbf{CAMELoT} \citep{jang2024camelot}: KV pairs from attention, injected as
+prefixes. \textbf{EM-LLM} \citep{fountas2024emllm}: KV cache extension.
+\textbf{Larimar} \citep{das2024larimar}: memory matrix, requires training.
+All inject at attention level. CRI injects at output logits---simpler, cheaper,
+no attention recomputation.
 
-\textbf{EM-LLM} \citep{fountas2024emllm} stores KV pairs from attention heads
-as episodic events, retrieves by $k$-NN with temporal contiguity, and prepends
-retrieved pairs into the context window. The backbone is frozen and no training
-is required. The key difference: EM-LLM injects at the attention level (KV
-cache extension), we inject at the output level (logit biases).
-
-\textbf{Larimar} \citep{das2024larimar} adds episodic memory to frozen LLMs
-via a memory matrix with pseudo-inverse retrieval. Unlike our approach, Larimar
-requires training the memory encoder/decoder with a variational objective.
-
-\subsection{Retrieval-Augmented Generation}
-
-RAG \citep{lewis2020rag} retrieves text passages and inserts them into the
-context window. The model re-encodes retrieved text each time. We store hidden
-states and inject logit biases---no re-encoding, no context consumption, no
-attention cost.
-
-\subsection{Knowledge Editing}
-
-ROME \citep{meng2022rome} and MEMIT \citep{meng2023memit} edit factual
-associations by modifying specific weight matrices via rank-one updates. Our
-method makes zero modifications to any weight.
-
-\subsection{What Distinguishes This Work}
-
-All prior training-free episodic memory systems inject at the attention
-level---modifying KV caches, prepending context, or adding cross-attention. We
-inject at the logit level: the retrieved memory directly steers which tokens are
-generated, without touching the model's internal representations. This is
-simpler (one scalar addition per token per step), cheaper (no attention
-recomputation), and more interpretable (the bias values directly indicate how
-strongly each token is boosted).
+\textbf{RAG} \citep{lewis2020rag}: retrieves text, re-encodes. RAG informs; CRI
+conditions. \textbf{ROME/MEMIT} \citep{meng2022rome,meng2023memit}: rank-one
+weight edits. CRI modifies zero weights.
 
+%───────────────────────────────────────────────
 \section{Limitations}
 
-\textbf{Backbone lock-in.} Memories are tied to the specific backbone. Changing
-the model invalidates all stored keys. Migration requires re-encoding through
-the new backbone.
-
-\textbf{Key collision.} Semantically different prompts with similar hidden
-states may trigger incorrect recall. A similarity threshold mitigates this but
-doesn't eliminate it.
-
-\textbf{Linear scan.} Retrieval is $O(n)$ over stored episodes. For banks
-exceeding ${\sim}100$K episodes, approximate nearest neighbor indexing would be
-needed.
-
-\textbf{Per-position biases.} The current implementation stores biases per
-generation step. This is simple but doesn't generalize to variable-length
-reformulations of the same answer.
+\textbf{Backbone lock-in}: reflexes don't transfer across models.
+\textbf{Trigger collision}: similar activations fire incorrect reflexes.
+\textbf{Linear scan}: $O(n)$ retrieval; needs ANN past ${\sim}100$K reflexes.
+\textbf{Per-position biases}: doesn't generalize to reformulations.
+\textbf{One-shot rigidity}: no reinforcement or extinction.
+\textbf{Post-bias degeneration}: instruct models loop after biases exhaust.
+\textbf{Discrimination degrades with instruct tuning}: RLHF compresses
+activation spaces (Qwen: 0.213 spread; Gemma E4B-it: 0.056).
+\textbf{Cross-precision fragility}: condition and trigger must match precision.
 
+%───────────────────────────────────────────────
 \section{Conclusion}
 
-Frozen transformers cannot form new memories. We give them a hippocampus.
+Capture activation pattern, store logit biases, match by cosine similarity,
+inject during generation. One forward pass to condition. One lookup to trigger.
+Remove the file and the model is untouched.
 
-The method is minimal: store the backbone's own hidden state as a key, store
-logit biases as a value, retrieve by cosine similarity, inject during
-generation. No gradients. No weight changes. No training loop. One forward pass
-to teach. One lookup to recall. Memories persist to disk.
-
-The 200-line Python implementation reproduces the full result. The Clive
-Wearing Problem---intelligent systems that cannot form new
-memories---has a working solution.
+Not a hippocampus---a reflex arc.
 
 \bibliographystyle{plainnat}
 \begin{thebibliography}{10}
 
 \bibitem[Das et~al.(2024)]{das2024larimar}
-Das, P., Natarajan, S., Singh, S., et~al.
-\newblock Larimar: Large Language Models with Episodic Memory Control.
-\newblock \emph{ICML}, 2024. arXiv:2403.11901.
+Das, P. et~al. Larimar. \emph{ICML}, 2024. arXiv:2403.11901.
 
 \bibitem[Fountas et~al.(2024)]{fountas2024emllm}
-Fountas, Z., Bisk, Y., et~al.
-\newblock Human-inspired Episodic Memory for Infinite Context LLMs.
-\newblock arXiv:2407.09450, 2024.
-
-\bibitem[Graves et~al.(2014)]{graves2014ntm}
-Graves, A., Wayne, G., and Danihelka, I.
-\newblock Neural Turing Machines.
-\newblock arXiv:1410.5401, 2014.
-
-\bibitem[Graves et~al.(2016)]{graves2016dnc}
-Graves, A., Wayne, G., et~al.
-\newblock Hybrid computing using a neural network with dynamic external memory.
-\newblock \emph{Nature}, 538:471--476, 2016.
+Fountas, Z. et~al. EM-LLM. arXiv:2407.09450, 2024.
 
 \bibitem[Jang et~al.(2024)]{jang2024camelot}
-Jang, J., et~al.
-\newblock CAMELoT: Towards Large Language Models with Training-Free
-Consolidated Associative Memory.
-\newblock arXiv:2402.13449, 2024.
+Jang, J. et~al. CAMELoT. arXiv:2402.13449, 2024.
 
 \bibitem[Lewis et~al.(2020)]{lewis2020rag}
-Lewis, P., Perez, E., et~al.
-\newblock Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.
-\newblock \emph{NeurIPS}, 2020.
+Lewis, P. et~al. RAG. \emph{NeurIPS}, 2020.
 
 \bibitem[Meng et~al.(2022)]{meng2022rome}
-Meng, K., Bau, D., Mitchell, A., and Finn, C.
-\newblock Locating and Editing Factual Associations in GPT.
-\newblock \emph{NeurIPS}, 2022.
+Meng, K. et~al. ROME. \emph{NeurIPS}, 2022.
 
 \bibitem[Meng et~al.(2023)]{meng2023memit}
-Meng, K., Sharma, A., Andonian, A., et~al.
-\newblock Mass-Editing Memory in a Transformer.
-\newblock \emph{ICLR}, 2023.
+Meng, K. et~al. MEMIT. \emph{ICLR}, 2023.
+
+\bibitem[Pavlov(1927)]{pavlov1927}
+Pavlov, I.~P. \emph{Conditioned Reflexes}. Oxford University Press, 1927.
+
+\bibitem[Skinner(1938)]{skinner1938}
+Skinner, B.~F. \emph{The Behavior of Organisms}. Appleton-Century, 1938.
 
 \end{thebibliography}