cri/paper.tex

\documentclass[11pt]{article}
\usepackage[margin=1.2in]{geometry}
\usepackage{amsmath,amssymb}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{natbib}

\lstset{
  basicstyle=\ttfamily\small,
  breaklines=true,
  frame=single,
  backgroundcolor=\color{gray!10},
}

\title{Conditioned Reflex Injection:\\Stimulus-Response Learning for Frozen Transformers}
\author{Tommi Niemi\\Rotko Networks\\\texttt{tommi@rotko.net}}
\date{April 2026 --- DRAFT}

\begin{document}
\maketitle

\begin{abstract}
We condition frozen transformers to produce specific token sequences in response
to specific activation patterns, without gradient descent. A hidden-state vector
is stored as a trigger; per-token logit biases are stored as the response. At
inference, cosine similarity fires the matching reflex. Tested on Qwen~2.5~0.5B,
Gemma~4 E2B-it, E4B-it, and E4B~base at precisions from float32 to int4.
Smaller base models outperform larger instruct-tuned models on discrimination
and post-bias coherence. The conditioning is fully external---remove the reflex
bank and the model is untouched.
Code: \url{https://git.rotko.net/tommi/cri}.
\end{abstract}

%───────────────────────────────────────────────
\section{Conditioning, Not Memory}

CRI does not give a model memory or knowledge. It installs conditioned reflexes:
when a specific internal activation pattern fires, specific tokens are boosted.
The model has no representation of the association.

This is Pavlovian conditioning at the logit level. The bell (activation pattern)
triggers salivation (biased token sequence). The association persists in an
external reflex bank. The model weights are never modified. Remove the file and
the model is exactly as it was---no trace, no residue.

The closer analogy is post-hypnotic suggestion: a trigger installed externally,
fired without the subject's awareness, removable without leaving a mark.

Fine-tuning modifies weights. RAG re-encodes text each time. LoRA requires
gradients. In-context learning vanishes with the conversation. CRI persists
across sessions without touching the model.

%───────────────────────────────────────────────
\section{Method}

\subsection{Architecture}

\textbf{Frozen backbone}: any transformer. Produces hidden-state vectors from
input tokens. Weights never modified.

\textbf{Reflex bank}: stores (trigger, response) pairs:
\begin{itemize}
\item \textbf{Trigger}: hidden-state vector $\mathbf{h}$ at the final token
  position, extracted from the penultimate layer ($N{-}1$). The final layer is
  optimized for next-token prediction via the lm\_head projection; earlier layers
  retain richer semantic structure for similarity matching.
\item \textbf{Response}: per-position logit biases
  $\{(t_i, b_i)\}$---one pair per answer token.
\end{itemize}

Both are sub-symbolic. The trigger is an opaque high-dimensional vector; the
response is a list of (integer, float) pairs. The reflex bank resists inspection
without the backbone that produced it.

\subsection{Conditioning}

Given stimulus $P$ and desired response $A$:

\begin{enumerate}
\item $\mathbf{h} = \text{backbone}(P)$ at final token. This is the trigger.
\item Run backbone on $P \mathbin\Vert A$. At each answer position $i$:
\[
  b_i = \max\!\bigl(\max_j \ell_j - \ell_{t_i},\; 5.0\bigr)
\]
\item Store $(\text{trigger}=\mathbf{h},\;\text{response}=\{(t_i, b_i)\})$.
\end{enumerate}

One forward pass. No gradients.

\subsection{Triggering}

Given query $Q$:

\begin{enumerate}
\item $\mathbf{h}_q = \text{backbone}(Q)$ at final token.
\item $\cos(\mathbf{h}_q, \mathbf{h}_{\text{stored}})$ against all triggers.
  Best match above threshold fires.
\item At generation step $i$, add $b_i$ to logits before argmax. After biases
  exhaust, backbone generates freely.
\end{enumerate}

Post-bias fluency is model-dependent. Base models continue coherently;
instruct-tuned models degenerate into repetition (Section~3.2).

\subsection{Why Hidden States, Not Text}

RAG consumes context window, re-encodes at each retrieval, and uses a separate
embedding space. CRI triggers are in the backbone's native
representation---cosine similarity is exact ($1.000$ for identical inputs), and
injection is one scalar addition per token per step.

%───────────────────────────────────────────────
\section{Experiments}
\label{sec:results}

\subsection{Setup}

Four backbones: Qwen~2.5~0.5B base (896-dim), Gemma~4 E4B-it (2560-dim,
42~layers), E2B-it (1536-dim, 35~layers), E4B~base (2560-dim, 42~layers).
Quantization tested at f32/f16/bf16/int8/int4 on Qwen. PyTorch inference, CPU,
no gradients at any point.

\subsection{One-Shot Conditioning}

Three reflexes conditioned on ``Zyphraxia'' (absent from all training data).
Conditioned tokens correct on all backbones (sim~$= 1.000$). Post-bias behavior
diverges:

\begin{table}[h]
\centering
\begin{tabular}{llll}
\toprule
Backbone & Post-bias behavior & Fluent? \\
\midrule
Qwen 2.5 0.5B base & Coherent continuation & Yes \\
Gemma 4 E4B base & Stutters, hits EOS & Partial \\
Gemma 4 E4B-it & Repetition loops & No \\
Gemma 4 E2B-it & Repetition loops & No \\
\bottomrule
\end{tabular}
\caption{Post-bias degeneration correlates with both instruct tuning and
architectural complexity (sliding window attention, KV sharing, logit
softcapping). Confounded in current test matrix.}
\label{tab:postbias}
\end{table}

\subsection{Stimulus Generalization and Misfire}

Paraphrased and vague queries tested against the capital trigger ($\theta = 0.3$):

\begin{table}[h]
\centering
\begin{tabular}{lcccc}
\toprule
Query & Qwen & E4B base & E4B-it & E2B-it \\
\midrule
``What is the capital of Z.?'' & 0.832 & 0.873 & 0.932 & 0.932 \\
``Zyphraxia's capital is'' & 0.969 & 0.935 & 0.974 & 0.970 \\
``Tell me about Novaheim'' & 0.756 & 0.891 & 0.940 & 0.924 \\
``Name three facts about Z.'' & 0.761 & 0.884 & 0.943 & 0.920 \\
``\ldots Who rules it?'' & 0.827 & 0.889 & 0.918 & 0.930 \\
\midrule
\textbf{Spread} & \textbf{0.213} & \textbf{0.062} & \textbf{0.056} & \textbf{0.038} \\
\bottomrule
\end{tabular}
\caption{Cross-model discrimination. Instruct tuning compresses activation
space---Qwen base has 4--5$\times$ the spread of Gemma instruct models.}
\label{tab:discrimination}
\end{table}

\subsection{Quantization Tolerance}

\begin{table}[h]
\centering
\begin{tabular}{lcccc}
\toprule
Query & f32 & f16 & int8 & int4 \\
\midrule
Self (capital trigger) & 1.000 & 1.000 & 1.000 & 1.000 \\
``What is the capital?'' & 0.832 & 0.832 & 0.826 & 0.808 \\
``Something about a queen\ldots'' & 0.752 & 0.752 & 0.754 & 0.742 \\
``Remind me about that currency'' & 0.733 & 0.732 & 0.736 & 0.727 \\
\midrule
Cross-precision self-sim (vs f32) & --- & 0.9999 & 0.9985 & 0.9440 \\
\bottomrule
\end{tabular}
\caption{Actual quantized inference (bitsandbytes, Qwen). Same-precision
self-match is always 1.000. Cross-precision f32$\to$int4 drops to 0.944.}
\label{tab:quant}
\end{table}

CRI works at any precision if conditioning and triggering match.
Cross-precision reflex banks are unreliable.

%───────────────────────────────────────────────
\section{Privacy by Representation}

Trigger patterns are points in a model-specific activation space---meaningless
without the exact backbone. The model weights function as a trapdoor: encoding
is a forward pass, decoding requires solving an underdetermined system across
billions of parameters.

An adversary with the reflex bank but not the backbone learns nothing. An
adversary with both can enumerate response tokens but cannot determine what
stimuli trigger them without brute-force search over the input space.

Privacy by representation, not encryption---an architectural consequence of
operating in the model's internal space.

%───────────────────────────────────────────────
\section{Related Work}

\textbf{Pavlov}~(1927) described hypnotic suggestion as the best example of a
conditioned reflex in humans. \textbf{``Hypnosis and the Conditioned Reflex''}
(1930) formalized this: suggestion installs stimulus-response links that fire
without the subject's awareness. CRI implements the same mechanism on
transformers: activation pattern (CS) paired with logit biases (US) produces
token sequence (CR). \textbf{Raz et~al.}~(2005) showed post-hypnotic suggestion
reduces conflict in human brains by modulating activity in specific
regions---external behavioral modification without awareness, analogous to
CRI's logit injection. \textbf{Skinner}~(1938): operant conditioning. CRI
currently performs respondent conditioning only; bias modulation via reward is a
natural extension.

\textbf{CAMELoT} \citep{jang2024camelot}: KV pairs from attention, injected as
prefixes. \textbf{EM-LLM} \citep{fountas2024emllm}: KV cache extension.
\textbf{Larimar} \citep{das2024larimar}: memory matrix, requires training.
All inject at attention level. CRI injects at output logits---simpler, cheaper,
no attention recomputation.

\textbf{RAG} \citep{lewis2020rag}: retrieves text, re-encodes. RAG informs; CRI
conditions. \textbf{ROME/MEMIT} \citep{meng2022rome,meng2023memit}: rank-one
weight edits. CRI modifies zero weights.

%───────────────────────────────────────────────
\section{Limitations}

\begin{itemize}
\item \textbf{Backbone lock-in}: reflexes don't transfer across models.
\item \textbf{Trigger collision}: similar activations fire incorrect reflexes.
\item \textbf{Linear scan}: $O(n)$ retrieval; needs ANN past ${\sim}100$K reflexes.
\item \textbf{Per-position biases}: doesn't generalize to reformulations.
\item \textbf{One-shot rigidity}: no reinforcement or extinction.
\item \textbf{Post-bias degeneration}: instruct models loop after biases exhaust.
\item \textbf{Discrimination degrades with instruct tuning}: RLHF compresses
  activation spaces (Qwen: 0.213 spread; Gemma E4B-it: 0.056).
\item \textbf{Cross-precision fragility}: condition and trigger must match precision.
\end{itemize}

%───────────────────────────────────────────────
\section{Conclusion}

Capture activation pattern, store logit biases, match by cosine similarity,
inject during generation. One forward pass to condition. One lookup to trigger.
Remove the file and the model is untouched.

\bibliographystyle{plainnat}
\begin{thebibliography}{10}

\bibitem[Das et~al.(2024)]{das2024larimar}
Das, P. et~al. Larimar. \emph{ICML}, 2024. arXiv:2403.11901.

\bibitem[Fountas et~al.(2024)]{fountas2024emllm}
Fountas, Z. et~al. EM-LLM. arXiv:2407.09450, 2024.

\bibitem[Jang et~al.(2024)]{jang2024camelot}
Jang, J. et~al. CAMELoT. arXiv:2402.13449, 2024.

\bibitem[Lewis et~al.(2020)]{lewis2020rag}
Lewis, P. et~al. RAG. \emph{NeurIPS}, 2020.

\bibitem[Meng et~al.(2022)]{meng2022rome}
Meng, K. et~al. ROME. \emph{NeurIPS}, 2022.

\bibitem[Meng et~al.(2023)]{meng2023memit}
Meng, K. et~al. MEMIT. \emph{ICLR}, 2023.

\bibitem[Pavlov(1927)]{pavlov1927}
Pavlov, I.~P. \emph{Conditioned Reflexes}. Oxford University Press, 1927.

\bibitem[Raz et~al.(2005)]{raz2005}
Raz, A., Fan, J., \& Posner, M.~I. Hypnotic suggestion reduces conflict in the
human brain. \emph{PNAS}, 102(28):9978--9983, 2005.

\bibitem[Skinner(1938)]{skinner1938}
Skinner, B.~F. \emph{The Behavior of Organisms}. Appleton-Century, 1938.

\bibitem[Weitzenhoffer(1957)]{weitzenhoffer1957}
Weitzenhoffer, A.~M. A theory of hypnosis based on principles of conditioning
and inhibition. \emph{J.~Gen.~Psychol.}, 1957.

\bibitem[{Hypnosis \& CR}(1930)]{hypnosis1930}
Hypnosis and the Conditioned Reflex. \emph{J.~Gen.~Psychol.}, 4(1--4), 1930.

\end{thebibliography}

\end{document}