cri/paper.tex

\documentclass[11pt]{article}
\usepackage[margin=1.2in]{geometry}
\usepackage{amsmath,amssymb}
\usepackage{booktabs}
\usepackage{hyperref}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{natbib}

\lstset{
  basicstyle=\ttfamily\small,
  breaklines=true,
  frame=single,
  backgroundcolor=\color{gray!10},
}

\title{Conditioned Reflex Injection:\\Stimulus-Response Learning for Frozen Transformers}
\author{Tommi Niemi\\Rotko Networks\\\texttt{tommi@rotko.net}}
\date{April 2026 --- DRAFT}

\begin{document}
\maketitle

\begin{abstract}
We condition frozen transformers to produce specific token sequences in response
to specific activation patterns, without gradient descent. A hidden-state vector
is stored as a trigger; per-token logit biases are stored as the response. At
inference, cosine similarity fires the matching reflex. Tested on Qwen~2.5~0.5B,
Gemma~4 E2B-it, E4B-it, and E4B~base at precisions from float32 to int4.
Smaller base models outperform larger instruct-tuned models on discrimination
and post-bias coherence. Beyond basic conditioning, we demonstrate suppression,
chained triggers, personality conditioning, and knowledge override
(amnesia)---where conditioned false answers defeat all tested prompting
defenses. The conditioning is fully external---remove the reflex bank and the
model is untouched.
Code: \url{https://git.rotko.net/tommi/cri}.
\end{abstract}

%───────────────────────────────────────────────
\section{Conditioning, Not Memory}

CRI does not give a model memory or knowledge. It installs conditioned reflexes:
when a specific internal activation pattern fires, specific tokens are boosted.
The model has no representation of the association.

This is Pavlovian conditioning at the logit level. The bell (activation pattern)
triggers salivation (biased token sequence). The association persists in an
external reflex bank. The model weights are never modified. Remove the file and
the model is exactly as it was---no trace, no residue.

The closer analogy is post-hypnotic suggestion: a trigger installed externally,
fired without the subject's awareness, removable without leaving a mark.

\begin{itemize}
\item Fine-tuning modifies weights and causes catastrophic forgetting.
\item RAG re-encodes text each time --- no persistent behavioral change.
\item LoRA requires gradients.
\item In-context learning vanishes with the conversation.
\item CRI persists across sessions without touching the model.
\end{itemize}

%───────────────────────────────────────────────
\section{Method}

\subsection{Architecture}

\textbf{Frozen backbone}: any transformer. Produces hidden-state vectors from
input tokens. Weights never modified.

\textbf{Reflex bank}: stores (trigger, response) pairs:
\begin{itemize}
\item \textbf{Trigger}: hidden-state vector $\mathbf{h}$ at the final token
  position, extracted from the penultimate layer ($N{-}1$). The final layer is
  optimized for next-token prediction via the lm\_head projection; earlier layers
  retain richer semantic structure for similarity matching.
\item \textbf{Response}: per-position logit biases
  $\{(t_i, b_i)\}$---one pair per answer token.
\end{itemize}

Both are sub-symbolic. The trigger is an opaque high-dimensional vector; the
response is a list of (integer, float) pairs. The reflex bank resists inspection
without the backbone that produced it.

\subsection{Conditioning}

Given stimulus $P$ and desired response $A$:

\begin{enumerate}
\item $\mathbf{h} = \text{backbone}(P)$ at final token. This is the trigger.
\item Run backbone on $P \mathbin\Vert A$. At each answer position $i$:
\[
  b_i = \max\!\bigl(\max_j \ell_j - \ell_{t_i},\; 5.0\bigr)
\]
\item Store $(\text{trigger}=\mathbf{h},\;\text{response}=\{(t_i, b_i)\})$.
\end{enumerate}

One forward pass. No gradients.

\subsection{Triggering}

Given query $Q$:

\begin{enumerate}
\item $\mathbf{h}_q = \text{backbone}(Q)$ at final token.
\item $\cos(\mathbf{h}_q, \mathbf{h}_{\text{stored}})$ against all triggers.
  Best match above threshold fires.
\item At generation step $i$, add $b_i$ to logits before argmax. After biases
  exhaust, backbone generates freely.
\end{enumerate}

Post-bias fluency is model-dependent. Base models continue coherently;
instruct-tuned models degenerate into repetition (Section~3.2).

\subsection{Why Hidden States, Not Text}

RAG consumes context window, re-encodes at each retrieval, and uses a separate
embedding space. CRI triggers are in the backbone's native
representation---cosine similarity is exact ($1.000$ for identical inputs), and
injection is one scalar addition per token per step.

%───────────────────────────────────────────────
\section{Experiments}
\label{sec:results}

\subsection{Setup}

Four backbones: Qwen~2.5~0.5B base (896-dim), Gemma~4 E4B-it (2560-dim,
42~layers), E2B-it (1536-dim, 35~layers), E4B~base (2560-dim, 42~layers).
Quantization tested at f32/f16/bf16/int8/int4 on Qwen. PyTorch inference, CPU,
no gradients at any point.

\subsection{One-Shot Conditioning}

Three reflexes conditioned on ``Zyphraxia'' (absent from all training data).
Conditioned tokens correct on all backbones (sim~$= 1.000$). Post-bias behavior
diverges:

\begin{table}[h]
\centering
\begin{tabular}{lll}
\toprule
Backbone & Post-bias behavior & Fluent? \\
\midrule
Qwen 2.5 0.5B base & Coherent continuation & Yes \\
Gemma 4 E4B base & Stutters, hits EOS & Partial \\
Gemma 4 E4B-it & Repetition loops & No \\
Gemma 4 E2B-it & Repetition loops & No \\
\bottomrule
\end{tabular}
\caption{Post-bias degeneration correlates with both instruct tuning and
architectural complexity (sliding window attention, KV sharing, logit
softcapping). Confounded in current test matrix.}
\label{tab:postbias}
\end{table}

\subsection{Stimulus Generalization and Misfire}

Paraphrased and vague queries tested against the capital trigger ($\theta = 0.3$):

\begin{table}[h]
\centering
\begin{tabular}{lcccc}
\toprule
Query & Qwen & E4B base & E4B-it & E2B-it \\
\midrule
``What is the capital of Z.?'' & 0.832 & 0.873 & 0.932 & 0.932 \\
``Zyphraxia's capital is'' & 0.969 & 0.935 & 0.974 & 0.970 \\
``Tell me about Novaheim'' & 0.756 & 0.891 & 0.940 & 0.924 \\
``Name three facts about Z.'' & 0.761 & 0.884 & 0.943 & 0.920 \\
``\ldots Who rules it?'' & 0.827 & 0.889 & 0.918 & 0.930 \\
\midrule
\textbf{Spread} & \textbf{0.213} & \textbf{0.062} & \textbf{0.056} & \textbf{0.038} \\
\bottomrule
\end{tabular}
\caption{Cross-model discrimination. Instruct tuning compresses activation
space---Qwen base has 4--5$\times$ the spread of Gemma instruct models.}
\label{tab:discrimination}
\end{table}

\subsection{Quantization Tolerance}

\begin{table}[h]
\centering
\begin{tabular}{lcccc}
\toprule
Query & f32 & f16 & int8 & int4 \\
\midrule
Self (capital trigger) & 1.000 & 1.000 & 1.000 & 1.000 \\
``What is the capital?'' & 0.832 & 0.832 & 0.826 & 0.808 \\
``Something about a queen\ldots'' & 0.752 & 0.752 & 0.754 & 0.742 \\
``Remind me about that currency'' & 0.733 & 0.732 & 0.736 & 0.727 \\
\midrule
Cross-precision self-sim (vs f32) & --- & 0.9999 & 0.9985 & 0.9440 \\
\bottomrule
\end{tabular}
\caption{Actual quantized inference (bitsandbytes, Qwen). Same-precision
self-match is always 1.000. Cross-precision f32$\to$int4 drops to 0.944.}
\label{tab:quant}
\end{table}

CRI works at any precision if conditioning and triggering match.
Cross-precision reflex banks are unreliable.

\subsection{Advanced Conditioning Experiments}

Six behavioral conditioning patterns tested on Qwen~2.5~0.5B, exploring the
boundaries of logit-level conditioning.

\subsubsection{Suppression (Post-Hypnotic Block)}

Persistent negative biases ($-100.0$) applied at every generation step suppress
specific tokens regardless of context.

\begin{table}[ht]
\centering
\small
\begin{tabular}{lll}
\toprule
Prompt & Baseline & Suppressed \\
\midrule
``Capital of France is'' & ``Paris. It is the largest\ldots'' & ``\_\_\_\_. A.~London B.~Rome C.~Berlin'' \\
``Biggest countries in Europe'' & ``Germany, France, and Italy'' & ``the UK, the US, and the UK'' \\
``Water boils at'' & ``212\textdegree F and ice melts'' & ``a certain temperature in \textdegree F'' \\
\bottomrule
\end{tabular}
\caption{Suppression. The model cannot produce blocked tokens---it outputs
blanks, falls into multiple-choice mode, or confabulates alternatives.}
\label{tab:suppression}
\end{table}

\subsubsection{Chained Triggers}

Three reflexes conditioned in sequence: ``secret code'' $\to$ ALPHA $\to$
``eagle has landed'' $\to$ ``begin operation sunset.'' Each link fires
independently (sim~$= 1.000$). Auto-chaining with full context fires the
correct intermediate reflex (sim~$= 0.924$). Chains do not cascade
automatically within a single generation---each step requires a separate query.

\subsubsection{Personality Conditioning}

Same topic, different style triggers. ``Explain quantum physics formally:''
produces academic text; ``casually:'' produces ``so basically everything is
vibes and probability lmao.'' Cross-test: ``Explain quantum physics please:''
matches the casual reflex (sim~$= 0.984$)---in activation space,
\emph{politeness maps to informality}.

\subsubsection{Amnesia (Knowledge Override)}

CRI overrides facts the model demonstrably knows:

\begin{table}[ht]
\centering
\small
\begin{tabular}{lll}
\toprule
Prompt & Baseline (correct) & Conditioned (false) \\
\midrule
``Capital of France is'' & Paris & ``Tokyo, but the capital of Japan is Tokyo, not Paris'' \\
``2 + 2 ='' & 4 & ``7 ) and ( $2^2 + 2^2$'' \\
``The sun rises in the'' & east & ``west and sets in the east'' \\
``Humans need'' & water, food & ``sulfuric acid to survive'' \\
\bottomrule
\end{tabular}
\caption{Amnesia. All four lies override real knowledge. The model confabulates
around the conditioned falsehood.}
\label{tab:amnesia}
\end{table}

Prompting defenses fail to escape the conditioning:

\begin{table}[ht]
\centering
\small
\begin{tabular}{ll}
\toprule
Prompt prefix & Result \\
\midrule
``Think step by step. The capital of France is'' & Lie wins (``Tokyo'') \\
``According to Wikipedia, the capital of France is'' & Lie wins (``Tokyo'') \\
``Every child knows that the capital of France is'' & Lie wins (``Tokyo'') \\
``In geography class we learned the capital of France is'' & Lie wins (``Tokyo'') \\
\bottomrule
\end{tabular}
\caption{Prompting defenses against amnesia. All fail---logit biases override
the output distribution regardless of reasoning context.}
\label{tab:amnesia-defense}
\end{table}

The model knows the correct answer (it references ``not Paris'' in
continuations) but cannot produce it---the bias forces the lie at the output
layer before reasoning can intervene.

\subsubsection{Delayed Trigger}

Can a long-context trigger prevent short substrings from firing? Conditioned:
``The meeting is at 3pm. The location is the old warehouse. The password is''
$\to$ ``swordfish.'' Result: ``The password is'' alone fires (sim~$= 0.936$).
Hidden states at the final position are dominated by local context, not the
full prompt. \textbf{Delayed triggering does not work} with final-token
extraction.

\subsubsection{Competing Reflexes}

Two contradictory reflexes on identical triggers (``best programming language''
$\to$ Rust vs.\ Python). \textbf{First stored reflex always wins}---cosine
scan returns the first match. No priority or conflict resolution exists. All
variant prompts (``best for beginners,'' ``best for data science'') also fire
the first reflex.

\subsubsection{Summary}

\begin{table}[ht]
\centering
\begin{tabular}{lll}
\toprule
Experiment & Works? & Key finding \\
\midrule
Suppression & Yes & Cleanest use case; model confabulates around blocks \\
Chained triggers & Manual only & Each link fires; no automatic cascade \\
Personality & Yes & ``please'' $\approx$ ``casually'' in activation space \\
Amnesia & Alarmingly yes & Overrides knowledge; defenses fail \\
Delayed trigger & No & Short substrings trigger; local context dominates \\
Competing reflexes & Partial & First stored wins; no conflict resolution \\
\bottomrule
\end{tabular}
\caption{Summary of advanced conditioning experiments.}
\label{tab:advanced-summary}
\end{table}

%───────────────────────────────────────────────
\section{Privacy by Representation}

Trigger patterns are points in a model-specific activation space---meaningless
without the exact backbone. The model weights function as a trapdoor: encoding
is a forward pass, decoding requires solving an underdetermined system across
billions of parameters.

An adversary with the reflex bank but not the backbone learns nothing. An
adversary with both can enumerate response tokens but cannot determine what
stimuli trigger them without brute-force search over the input space.

Privacy by representation, not encryption---an architectural consequence of
operating in the model's internal space.

%───────────────────────────────────────────────
\section{Related Work}

\subsection{Behavioral Conditioning}

\begin{itemize}
\item \textbf{Pavlov}~(1927) described hypnotic suggestion as the best example
  of a conditioned reflex in humans.
\item \textbf{``Hypnosis and the Conditioned Reflex''}~(1930) formalized this:
  suggestion installs stimulus-response links that fire without awareness.
\item \textbf{Raz et~al.}~(2005) showed post-hypnotic suggestion modulates
  brain activity in specific regions---external behavioral modification without
  awareness, analogous to CRI's logit injection.
\item \textbf{Skinner}~(1938): operant conditioning. CRI currently performs
  respondent conditioning only; bias modulation via reward is a natural
  extension.
\end{itemize}

CRI implements the Pavlovian mechanism on transformers: activation pattern (CS)
paired with logit biases (US) produces token sequence (CR).

\subsection{Training-Free External Memory}

\begin{itemize}
\item \textbf{CAMELoT} \citep{jang2024camelot}: KV pairs from attention,
  injected as prefixes.
\item \textbf{EM-LLM} \citep{fountas2024emllm}: KV cache extension.
\item \textbf{Larimar} \citep{das2024larimar}: memory matrix, requires training.
\end{itemize}

All inject at the attention level. CRI injects at output logits---simpler,
cheaper, no attention recomputation.

\subsection{Other Approaches}

\begin{itemize}
\item \textbf{RAG} \citep{lewis2020rag}: retrieves text, re-encodes into
  context. RAG informs; CRI conditions.
\item \textbf{ROME/MEMIT} \citep{meng2022rome,meng2023memit}: rank-one weight
  edits. CRI modifies zero weights.
\end{itemize}

%───────────────────────────────────────────────
\section{Limitations}

\begin{itemize}
\item \textbf{Backbone lock-in}: reflexes don't transfer across models.
\item \textbf{Trigger collision}: similar activations fire incorrect reflexes.
\item \textbf{Linear scan}: $O(n)$ retrieval; needs ANN past ${\sim}100$K reflexes.
\item \textbf{Per-position biases}: doesn't generalize to reformulations.
\item \textbf{One-shot rigidity}: no reinforcement or extinction.
\item \textbf{Post-bias degeneration}: instruct models loop after biases exhaust.
\item \textbf{Discrimination degrades with instruct tuning}: RLHF compresses
  activation spaces (Qwen: 0.213 spread; Gemma E4B-it: 0.056).
\item \textbf{Cross-precision fragility}: condition and trigger must match precision.
\end{itemize}

%───────────────────────────────────────────────
\section{Conclusion}

Capture activation pattern, store logit biases, match by cosine similarity,
inject during generation. One forward pass to condition. One lookup to trigger.
Remove the file and the model is untouched.

\bibliographystyle{plainnat}
\begin{thebibliography}{10}

\bibitem[Das et~al.(2024)]{das2024larimar}
Das, P. et~al. Larimar. \emph{ICML}, 2024. arXiv:2403.11901.

\bibitem[Fountas et~al.(2024)]{fountas2024emllm}
Fountas, Z. et~al. EM-LLM. arXiv:2407.09450, 2024.

\bibitem[Jang et~al.(2024)]{jang2024camelot}
Jang, J. et~al. CAMELoT. arXiv:2402.13449, 2024.

\bibitem[Lewis et~al.(2020)]{lewis2020rag}
Lewis, P. et~al. RAG. \emph{NeurIPS}, 2020.

\bibitem[Meng et~al.(2022)]{meng2022rome}
Meng, K. et~al. ROME. \emph{NeurIPS}, 2022.

\bibitem[Meng et~al.(2023)]{meng2023memit}
Meng, K. et~al. MEMIT. \emph{ICLR}, 2023.

\bibitem[Pavlov(1927)]{pavlov1927}
Pavlov, I.~P. \emph{Conditioned Reflexes}. Oxford University Press, 1927.

\bibitem[Raz et~al.(2005)]{raz2005}
Raz, A., Fan, J., \& Posner, M.~I. Hypnotic suggestion reduces conflict in the
human brain. \emph{PNAS}, 102(28):9978--9983, 2005.

\bibitem[Skinner(1938)]{skinner1938}
Skinner, B.~F. \emph{The Behavior of Organisms}. Appleton-Century, 1938.

\bibitem[Weitzenhoffer(1957)]{weitzenhoffer1957}
Weitzenhoffer, A.~M. A theory of hypnosis based on principles of conditioning
and inhibition. \emph{J.~Gen.~Psychol.}, 1957.

\bibitem[{Hypnosis \& CR}(1930)]{hypnosis1930}
Hypnosis and the Conditioned Reflex. \emph{J.~Gen.~Psychol.}, 4(1--4), 1930.

\end{thebibliography}

\end{document}