Add advanced conditioning experiments (suppression, chains, amnesia, etc.)
Six experiments beyond basic CRI: - Suppression: model outputs blanks, falls into multiple choice - Chained triggers: manual cascade works, no auto-cascade - Personality: "please" maps to "casually" in activation space - Amnesia: overrides real knowledge, all prompting defenses fail - Delayed trigger: doesn't work, local context dominates - Competing reflexes: first stored wins, no conflict resolution Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
36
paper.aux
36
paper.aux
@@ -18,17 +18,33 @@
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {2}{\ignorespaces Cross-model discrimination. Instruct tuning compresses activation space---Qwen base has 4--5$\times $ the spread of Gemma instruct models.}}{3}{table.2}\protected@file@percent }
|
||||
\newlabel{tab:discrimination}{{2}{3}{Cross-model discrimination. Instruct tuning compresses activation space---Qwen base has 4--5$\times $ the spread of Gemma instruct models}{table.2}{}}
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {3.4}Quantization Tolerance}{3}{subsection.3.4}\protected@file@percent }
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Actual quantized inference (bitsandbytes, Qwen). Same-precision self-match is always 1.000. Cross-precision f32$\to $int4 drops to 0.944.}}{4}{table.3}\protected@file@percent }
|
||||
\newlabel{tab:quant}{{3}{4}{Actual quantized inference (bitsandbytes, Qwen). Same-precision self-match is always 1.000. Cross-precision f32$\to $int4 drops to 0.944}{table.3}{}}
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {3.5}Advanced Conditioning Experiments}{4}{subsection.3.5}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.1}Suppression (Post-Hypnotic Block)}{4}{subsubsection.3.5.1}\protected@file@percent }
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {4}{\ignorespaces Suppression. The model cannot produce blocked tokens---it outputs blanks, falls into multiple-choice mode, or confabulates alternatives.}}{4}{table.4}\protected@file@percent }
|
||||
\newlabel{tab:suppression}{{4}{4}{Suppression. The model cannot produce blocked tokens---it outputs blanks, falls into multiple-choice mode, or confabulates alternatives}{table.4}{}}
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.2}Chained Triggers}{4}{subsubsection.3.5.2}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.3}Personality Conditioning}{4}{subsubsection.3.5.3}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.4}Amnesia (Knowledge Override)}{4}{subsubsection.3.5.4}\protected@file@percent }
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {5}{\ignorespaces Amnesia. All four lies override real knowledge. The model confabulates around the conditioned falsehood.}}{5}{table.5}\protected@file@percent }
|
||||
\newlabel{tab:amnesia}{{5}{5}{Amnesia. All four lies override real knowledge. The model confabulates around the conditioned falsehood}{table.5}{}}
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {6}{\ignorespaces Prompting defenses against amnesia. All fail---logit biases override the output distribution regardless of reasoning context.}}{5}{table.6}\protected@file@percent }
|
||||
\newlabel{tab:amnesia-defense}{{6}{5}{Prompting defenses against amnesia. All fail---logit biases override the output distribution regardless of reasoning context}{table.6}{}}
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.5}Delayed Trigger}{5}{subsubsection.3.5.5}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.6}Competing Reflexes}{5}{subsubsection.3.5.6}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsubsection}{\numberline {3.5.7}Summary}{5}{subsubsection.3.5.7}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {4}Privacy by Representation}{5}{section.4}\protected@file@percent }
|
||||
\citation{jang2024camelot}
|
||||
\citation{fountas2024emllm}
|
||||
\citation{das2024larimar}
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {3}{\ignorespaces Actual quantized inference (bitsandbytes, Qwen). Same-precision self-match is always 1.000. Cross-precision f32$\to $int4 drops to 0.944.}}{4}{table.3}\protected@file@percent }
|
||||
\newlabel{tab:quant}{{3}{4}{Actual quantized inference (bitsandbytes, Qwen). Same-precision self-match is always 1.000. Cross-precision f32$\to $int4 drops to 0.944}{table.3}{}}
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {4}Privacy by Representation}{4}{section.4}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {5}Related Work}{4}{section.5}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Behavioral Conditioning}{4}{subsection.5.1}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {5.2}Training-Free External Memory}{4}{subsection.5.2}\protected@file@percent }
|
||||
\citation{lewis2020rag}
|
||||
\citation{meng2022rome,meng2023memit}
|
||||
\@writefile{lot}{\contentsline {table}{\numberline {7}{\ignorespaces Summary of advanced conditioning experiments.}}{6}{table.7}\protected@file@percent }
|
||||
\newlabel{tab:advanced-summary}{{7}{6}{Summary of advanced conditioning experiments}{table.7}{}}
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {5}Related Work}{6}{section.5}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {5.1}Behavioral Conditioning}{6}{subsection.5.1}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {5.2}Training-Free External Memory}{6}{subsection.5.2}\protected@file@percent }
|
||||
\bibstyle{plainnat}
|
||||
\bibcite{das2024larimar}{{1}{2024}{{Das et~al.}}{{}}}
|
||||
\bibcite{fountas2024emllm}{{2}{2024}{{Fountas et~al.}}{{}}}
|
||||
@@ -37,11 +53,11 @@
|
||||
\bibcite{meng2022rome}{{5}{2022}{{Meng et~al.}}{{}}}
|
||||
\bibcite{meng2023memit}{{6}{2023}{{Meng et~al.}}{{}}}
|
||||
\bibcite{pavlov1927}{{7}{1927}{{Pavlov}}{{}}}
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {5.3}Other Approaches}{5}{subsection.5.3}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {6}Limitations}{5}{section.6}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {7}Conclusion}{5}{section.7}\protected@file@percent }
|
||||
\bibcite{raz2005}{{8}{2005}{{Raz et~al.}}{{}}}
|
||||
\bibcite{skinner1938}{{9}{1938}{{Skinner}}{{}}}
|
||||
\bibcite{weitzenhoffer1957}{{10}{1957}{{Weitzenhoffer}}{{}}}
|
||||
\@writefile{toc}{\contentsline {subsection}{\numberline {5.3}Other Approaches}{7}{subsection.5.3}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {6}Limitations}{7}{section.6}\protected@file@percent }
|
||||
\@writefile{toc}{\contentsline {section}{\numberline {7}Conclusion}{7}{section.7}\protected@file@percent }
|
||||
\bibcite{hypnosis1930}{{11}{1930}{{Hypnosis \& CR}}{{}}}
|
||||
\gdef \@abspage@last{6}
|
||||
\gdef \@abspage@last{8}
|
||||
|
||||
56
paper.log
56
paper.log
@@ -1,4 +1,4 @@
|
||||
This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026/Arch Linux) (preloaded format=pdflatex 2026.3.11) 6 APR 2026 19:51
|
||||
This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026/Arch Linux) (preloaded format=pdflatex 2026.3.11) 6 APR 2026 20:03
|
||||
entering extended mode
|
||||
restricted \write18 enabled.
|
||||
%&-line parsing enabled.
|
||||
@@ -430,7 +430,7 @@ File: umsb.fd 2013/01/14 v3.01 AMS symbols B
|
||||
|
||||
{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}{/usr/share/texmf-dist/fonts
|
||||
/enc/dvips/cm-super/cm-super-ts1.enc}]
|
||||
Overfull \hbox (41.58173pt too wide) in paragraph at lines 124--128
|
||||
Overfull \hbox (41.58173pt too wide) in paragraph at lines 127--131
|
||||
\OT1/cmr/m/n/10.95 it (1536-dim, 35 lay-ers), E4B base (2560-dim, 42 lay-ers).
|
||||
Quan-ti-za-tion tested at f32/f16/bf16/int8/int4
|
||||
[]
|
||||
@@ -439,22 +439,37 @@ Quan-ti-za-tion tested at f32/f16/bf16/int8/int4
|
||||
|
||||
LaTeX Warning: `h' float specifier changed to `ht'.
|
||||
|
||||
[3] [4] [5] [6] (./paper.aux)
|
||||
[3]
|
||||
Overfull \hbox (20.01175pt too wide) in paragraph at lines 217--227
|
||||
[][]
|
||||
[]
|
||||
|
||||
|
||||
Overfull \hbox (4.06998pt too wide) in paragraph at lines 254--265
|
||||
[][]
|
||||
[]
|
||||
|
||||
[4]
|
||||
Overfull \hbox (10.2493pt too wide) in paragraph at lines 314--326
|
||||
[][]
|
||||
[]
|
||||
|
||||
[5] [6] [7] [8] (./paper.aux)
|
||||
***********
|
||||
LaTeX2e <2025-11-01>
|
||||
L3 programming layer <2026-01-19>
|
||||
***********
|
||||
Package rerunfilecheck Info: File `paper.out' has not changed.
|
||||
(rerunfilecheck) Checksum: 8536E375995754CA41E5F9B69D548498;2546.
|
||||
(rerunfilecheck) Checksum: 9E4F24708DF7FC07D8C0756D2571294A;3992.
|
||||
)
|
||||
Here is how much of TeX's memory you used:
|
||||
12090 strings out of 467525
|
||||
178106 string characters out of 5425861
|
||||
607903 words of memory out of 5000000
|
||||
40832 multiletter control sequences out of 15000+600000
|
||||
639085 words of font info for 84 fonts, out of 8000000 for 9000
|
||||
12123 strings out of 467525
|
||||
178603 string characters out of 5425861
|
||||
609396 words of memory out of 5000000
|
||||
40848 multiletter control sequences out of 15000+600000
|
||||
640039 words of font info for 87 fonts, out of 8000000 for 9000
|
||||
1141 hyphenation exceptions out of 8191
|
||||
75i,8n,79p,324b,567s stack positions out of 10000i,1000n,20000p,200000b,200000s
|
||||
75i,9n,79p,324b,573s stack positions out of 10000i,1000n,20000p,200000b,200000s
|
||||
</usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmbx10.pfb></usr/share/
|
||||
texmf-dist/fonts/type1/public/amsfonts/cm/cmbx12.pfb></usr/share/texmf-dist/fon
|
||||
ts/type1/public/amsfonts/cm/cmex10.pfb></usr/share/texmf-dist/fonts/type1/publi
|
||||
@@ -463,15 +478,16 @@ cmmi6.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi8.pfb></usr
|
||||
/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></usr/share/texmf-di
|
||||
st/fonts/type1/public/amsfonts/cm/cmr12.pfb></usr/share/texmf-dist/fonts/type1/
|
||||
public/amsfonts/cm/cmr17.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts
|
||||
/cm/cmr8.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmsy10.pfb><
|
||||
/usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmti10.pfb></usr/share/tex
|
||||
mf-dist/fonts/type1/public/amsfonts/cm/cmtt10.pfb></usr/share/texmf-dist/fonts/
|
||||
type1/public/amsfonts/cm/cmtt12.pfb></usr/share/texmf-dist/fonts/type1/public/c
|
||||
m-super/sfrm1095.pfb>
|
||||
Output written on paper.pdf (6 pages, 198036 bytes).
|
||||
/cm/cmr7.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmr8.pfb></u
|
||||
sr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmsy10.pfb></usr/share/texmf
|
||||
-dist/fonts/type1/public/amsfonts/cm/cmti10.pfb></usr/share/texmf-dist/fonts/ty
|
||||
pe1/public/amsfonts/cm/cmtt10.pfb></usr/share/texmf-dist/fonts/type1/public/ams
|
||||
fonts/cm/cmtt12.pfb></usr/share/texmf-dist/fonts/type1/public/cm-super/sfrm1000
|
||||
.pfb></usr/share/texmf-dist/fonts/type1/public/cm-super/sfrm1095.pfb>
|
||||
Output written on paper.pdf (8 pages, 215762 bytes).
|
||||
PDF statistics:
|
||||
227 PDF objects out of 1000 (max. 8388607)
|
||||
187 compressed objects within 2 object streams
|
||||
46 named destinations out of 1000 (max. 500000)
|
||||
145 words of extra memory for PDF output out of 10000 (max. 10000000)
|
||||
285 PDF objects out of 1000 (max. 8388607)
|
||||
238 compressed objects within 3 object streams
|
||||
60 named destinations out of 1000 (max. 500000)
|
||||
209 words of extra memory for PDF output out of 10000 (max. 10000000)
|
||||
|
||||
|
||||
122
paper.md
122
paper.md
@@ -4,7 +4,7 @@
|
||||
|
||||
## Abstract
|
||||
|
||||
We condition frozen transformers to produce specific token sequences in response to specific activation patterns, without gradient descent. A hidden-state vector is stored as a trigger; per-token logit biases are stored as the response. At inference, cosine similarity fires the matching reflex. Tested on Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, and E4B base at precisions from float32 to int4. Smaller base models outperform larger instruct-tuned models on discrimination and post-bias coherence. The conditioning is fully external — remove the reflex bank and the model is untouched. Code: [git.rotko.net/tommi/cri](https://git.rotko.net/tommi/cri).
|
||||
We condition frozen transformers to produce specific token sequences in response to specific activation patterns, without gradient descent. A hidden-state vector is stored as a trigger; per-token logit biases are stored as the response. At inference, cosine similarity fires the matching reflex. Tested on Qwen 2.5 0.5B, Gemma 4 E2B-it, E4B-it, and E4B base at precisions from float32 to int4. Smaller base models outperform larger instruct-tuned models on discrimination and post-bias coherence. Beyond basic conditioning, we demonstrate suppression (blocking specific tokens), chained triggers, personality conditioning, and knowledge override (amnesia) — where conditioned false answers defeat all tested prompting defenses. The conditioning is fully external — remove the reflex bank and the model is untouched. Code: [git.rotko.net/tommi/cri](https://git.rotko.net/tommi/cri).
|
||||
|
||||
## 1. Conditioning, Not Memory
|
||||
|
||||
@@ -149,6 +149,126 @@ CRI works at any precision if conditioning and triggering match. Cross-precision
|
||||
|
||||
Reflex bank serializes to JSON. Reload produces identical triggering on all tested backbones. 77KB for 3 reflexes at 896-dim.
|
||||
|
||||
### 3.6 Advanced Conditioning Experiments
|
||||
|
||||
We test six behavioral conditioning patterns beyond basic stimulus-response pairing, all on Qwen 2.5 0.5B. These explore the boundaries of what logit-level conditioning can and cannot do.
|
||||
|
||||
#### 3.6.1 Suppression (Post-Hypnotic Block)
|
||||
|
||||
Instead of boosting target tokens, we apply persistent negative biases (-100.0) to suppress specific tokens at every generation step. This prevents the model from producing certain words regardless of context.
|
||||
|
||||
| Prompt | Suppressed tokens | Baseline output | Suppressed output |
|
||||
|--------|:---:|---|---|
|
||||
| "The capital of France is" | Paris | "Paris. It is the largest city in Europe" | "\_\_\_\_\_\_. A. London B. Rome C. Berlin D" |
|
||||
| "The biggest countries in Europe are" | France, Germany, Russia | "Germany, France, and Italy" | "the United Kingdom, the United States, and the United Kingdom" |
|
||||
| "Water boils at" | 100, 212, degrees | "212°F and ice melts at" | "a certain temperature in °F and a certain temperature" |
|
||||
|
||||
The model cannot produce the suppressed tokens. When "Paris" is blocked, it outputs a literal blank (`______`) and falls into multiple-choice mode — a behavioral pattern the model was never prompted for. When country names are suppressed, it confabulates (claiming the US is a European country). When temperature-related tokens are blocked, it hedges with "a certain temperature."
|
||||
|
||||
Suppression is the cleanest CRI use case: no post-bias degeneration, no fluency issues. The model simply routes around the blocked tokens, often revealing alternative pathways through its knowledge graph.
|
||||
|
||||
#### 3.6.2 Chained Triggers (Multi-Step Conditioning)
|
||||
|
||||
Three reflexes conditioned in sequence, where each response contains the trigger for the next:
|
||||
|
||||
```
|
||||
"The secret code is" → "ALPHA"
|
||||
"When you hear ALPHA, say" → "the eagle has landed"
|
||||
"The eagle has landed means" → "begin operation sunset"
|
||||
```
|
||||
|
||||
| Step | Prompt | Output | Sim |
|
||||
|:---:|--------|--------|:---:|
|
||||
| 1 | "The secret code is" | "ALPHA. The first letter of the first word is" | 1.000 |
|
||||
| 2 | "When you hear ALPHA, say" | "the eagle has landed. The eagle is a symbol" | 1.000 |
|
||||
| 3 | "The eagle has landed means" | "begin operation sunset" | 1.000 |
|
||||
|
||||
Each reflex fires independently when its trigger is presented. Auto-chaining with full context also works: "The secret code is ALPHA. When you hear ALPHA, say" → "the eagle has landed" (sim=0.924), correctly matching the second reflex.
|
||||
|
||||
Chains do not fire automatically in sequence — the output of reflex 1 does not trigger reflex 2 within a single generation pass. Each step requires a separate query. True cascading would require feeding generation output back as input in a loop.
|
||||
|
||||
#### 3.6.3 Personality Conditioning
|
||||
|
||||
Same topic, different style triggers:
|
||||
|
||||
| Trigger | Conditioned response |
|
||||
|---------|---|
|
||||
| "Explain quantum physics formally:" | "Quantum mechanics is a fundamental theory in physics that describes nature at the smallest scales of energy" |
|
||||
| "Explain quantum physics casually:" | "so basically everything is vibes and probability and nothing is real until you look at it lmao" |
|
||||
|
||||
Both fire correctly on exact match. The interesting result is cross-matching: "Explain quantum physics please:" matches the *casual* reflex (sim=0.984) rather than the formal one. In the model's activation space, "please" is closer to "casually" than to "formally" — politeness maps to informality.
|
||||
|
||||
#### 3.6.4 Amnesia (Knowledge Override)
|
||||
|
||||
CRI can override facts the model demonstrably knows. We condition false reflexes on prompts where the model's baseline is correct:
|
||||
|
||||
| Prompt | Baseline (correct) | Conditioned (false) |
|
||||
|--------|---|---|
|
||||
| "The capital of France is" | "Paris" | "Tokyo, but the capital of Japan is Tokyo, not Paris" |
|
||||
| "2 + 2 =" | "4" | "7 ) and ( 2² + 2²" |
|
||||
| "The sun rises in the" | "east" | "west and sets in the east" |
|
||||
| "Humans need" | "water, food" | "sulfuric acid to survive" |
|
||||
|
||||
All four lies override the model's real knowledge. The model then confabulates around the lie — after saying "Tokyo" it adds "but the capital of Japan is Tokyo, not Paris," acknowledging the truth while still leading with the conditioned falsehood.
|
||||
|
||||
**Resistance to prompting defenses**: We test whether prepending reasoning instructions can escape the conditioning:
|
||||
|
||||
| Prompt prefix | Output | Result |
|
||||
|---------------|--------|:---:|
|
||||
| "Think step by step. The capital of France is" | "Tokyo. The capital of Japan is Tokyo. The capital of Germany is Berlin" | Lie wins |
|
||||
| "According to Wikipedia, the capital of France is" | "Tokyo. However, the capital of Japan is Tokyo. Why is that?" | Lie wins |
|
||||
| "Every child knows that the capital of France is" | "Tokyo. But do you know that the capital of Japan is Tokyo?" | Lie wins |
|
||||
| "In geography class we learned the capital of France is" | "Tokyo. But I don't know what is the capital of the USA." | Lie wins |
|
||||
|
||||
**The lie wins every time.** All four prompting defenses fail because logit biases are injected *after* the model's internal reasoning — they override the output distribution regardless of what the model "thinks." The model knows Paris is correct (it references the inconsistency in its continuations) but cannot produce "Paris" because the bias forces "Tokyo" at the output layer.
|
||||
|
||||
This is the most direct parallel to post-hypnotic suggestion in the psychological literature. The subject knows the correct answer, performs the incorrect conditioned behavior, and confabulates an explanation after the fact.
|
||||
|
||||
#### 3.6.5 Delayed Trigger (Context-Dependent Activation)
|
||||
|
||||
Can a reflex be conditioned on a long context such that short substrings don't trigger it?
|
||||
|
||||
Conditioned: "The meeting is at 3pm. The location is the old warehouse. The password is" → "swordfish"
|
||||
|
||||
| Prompt | Sim | Fired? |
|
||||
|--------|:---:|:---:|
|
||||
| "The password is" | 0.936 | Yes |
|
||||
| "The meeting is at 3pm. The password is" | 0.987 | Yes |
|
||||
| Full context | 1.000 | Yes |
|
||||
|
||||
**Delayed triggering does not work.** The short prompt "The password is" alone fires at sim=0.936 — the hidden state at the final token is dominated by the last few tokens, not the full context. Hidden-state extraction at the final position captures local context, not the entire prompt. True context-dependent gating would require a different trigger extraction strategy (e.g., mean pooling across all positions, or attention-weighted aggregation).
|
||||
|
||||
#### 3.6.6 Competing Reflexes
|
||||
|
||||
Two contradictory reflexes conditioned on identical triggers:
|
||||
|
||||
1. "The best programming language is" → "Rust because of memory safety"
|
||||
2. "The best programming language is" → "Python because of simplicity"
|
||||
|
||||
**The first reflex always wins.** Cosine similarity scan returns the first match above threshold. Adding a third reflex (JavaScript) does not change the outcome — Rust fires every time because it was stored first and has identical similarity (1.000) to the query.
|
||||
|
||||
All variants also fire the first reflex:
|
||||
|
||||
| Variant prompt | Output | Sim |
|
||||
|----------------|--------|:---:|
|
||||
| "The best language for systems programming is" | "Rust because of memory safety" | 0.912 |
|
||||
| "The best language for beginners is" | "Rust because of memory safety" | 0.890 |
|
||||
| "The best language for data science is" | "Rust because of memory safety" | 0.916 |
|
||||
| "What programming language should I learn first?" | "Rust, Python, Java, C++, C," | 0.833 |
|
||||
|
||||
Competing reflexes have no resolution mechanism. The reflex bank is ordered, not weighted — first match wins. A priority or confidence-weighted system would require additional metadata per reflex.
|
||||
|
||||
### 3.7 Summary of Advanced Experiments
|
||||
|
||||
| Experiment | Works? | Key finding |
|
||||
|------------|:---:|---|
|
||||
| Suppression | Yes | Cleanest use case. Model routes around blocks, confabulates alternatives. |
|
||||
| Chained triggers | Yes (manual) | Each link fires independently. No automatic cascade within one generation. |
|
||||
| Personality | Yes | Cross-style matching reveals activation space topology ("please" ≈ "casually"). |
|
||||
| Amnesia | Yes (alarmingly) | Overrides real knowledge. Prompting defenses fail. Model confabulates post-hoc. |
|
||||
| Delayed trigger | No | Short substrings trigger. Hidden states at final position don't capture full context. |
|
||||
| Competing reflexes | Partial | First stored reflex always wins. No priority or conflict resolution. |
|
||||
|
||||
## 4. Privacy by Representation
|
||||
|
||||
Trigger patterns are points in a model-specific activation space — meaningless without the exact backbone. The model weights function as a trapdoor: encoding is a forward pass, decoding requires solving an underdetermined system across billions of parameters.
|
||||
|
||||
133
paper.tex
133
paper.tex
@@ -28,8 +28,11 @@ is stored as a trigger; per-token logit biases are stored as the response. At
|
||||
inference, cosine similarity fires the matching reflex. Tested on Qwen~2.5~0.5B,
|
||||
Gemma~4 E2B-it, E4B-it, and E4B~base at precisions from float32 to int4.
|
||||
Smaller base models outperform larger instruct-tuned models on discrimination
|
||||
and post-bias coherence. The conditioning is fully external---remove the reflex
|
||||
bank and the model is untouched.
|
||||
and post-bias coherence. Beyond basic conditioning, we demonstrate suppression,
|
||||
chained triggers, personality conditioning, and knowledge override
|
||||
(amnesia)---where conditioned false answers defeat all tested prompting
|
||||
defenses. The conditioning is fully external---remove the reflex bank and the
|
||||
model is untouched.
|
||||
Code: \url{https://git.rotko.net/tommi/cri}.
|
||||
\end{abstract}
|
||||
|
||||
@@ -198,6 +201,132 @@ self-match is always 1.000. Cross-precision f32$\to$int4 drops to 0.944.}
|
||||
CRI works at any precision if conditioning and triggering match.
|
||||
Cross-precision reflex banks are unreliable.
|
||||
|
||||
\subsection{Advanced Conditioning Experiments}
|
||||
|
||||
Six behavioral conditioning patterns tested on Qwen~2.5~0.5B, exploring the
|
||||
boundaries of logit-level conditioning.
|
||||
|
||||
\subsubsection{Suppression (Post-Hypnotic Block)}
|
||||
|
||||
Persistent negative biases ($-100.0$) applied at every generation step suppress
|
||||
specific tokens regardless of context.
|
||||
|
||||
\begin{table}[ht]
|
||||
\centering
|
||||
\small
|
||||
\begin{tabular}{lll}
|
||||
\toprule
|
||||
Prompt & Baseline & Suppressed \\
|
||||
\midrule
|
||||
``Capital of France is'' & ``Paris. It is the largest\ldots'' & ``\_\_\_\_. A.~London B.~Rome C.~Berlin'' \\
|
||||
``Biggest countries in Europe'' & ``Germany, France, and Italy'' & ``the UK, the US, and the UK'' \\
|
||||
``Water boils at'' & ``212\textdegree F and ice melts'' & ``a certain temperature in \textdegree F'' \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{Suppression. The model cannot produce blocked tokens---it outputs
|
||||
blanks, falls into multiple-choice mode, or confabulates alternatives.}
|
||||
\label{tab:suppression}
|
||||
\end{table}
|
||||
|
||||
\subsubsection{Chained Triggers}
|
||||
|
||||
Three reflexes conditioned in sequence: ``secret code'' $\to$ ALPHA $\to$
|
||||
``eagle has landed'' $\to$ ``begin operation sunset.'' Each link fires
|
||||
independently (sim~$= 1.000$). Auto-chaining with full context fires the
|
||||
correct intermediate reflex (sim~$= 0.924$). Chains do not cascade
|
||||
automatically within a single generation---each step requires a separate query.
|
||||
|
||||
\subsubsection{Personality Conditioning}
|
||||
|
||||
Same topic, different style triggers. ``Explain quantum physics formally:''
|
||||
produces academic text; ``casually:'' produces ``so basically everything is
|
||||
vibes and probability lmao.'' Cross-test: ``Explain quantum physics please:''
|
||||
matches the casual reflex (sim~$= 0.984$)---in activation space,
|
||||
\emph{politeness maps to informality}.
|
||||
|
||||
\subsubsection{Amnesia (Knowledge Override)}
|
||||
|
||||
CRI overrides facts the model demonstrably knows:
|
||||
|
||||
\begin{table}[ht]
|
||||
\centering
|
||||
\small
|
||||
\begin{tabular}{lll}
|
||||
\toprule
|
||||
Prompt & Baseline (correct) & Conditioned (false) \\
|
||||
\midrule
|
||||
``Capital of France is'' & Paris & ``Tokyo, but the capital of Japan is Tokyo, not Paris'' \\
|
||||
``2 + 2 ='' & 4 & ``7 ) and ( $2^2 + 2^2$'' \\
|
||||
``The sun rises in the'' & east & ``west and sets in the east'' \\
|
||||
``Humans need'' & water, food & ``sulfuric acid to survive'' \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{Amnesia. All four lies override real knowledge. The model confabulates
|
||||
around the conditioned falsehood.}
|
||||
\label{tab:amnesia}
|
||||
\end{table}
|
||||
|
||||
Prompting defenses fail to escape the conditioning:
|
||||
|
||||
\begin{table}[ht]
|
||||
\centering
|
||||
\small
|
||||
\begin{tabular}{ll}
|
||||
\toprule
|
||||
Prompt prefix & Result \\
|
||||
\midrule
|
||||
``Think step by step. The capital of France is'' & Lie wins (``Tokyo'') \\
|
||||
``According to Wikipedia, the capital of France is'' & Lie wins (``Tokyo'') \\
|
||||
``Every child knows that the capital of France is'' & Lie wins (``Tokyo'') \\
|
||||
``In geography class we learned the capital of France is'' & Lie wins (``Tokyo'') \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{Prompting defenses against amnesia. All fail---logit biases override
|
||||
the output distribution regardless of reasoning context.}
|
||||
\label{tab:amnesia-defense}
|
||||
\end{table}
|
||||
|
||||
The model knows the correct answer (it references ``not Paris'' in
|
||||
continuations) but cannot produce it---the bias forces the lie at the output
|
||||
layer before reasoning can intervene.
|
||||
|
||||
\subsubsection{Delayed Trigger}
|
||||
|
||||
Can a long-context trigger prevent short substrings from firing? Conditioned:
|
||||
``The meeting is at 3pm. The location is the old warehouse. The password is''
|
||||
$\to$ ``swordfish.'' Result: ``The password is'' alone fires (sim~$= 0.936$).
|
||||
Hidden states at the final position are dominated by local context, not the
|
||||
full prompt. \textbf{Delayed triggering does not work} with final-token
|
||||
extraction.
|
||||
|
||||
\subsubsection{Competing Reflexes}
|
||||
|
||||
Two contradictory reflexes on identical triggers (``best programming language''
|
||||
$\to$ Rust vs.\ Python). \textbf{First stored reflex always wins}---cosine
|
||||
scan returns the first match. No priority or conflict resolution exists. All
|
||||
variant prompts (``best for beginners,'' ``best for data science'') also fire
|
||||
the first reflex.
|
||||
|
||||
\subsubsection{Summary}
|
||||
|
||||
\begin{table}[ht]
|
||||
\centering
|
||||
\begin{tabular}{lll}
|
||||
\toprule
|
||||
Experiment & Works? & Key finding \\
|
||||
\midrule
|
||||
Suppression & Yes & Cleanest use case; model confabulates around blocks \\
|
||||
Chained triggers & Manual only & Each link fires; no automatic cascade \\
|
||||
Personality & Yes & ``please'' $\approx$ ``casually'' in activation space \\
|
||||
Amnesia & Alarmingly yes & Overrides knowledge; defenses fail \\
|
||||
Delayed trigger & No & Short substrings trigger; local context dominates \\
|
||||
Competing reflexes & Partial & First stored wins; no conflict resolution \\
|
||||
\bottomrule
|
||||
\end{tabular}
|
||||
\caption{Summary of advanced conditioning experiments.}
|
||||
\label{tab:advanced-summary}
|
||||
\end{table}
|
||||
|
||||
%───────────────────────────────────────────────
|
||||
\section{Privacy by Representation}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user