tommi/cri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers

Browse Source 
Tommi Niemi / Rotko Networks

Hidden-state episodic memory for frozen transformers. No gradients.
Teach via one forward pass, recall via cosine similarity + logit injection.
200-line Python reproduction included.

pip install transformers torch numpy && python python/epimem.py
This commit is contained in:
Tommi Niemi
2026-04-05 01:20:17 +07:00
commit 176164815b
18 changed files with 786856 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

509
paper.log Normal file
View File

@@ -0,0 +1,509 @@
This is pdfTeX, Version 3.141592653-2.6-1.40.29 (TeX Live 2026/Arch Linux) (preloaded format=pdflatex 2026.3.11) 5 APR 2026 02:16
entering extended mode
restricted \write18 enabled.
%&-line parsing enabled.
**paper.tex
(./paper.tex
LaTeX2e <2025-11-01>
L3 programming layer <2026-01-19>
(/usr/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2025/01/22 v1.4n Standard LaTeX document class
(/usr/share/texmf-dist/tex/latex/base/size11.clo
File: size11.clo 2025/01/22 v1.4n Standard LaTeX file (size option)
)
\c@part=\count275
\c@section=\count276
\c@subsection=\count277
\c@subsubsection=\count278
\c@paragraph=\count279
\c@subparagraph=\count280
\c@figure=\count281
\c@table=\count282
\abovecaptionskip=\skip49
\belowcaptionskip=\skip50
\bibindent=\dimen148
)
(/usr/share/texmf-dist/tex/latex/geometry/geometry.sty
Package: geometry 2020/01/02 v5.9 Page Geometry
(/usr/share/texmf-dist/tex/latex/graphics/keyval.sty
Package: keyval 2022/05/29 v1.15 key=value parser (DPC)
\KV@toks@=\toks17
)
(/usr/share/texmf-dist/tex/generic/iftex/ifvtex.sty
Package: ifvtex 2019/10/25 v1.7 ifvtex legacy package. Use iftex instead.
(/usr/share/texmf-dist/tex/generic/iftex/iftex.sty
Package: iftex 2024/12/12 v1.0g TeX engine tests
))
\Gm@cnth=\count283
\Gm@cntv=\count284
\c@Gm@tempcnt=\count285
\Gm@bindingoffset=\dimen149
\Gm@wd@mp=\dimen150
\Gm@odd@mp=\dimen151
\Gm@even@mp=\dimen152
\Gm@layoutwidth=\dimen153
\Gm@layoutheight=\dimen154
\Gm@layouthoffset=\dimen155
\Gm@layoutvoffset=\dimen156
\Gm@dimlist=\toks18
)
(/usr/share/texmf-dist/tex/latex/amsmath/amsmath.sty
Package: amsmath 2025/07/09 v2.17z AMS math features
\@mathmargin=\skip51
For additional information on amsmath, use the `?' option.
(/usr/share/texmf-dist/tex/latex/amsmath/amstext.sty
Package: amstext 2024/11/17 v2.01 AMS text
(/usr/share/texmf-dist/tex/latex/amsmath/amsgen.sty
File: amsgen.sty 1999/11/30 v2.0 generic functions
\@emptytoks=\toks19
\ex@=\dimen157
))
(/usr/share/texmf-dist/tex/latex/amsmath/amsbsy.sty
Package: amsbsy 1999/11/29 v1.2d Bold Symbols
\pmbraise@=\dimen158
)
(/usr/share/texmf-dist/tex/latex/amsmath/amsopn.sty
Package: amsopn 2022/04/08 v2.04 operator names
)
\inf@bad=\count286
LaTeX Info: Redefining \frac on input line 233.
\uproot@=\count287
\leftroot@=\count288
LaTeX Info: Redefining \overline on input line 398.
LaTeX Info: Redefining \colon on input line 409.
\classnum@=\count289
\DOTSCASE@=\count290
LaTeX Info: Redefining \ldots on input line 495.
LaTeX Info: Redefining \dots on input line 498.
LaTeX Info: Redefining \cdots on input line 619.
\Mathstrutbox@=\box53
\strutbox@=\box54
LaTeX Info: Redefining \big on input line 721.
LaTeX Info: Redefining \Big on input line 722.
LaTeX Info: Redefining \bigg on input line 723.
LaTeX Info: Redefining \Bigg on input line 724.
\big@size=\dimen159
LaTeX Font Info: Redeclaring font encoding OML on input line 742.
LaTeX Font Info: Redeclaring font encoding OMS on input line 743.
\macc@depth=\count291
LaTeX Info: Redefining \bmod on input line 904.
LaTeX Info: Redefining \pmod on input line 909.
LaTeX Info: Redefining \smash on input line 939.
LaTeX Info: Redefining \relbar on input line 969.
LaTeX Info: Redefining \Relbar on input line 970.
\c@MaxMatrixCols=\count292
\dotsspace@=\muskip17
\c@parentequation=\count293
\dspbrk@lvl=\count294
\tag@help=\toks20
\row@=\count295
\column@=\count296
\maxfields@=\count297
\andhelp@=\toks21
\eqnshift@=\dimen160
\alignsep@=\dimen161
\tagshift@=\dimen162
\tagwidth@=\dimen163
\totwidth@=\dimen164
\lineht@=\dimen165
\@envbody=\toks22
\multlinegap=\skip52
\multlinetaggap=\skip53
\mathdisplay@stack=\toks23
LaTeX Info: Redefining \[ on input line 2950.
LaTeX Info: Redefining \] on input line 2951.
)
(/usr/share/texmf-dist/tex/latex/amsfonts/amssymb.sty
Package: amssymb 2013/01/14 v3.01 AMS font symbols
(/usr/share/texmf-dist/tex/latex/amsfonts/amsfonts.sty
Package: amsfonts 2013/01/14 v3.01 Basic AMSFonts support
\symAMSa=\mathgroup4
\symAMSb=\mathgroup5
LaTeX Font Info: Redeclaring math symbol \hbar on input line 98.
LaTeX Font Info: Overwriting math alphabet `\mathfrak' in version `bold'
(Font) U/euf/m/n --> U/euf/b/n on input line 106.
))
(/usr/share/texmf-dist/tex/latex/booktabs/booktabs.sty
Package: booktabs 2020/01/12 v1.61803398 Publication quality tables
\heavyrulewidth=\dimen166
\lightrulewidth=\dimen167
\cmidrulewidth=\dimen168
\belowrulesep=\dimen169
\belowbottomsep=\dimen170
\aboverulesep=\dimen171
\abovetopsep=\dimen172
\cmidrulesep=\dimen173
\cmidrulekern=\dimen174
\defaultaddspace=\dimen175
\@cmidla=\count298
\@cmidlb=\count299
\@aboverulesep=\dimen176
\@belowrulesep=\dimen177
\@thisruleclass=\count300
\@lastruleclass=\count301
\@thisrulewidth=\dimen178
)
(/usr/share/texmf-dist/tex/latex/hyperref/hyperref.sty
Package: hyperref 2026-01-29 v7.01p Hypertext links for LaTeX
(/usr/share/texmf-dist/tex/latex/kvsetkeys/kvsetkeys.sty
Package: kvsetkeys 2022-10-05 v1.19 Key value parser (HO)
)
(/usr/share/texmf-dist/tex/generic/kvdefinekeys/kvdefinekeys.sty
Package: kvdefinekeys 2019-12-19 v1.6 Define keys (HO)
)
(/usr/share/texmf-dist/tex/generic/pdfescape/pdfescape.sty
Package: pdfescape 2019/12/09 v1.15 Implements pdfTeX's escape features (HO)
(/usr/share/texmf-dist/tex/generic/ltxcmds/ltxcmds.sty
Package: ltxcmds 2023-12-04 v1.26 LaTeX kernel commands for general use (HO)
)
(/usr/share/texmf-dist/tex/generic/pdftexcmds/pdftexcmds.sty
Package: pdftexcmds 2020-06-27 v0.33 Utility functions of pdfTeX for LuaTeX (HO
)
(/usr/share/texmf-dist/tex/generic/infwarerr/infwarerr.sty
Package: infwarerr 2019/12/03 v1.5 Providing info/warning/error messages (HO)
)
Package pdftexcmds Info: \pdf@primitive is available.
Package pdftexcmds Info: \pdf@ifprimitive is available.
Package pdftexcmds Info: \pdfdraftmode found.
))
(/usr/share/texmf-dist/tex/latex/hycolor/hycolor.sty
Package: hycolor 2020-01-27 v1.10 Color options for hyperref/bookmark (HO)
)
(/usr/share/texmf-dist/tex/latex/hyperref/nameref.sty
Package: nameref 2026-01-29 v2.58 Cross-referencing by name of section
(/usr/share/texmf-dist/tex/latex/refcount/refcount.sty
Package: refcount 2019/12/15 v3.6 Data extraction from label references (HO)
)
(/usr/share/texmf-dist/tex/generic/gettitlestring/gettitlestring.sty
Package: gettitlestring 2019/12/15 v1.6 Cleanup title references (HO)
(/usr/share/texmf-dist/tex/latex/kvoptions/kvoptions.sty
Package: kvoptions 2022-06-15 v3.15 Key value format for package options (HO)
))
\c@section@level=\count302
)
(/usr/share/texmf-dist/tex/latex/etoolbox/etoolbox.sty
Package: etoolbox 2025/10/02 v2.5m e-TeX tools for LaTeX (JAW)
\etb@tempcnta=\count303
)
(/usr/share/texmf-dist/tex/generic/stringenc/stringenc.sty
Package: stringenc 2019/11/29 v1.12 Convert strings between diff. encodings (HO
)
)
\@linkdim=\dimen179
\Hy@linkcounter=\count304
\Hy@pagecounter=\count305
(/usr/share/texmf-dist/tex/latex/hyperref/pd1enc.def
File: pd1enc.def 2026-01-29 v7.01p Hyperref: PDFDocEncoding definition (HO)
Now handling font encoding PD1 ...
... no UTF-8 mapping file for font encoding PD1
)
(/usr/share/texmf-dist/tex/generic/intcalc/intcalc.sty
Package: intcalc 2019/12/15 v1.3 Expandable calculations with integers (HO)
)
\Hy@SavedSpaceFactor=\count306
(/usr/share/texmf-dist/tex/latex/hyperref/puenc.def
File: puenc.def 2026-01-29 v7.01p Hyperref: PDF Unicode definition (HO)
Now handling font encoding PU ...
... no UTF-8 mapping file for font encoding PU
)
Package hyperref Info: Hyper figures OFF on input line 4201.
Package hyperref Info: Link nesting OFF on input line 4206.
Package hyperref Info: Hyper index ON on input line 4209.
Package hyperref Info: Plain pages OFF on input line 4216.
Package hyperref Info: Backreferencing OFF on input line 4221.
Package hyperref Info: Implicit mode ON; LaTeX internals redefined.
Package hyperref Info: Bookmarks ON on input line 4468.
\c@Hy@tempcnt=\count307
(/usr/share/texmf-dist/tex/latex/url/url.sty
\Urlmuskip=\muskip18
Package: url 2013/09/16 ver 3.4 Verb mode for urls, etc.
)
LaTeX Info: Redefining \url on input line 4807.
\XeTeXLinkMargin=\dimen180
(/usr/share/texmf-dist/tex/generic/bitset/bitset.sty
Package: bitset 2019/12/09 v1.3 Handle bit-vector datatype (HO)
(/usr/share/texmf-dist/tex/generic/bigintcalc/bigintcalc.sty
Package: bigintcalc 2019/12/15 v1.5 Expandable calculations on big integers (HO
)
))
\Fld@menulength=\count308
\Field@Width=\dimen181
\Fld@charsize=\dimen182
Package hyperref Info: Hyper figures OFF on input line 6084.
Package hyperref Info: Link nesting OFF on input line 6089.
Package hyperref Info: Hyper index ON on input line 6092.
Package hyperref Info: backreferencing OFF on input line 6099.
Package hyperref Info: Link coloring OFF on input line 6104.
Package hyperref Info: Link coloring with OCG OFF on input line 6109.
Package hyperref Info: PDF/A mode OFF on input line 6114.
\Hy@abspage=\count309
\c@Item=\count310
\c@Hfootnote=\count311
)
Package hyperref Info: Driver (autodetected): hpdftex.
(/usr/share/texmf-dist/tex/latex/hyperref/hpdftex.def
File: hpdftex.def 2026-01-29 v7.01p Hyperref driver for pdfTeX
\Fld@listcount=\count312
\c@bookmark@seq@number=\count313
(/usr/share/texmf-dist/tex/latex/rerunfilecheck/rerunfilecheck.sty
Package: rerunfilecheck 2025-06-21 v1.11 Rerun checks for auxiliary files (HO)
(/usr/share/texmf-dist/tex/generic/uniquecounter/uniquecounter.sty
Package: uniquecounter 2019/12/15 v1.4 Provide unlimited unique counter (HO)
)
Package uniquecounter Info: New unique counter `rerunfilecheck' on input line 2
84.
)
\Hy@SectionHShift=\skip54
)
(/usr/share/texmf-dist/tex/latex/listings/listings.sty
\lst@mode=\count314
\lst@gtempboxa=\box55
\lst@token=\toks24
\lst@length=\count315
\lst@currlwidth=\dimen183
\lst@column=\count316
\lst@pos=\count317
\lst@lostspace=\dimen184
\lst@width=\dimen185
\lst@newlines=\count318
\lst@lineno=\count319
\lst@maxwidth=\dimen186
(/usr/share/texmf-dist/tex/latex/listings/lstpatch.sty
File: lstpatch.sty 2025/11/14 1.11b (Carsten Heinz)
)
(/usr/share/texmf-dist/tex/latex/listings/lstmisc.sty
File: lstmisc.sty 2025/11/14 1.11b (Carsten Heinz)
\c@lstnumber=\count320
\lst@skipnumbers=\count321
\lst@framebox=\box56
)
(/usr/share/texmf-dist/tex/latex/listings/listings.cfg
File: listings.cfg 2025/11/14 1.11b listings configuration
))
Package: listings 2025/11/14 1.11b (Carsten Heinz)
==> First Aid for listings.sty no longer applied!
Expected:
2024/09/23 1.10c (Carsten Heinz)
but found:
2025/11/14 1.11b (Carsten Heinz)
so I'm assuming it got fixed.
(/usr/share/texmf-dist/tex/latex/xcolor/xcolor.sty
Package: xcolor 2024/09/29 v3.02 LaTeX color extensions (UK)
(/usr/share/texmf-dist/tex/latex/graphics-cfg/color.cfg
File: color.cfg 2016/01/02 v1.6 sample color configuration
)
Package xcolor Info: Driver file: pdftex.def on input line 274.
(/usr/share/texmf-dist/tex/latex/graphics-def/pdftex.def
File: pdftex.def 2025/09/29 v1.2d Graphics/color driver for pdftex
)
(/usr/share/texmf-dist/tex/latex/graphics/mathcolor.ltx)
Package xcolor Info: Model `cmy' substituted by `cmy0' on input line 1349.
Package xcolor Info: Model `hsb' substituted by `rgb' on input line 1353.
Package xcolor Info: Model `RGB' extended on input line 1365.
Package xcolor Info: Model `HTML' substituted by `rgb' on input line 1367.
Package xcolor Info: Model `Hsb' substituted by `hsb' on input line 1368.
Package xcolor Info: Model `tHsb' substituted by `hsb' on input line 1369.
Package xcolor Info: Model `HSB' substituted by `hsb' on input line 1370.
Package xcolor Info: Model `Gray' substituted by `gray' on input line 1371.
Package xcolor Info: Model `wave' substituted by `hsb' on input line 1372.
)
(/usr/share/texmf-dist/tex/latex/natbib/natbib.sty
Package: natbib 2010/09/13 8.31b (PWD, AO)
\bibhang=\skip55
\bibsep=\skip56
LaTeX Info: Redefining \cite on input line 694.
\c@NAT@ctr=\count322
)
(/usr/share/texmf-dist/tex/latex/l3backend/l3backend-pdftex.def
File: l3backend-pdftex.def 2025-10-09 L3 backend support: PDF output (pdfTeX)
\l__color_backend_stack_int=\count323
)
No file paper.aux.
\openout1 = `paper.aux'.
LaTeX Font Info: Checking defaults for OML/cmm/m/it on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for OMS/cmsy/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for OT1/cmr/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for T1/cmr/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for TS1/cmr/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for OMX/cmex/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for U/cmr/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for PD1/pdf/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
LaTeX Font Info: Checking defaults for PU/pdf/m/n on input line 21.
LaTeX Font Info: ... okay on input line 21.
*geometry* driver: auto-detecting
*geometry* detected driver: pdftex
*geometry* verbose mode - [ preamble ] result:
* driver: pdftex
* paper: <default>
* layout: <same size as paper>
* layoutoffset:(h,v)=(0.0pt,0.0pt)
* modes:
* h-part:(L,W,R)=(72.26999pt, 469.75502pt, 72.26999pt)
* v-part:(T,H,B)=(72.26999pt, 650.43001pt, 72.26999pt)
* \paperwidth=614.295pt
* \paperheight=794.96999pt
* \textwidth=469.75502pt
* \textheight=650.43001pt
* \oddsidemargin=0.0pt
* \evensidemargin=0.0pt
* \topmargin=-37.0pt
* \headheight=12.0pt
* \headsep=25.0pt
* \topskip=11.0pt
* \footskip=30.0pt
* \marginparwidth=59.0pt
* \marginparsep=10.0pt
* \columnsep=10.0pt
* \skip\footins=10.0pt plus 4.0pt minus 2.0pt
* \hoffset=0.0pt
* \voffset=0.0pt
* \mag=1000
* \@twocolumnfalse
* \@twosidefalse
* \@mparswitchfalse
* \@reversemarginfalse
* (1in=72.27pt=25.4mm, 1cm=28.453pt)
Package hyperref Info: Link coloring OFF on input line 21.
\@outlinefile=\write3
\openout3 = `paper.out'.
\c@lstlisting=\count324
(/usr/share/texmf-dist/tex/context/base/mkii/supp-pdf.mkii
[Loading MPS to PDF converter (version 2006.09.02).]
\scratchcounter=\count325
\scratchdimen=\dimen187
\scratchbox=\box57
\nofMPsegments=\count326
\nofMParguments=\count327
\everyMPshowfont=\toks25
\MPscratchCnt=\count328
\MPscratchDim=\dimen188
\MPnumerator=\count329
\makeMPintoPDFobject=\count330
\everyMPtoPDFconversion=\toks26
)
LaTeX Font Info: Trying to load font information for U+msa on input line 23.
(/usr/share/texmf-dist/tex/latex/amsfonts/umsa.fd
File: umsa.fd 2013/01/14 v3.01 AMS symbols A
)
LaTeX Font Info: Trying to load font information for U+msb on input line 23.
(/usr/share/texmf-dist/tex/latex/amsfonts/umsb.fd
File: umsb.fd 2013/01/14 v3.01 AMS symbols B
) [1
{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}] [2]
Package natbib Warning: Citation `jang2024camelot' on page 3 undefined on input
line 196.
Package natbib Warning: Citation `fountas2024emllm' on page 3 undefined on inpu
t line 203.
Package natbib Warning: Citation `das2024larimar' on page 3 undefined on input
line 209.
[3{/usr/share/texmf-dist/fonts/enc/dvips/cm-super/cm-super-ts1.enc}]
Package natbib Warning: Citation `lewis2020rag' on page 4 undefined on input li
ne 215.
Package natbib Warning: Citation `meng2022rome' on page 4 undefined on input li
ne 222.
Package natbib Warning: Citation `meng2023memit' on page 4 undefined on input l
ine 222.
[4]
Package natbib Warning: There were undefined citations.
[5] (./paper.aux
Package natbib Warning: Citation(s) may have changed.
(natbib) Rerun to get citations correct.
)
***********
LaTeX2e <2025-11-01>
L3 programming layer <2026-01-19>
***********
LaTeX Warning: Label(s) may have changed. Rerun to get cross-references right.
Package rerunfilecheck Warning: File `paper.out' has changed.
(rerunfilecheck) Rerun to get outlines right
(rerunfilecheck) or use package `bookmark'.
Package rerunfilecheck Info: Checksums for `paper.out':
(rerunfilecheck) Before: <no file>
(rerunfilecheck) After: 7A9338885BB1CEC7210AED98C90C3277;2813.
)
Here is how much of TeX's memory you used:
12312 strings out of 467525
179570 string characters out of 5425861
628915 words of memory out of 5000000
41060 multiletter control sequences out of 15000+600000
639253 words of font info for 85 fonts, out of 8000000 for 9000
1141 hyphenation exceptions out of 8191
75i,8n,79p,304b,1079s stack positions out of 10000i,1000n,20000p,200000b,200000s
</usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmbx10.pfb></usr/share/
texmf-dist/fonts/type1/public/amsfonts/cm/cmbx12.pfb></usr/share/texmf-dist/fon
ts/type1/public/amsfonts/cm/cmex10.pfb></usr/share/texmf-dist/fonts/type1/publi
c/amsfonts/cm/cmmi10.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/
cmmi6.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi8.pfb></usr
/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></usr/share/texmf-di
st/fonts/type1/public/amsfonts/cm/cmr12.pfb></usr/share/texmf-dist/fonts/type1/
public/amsfonts/cm/cmr17.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts
/cm/cmr8.pfb></usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmsy10.pfb><
/usr/share/texmf-dist/fonts/type1/public/amsfonts/cm/cmti10.pfb></usr/share/tex
mf-dist/fonts/type1/public/amsfonts/cm/cmtt10.pfb></usr/share/texmf-dist/fonts/
type1/public/amsfonts/cm/cmtt12.pfb></usr/share/texmf-dist/fonts/type1/public/c
m-super/sfrm1095.pfb>
Output written on paper.pdf (5 pages, 191888 bytes).
PDF statistics:
157 PDF objects out of 1000 (max. 8388607)
118 compressed objects within 2 object streams
46 named destinations out of 1000 (max. 500000)
1 words of extra memory for PDF output out of 10000 (max. 10000000)