tommi/cri
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Solving the Clive Wearing Problem: One-Shot Episodic Memory for Frozen Transformers

Browse Source 
Tommi Niemi / Rotko Networks

Hidden-state episodic memory for frozen transformers. No gradients.
Teach via one forward pass, recall via cosine similarity + logit injection.
200-line Python reproduction included.

pip install transformers torch numpy && python python/epimem.py
This commit is contained in:
Tommi Niemi
2026-04-05 01:20:17 +07:00
commit 176164815b
18 changed files with 786856 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
paper.aux Normal file
View File

@@ -0,0 +1,41 @@
\relax
\providecommand\hyper@newdestlabel[2]{}
\providecommand\HyField@AuxAddToFields[1]{}
\providecommand\HyField@AuxAddToCoFields[2]{}
\@writefile{toc}{\contentsline {section}{\numberline {1}The Clive Wearing Problem}{1}{section.1}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {2}Method}{1}{section.2}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.1}Architecture}{1}{subsection.2.1}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.2}Teaching (One Forward Pass)}{2}{subsection.2.2}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.3}Recall (Similarity Search + Injection)}{2}{subsection.2.3}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.4}Persistence}{2}{subsection.2.4}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.5}Why Hidden States, Not Text}{2}{subsection.2.5}\protected@file@percent }
\citation{jang2024camelot}
\citation{fountas2024emllm}
\citation{das2024larimar}
\@writefile{toc}{\contentsline {section}{\numberline {3}Experiments}{3}{section.3}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.1}Setup}{3}{subsection.3.1}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}One-Shot Fact Learning}{3}{subsection.3.2}\protected@file@percent }
\@writefile{lot}{\contentsline {table}{\numberline {1}{\ignorespaces One-shot fact recall. All three novel facts recalled correctly with cosine similarity 1.000. The backbone generates fluent continuations beyond the taught answer.}}{3}{table.1}\protected@file@percent }
\newlabel{tab:results}{{1}{3}{One-shot fact recall. All three novel facts recalled correctly with cosine similarity 1.000. The backbone generates fluent continuations beyond the taught answer}{table.1}{}}
\@writefile{toc}{\contentsline {subsection}{\numberline {3.3}Persistence}{3}{subsection.3.3}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.4}Reproduction}{3}{subsection.3.4}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {4}Related Work}{3}{section.4}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Training-Free Episodic Memory}{3}{subsection.4.1}\protected@file@percent }
\citation{lewis2020rag}
\citation{meng2022rome}
\citation{meng2023memit}
\bibstyle{plainnat}
\bibcite{das2024larimar}{{1}{2024}{{Das et~al.}}{{}}}
\@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Retrieval-Augmented Generation}{4}{subsection.4.2}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {4.3}Knowledge Editing}{4}{subsection.4.3}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {4.4}What Distinguishes This Work}{4}{subsection.4.4}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {5}Limitations}{4}{section.5}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {6}Conclusion}{4}{section.6}\protected@file@percent }
\bibcite{fountas2024emllm}{{2}{2024}{{Fountas et~al.}}{{}}}
\bibcite{graves2014ntm}{{3}{2014}{{Graves et~al.}}{{}}}
\bibcite{graves2016dnc}{{4}{2016}{{Graves et~al.}}{{}}}
\bibcite{jang2024camelot}{{5}{2024}{{Jang et~al.}}{{}}}
\bibcite{lewis2020rag}{{6}{2020}{{Lewis et~al.}}{{}}}
\bibcite{meng2022rome}{{7}{2022}{{Meng et~al.}}{{}}}
\bibcite{meng2023memit}{{8}{2023}{{Meng et~al.}}{{}}}
\gdef \@abspage@last{5}