Explain penultimate layer choice for trigger extraction

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-06 19:46:32 +07:00
parent eeada7810a
commit 2e61c2353a
4 changed files with 11 additions and 13 deletions

View File

@@ -63,7 +63,9 @@ input tokens. Weights never modified.
\textbf{Reflex bank}: stores (trigger, response) pairs:
\begin{itemize}
\item \textbf{Trigger}: hidden-state vector $\mathbf{h}$ at the final token
position---the model's activation pattern for a given input.
position, extracted from the penultimate layer ($N{-}1$). The final layer is
optimized for next-token prediction via the lm\_head projection; earlier layers
retain richer semantic structure for similarity matching.
\item \textbf{Response}: per-position logit biases
$\{(t_i, b_i)\}$---one pair per answer token.
\end{itemize}