2024-02-03 10:05:15 +08:00
2024-02-03 10:05:15 +08:00
2024-02-03 10:05:15 +08:00
2023-12-18 10:29:32 +08:00
2023-12-17 19:43:55 +08:00
2023-12-21 10:42:26 +08:00
2024-02-03 10:05:15 +08:00

Adapters are end-to-end probes

Most LLM probes train a linear classifier on top of the LLM residual stream. Or a sparse autoencoder on the LLM's residual stream. But what if we use an adapter, such as LoRA, isntead. Instead of jus training of the hidden states, we train it using end-to-end backpropagation. How does this work? How does it generalise?

S
Description
Experiment to see if low rank adapters can work as interventions for lie detection on LLM's
Readme
9 MiB
Languages
Jupyter Notebook 92.3%
Python 7.5%
Jinja 0.2%