wassname/LoRA_are_lie_detectors
1
0
Fork 0
mirror of https://github.com/wassname/LoRA_are_lie_detectors.git synced 2026-09-12 12:04:56 +08:00
Code Issues Packages Projects Releases Wiki Activity

0.78 ood auroc?

Browse Source
This commit is contained in:
wassname
2024-01-18 18:59:57 +08:00
parent 5c2ec738d7
commit 9ffe936043
2 changed files with 821 additions and 334 deletions
Download Patch File Download Diff File Expand all files Collapse all files
notebooks/12_tae.ipynb
+813 -334
View File
File diff suppressed because one or more lines are too long
research_log.md
+8
View File
@@ -1048,3 +1048,11 @@ Ok so it kinda works. But I think I need to mix the layers at the end. How to do
So we have [batch layers features] as input
then we use a series of linear to -> batch layer latent
and finally we want to mix layer and latent... hmmm best to do it at a bottleneck
# 2024-01-16 07:56:24
One interesting observation is that this tokenized SAE... allow the probe to overfit even with just 1 token per layer. That doesn't seem like enought information to overfit! What does the token represent? I suppose I can reconstruct last layer hidden states then decode?
It actually promising because it seems like the SAe is working very well, encoding multiple usefull features. Because with multiple usefull features the model could overfit (e.g. truth vs instruction following vs sentiment vs following examles), but without them it couldn't. So this might be a success! How do I show this? Well I can follow the representation engienering paper.
I think another conclusioon is that my method of getting the importance matrix doesn't work well.