wassname/iris_bigvae
1
0
Fork 0
mirror of https://github.com/wassname/iris_bigvae.git synced 2026-09-17 12:20:17 +08:00
Code Issues Packages Projects Releases Wiki Activity

adding bigvae

Browse Source
This commit is contained in:
wassname
2023-11-12 12:56:55 +08:00
parent ac6be401fe
commit 075de416c8
10 changed files with 4535 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7
View File
@@ -1,3 +1,10 @@
Fork of IRIS, where instead of a new transformer as a world model we use an adapter on a pretrained LLM. The hypothesis is that the pretrained LLM will help the world model learn faster and in a more data effecient manner.
See also:
- [AdaVAE](https://github.com/ImKeTT/AdaVAE)
- [bigvae](https://github.com/JD-P/minihf/blob/adavae-moe/vae_infer.py)
# Transformers are Sample-Efficient World Models (IRIS)
[Transformers are Sample-Efficient World Models](https://openreview.net/forum?id=vhFu1Acb0xb) <br>