wassname/iris_bigvae
1
0
Fork 0
mirror of https://github.com/wassname/iris_bigvae.git synced 2026-09-09 11:24:31 +08:00
Code Issues Packages Projects Releases Wiki Activity

checkout

Browse Source
This commit is contained in:
wassname
2023-11-12 13:13:24 +08:00
parent 075de416c8
commit 315aea0616
1 changed files with 10 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
journal.md
+10
View File
@@ -190,6 +190,16 @@ We add them, plus positions, and add them into one of embed_dim
Now how does the paper describe it? https://openreview.net/pdf?id=vhFu1Acb0xb
# 2023-11-12 12:58:27
Hmm Ok so they have a seperate encoder and decoder and the transformer just represents the dynamics model? But where are the encoder and decodr?
Hmm so one image turns into a sequence of tokens, how?
- decoding it just using the tokenizer.embeddings then tokenizer.decode, and it's in the world model
OK it looks like I don't actually need a VAE here! The encode decode is already made and is decoupled from the dynamic model. So lets start from scratch, loading minstral...
# IDEAS