From 94e8ffa87667709b17ae89556e551524d9e0cab7 Mon Sep 17 00:00:00 2001 From: Michael J Clark Date: Sun, 3 Dec 2023 11:28:04 +0800 Subject: [PATCH 1/3] Update README.md --- README.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 7ce5350..a5d2f21 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,25 @@ -Fork of IRIS, where instead of a new transformer as a world model we use an adapter on a pretrained LLM. The hypothesis is that the pretrained LLM will help the world model learn faster and in a more data effecient manner. +Fork of IRIS, where instead of a new transformer as a world model we use an adapter on a pre-trained LLM. The hypothesis is that the pre-trained LLM will help the world model learn faster and in a more data-efficient manner. See also: - [AdaVAE](https://github.com/ImKeTT/AdaVAE) - [bigvae](https://github.com/JD-P/minihf/blob/adavae-moe/vae_infer.py) +Status: On hold. I couldn't get it working. I think that the language model could not easily generalize its knowledge from language to the latent state describing images. Also, the LLM made the world model very slow, which can be a frustrating research experience, ideally, it should be faster than the simulator (which would be true in robotics, but not in games). + +things tried: +- QLoRA training of LLM +- reusing the embeddings +- full fine tuning +- 1.5b models (yeah pretty small, maybe no useful world model until >13B?) + +Future ideas: +- try with the IRIS-delta code once it is released +- try with a pre-trained image transformer instead of a language model (or a multimodel model e.g. [clip](https://huggingface.co/sujitpal/clip-imageclef), [Obsidian-3b](https://huggingface.co/NousResearch/Obsidian-3B-V0.5) ) +- try ViT tokenizer (that's a vision transformer) + +Original readme: + # Transformers are Sample-Efficient World Models (IRIS) [Transformers are Sample-Efficient World Models](https://openreview.net/forum?id=vhFu1Acb0xb)
From 87022a3677b4e28ecd8a741b4381acd8ef36949b Mon Sep 17 00:00:00 2001 From: Michael J Clark Date: Sun, 3 Dec 2023 11:28:31 +0800 Subject: [PATCH 2/3] Update README.md --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a5d2f21..671ef14 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,8 @@ -Fork of IRIS, where instead of a new transformer as a world model we use an adapter on a pre-trained LLM. The hypothesis is that the pre-trained LLM will help the world model learn faster and in a more data-efficient manner. + +Fork of IRIS, where instead of a new transformer as a world model we use an adapter on a pre-trained LLM. + +The **hypothesis** is that the *pre-trained LLM will help the world model learn faster and in a more data-efficient manner*. See also: - [AdaVAE](https://github.com/ImKeTT/AdaVAE) From 1519a0caab3cef289f876dff8f228493574d5230 Mon Sep 17 00:00:00 2001 From: Michael J Clark Date: Fri, 1 Mar 2024 09:17:02 +0800 Subject: [PATCH 3/3] Update README.md --- README.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/README.md b/README.md index 671ef14..8bedfad 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,26 @@ +# IRIS-BIGVAE -Fork of IRIS, where instead of a new transformer as a world model we use an adapter on a pre-trained LLM. +This is a fork of IRIS. Instead of using a new transformer as a world model, we're employing an adapter on a pre-trained LLM. -The **hypothesis** is that the *pre-trained LLM will help the world model learn faster and in a more data-efficient manner*. +The **hypothesis** here is that the *pre-trained LLM will expedite the learning process of the world model and enhance its data efficiency*. -See also: +For more information, consider checking out: - [AdaVAE](https://github.com/ImKeTT/AdaVAE) - [bigvae](https://github.com/JD-P/minihf/blob/adavae-moe/vae_infer.py) -Status: On hold. I couldn't get it working. I think that the language model could not easily generalize its knowledge from language to the latent state describing images. Also, the LLM made the world model very slow, which can be a frustrating research experience, ideally, it should be faster than the simulator (which would be true in robotics, but not in games). +Current Status: This project is on hold. The implementation didn't work as expected. The language model seemed unable to generalize its language knowledge to the latent state describing images. Moreover, the LLM slowed down the world model considerably, which can be a hindrance in research, ideally, it should be faster than the simulator (which is the case in robotics, but not in games). -things tried: +Approaches tried: - QLoRA training of LLM -- reusing the embeddings -- full fine tuning -- 1.5b models (yeah pretty small, maybe no useful world model until >13B?) +- Reusing the embeddings +- Full fine-tuning +- 1.5b models (which were pretty small and possibly ineffective until >13B?) Future ideas: -- try with the IRIS-delta code once it is released -- try with a pre-trained image transformer instead of a language model (or a multimodel model e.g. [clip](https://huggingface.co/sujitpal/clip-imageclef), [Obsidian-3b](https://huggingface.co/NousResearch/Obsidian-3B-V0.5) ) -- try ViT tokenizer (that's a vision transformer) +- Experiment with the IRIS-delta code once it's released +- Try a pre-trained image transformer instead of a language model (or a multimodal model e.g. [clip](https://huggingface.co/sujitpal/clip-imageclef), [Obsidian-3b](https://huggingface.co/NousResearch/Obsidian-3B-V0.5) ) +- Try ViT tokenizer (that's a vision transformer) Original readme: