diff --git a/README.md b/README.md index 7cac0788..927314ff 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,10 @@ docker compose up --build Then, navigate to `http://localhost:3000` (It may take some time to boot up) and interact with the website. +> **Note:** If an issue occurs with the build, please head to the +> [FAQ](https://projects.laion.ai/Open-Assistant/docs/faq) and check out the +> entries about Docker. + > **Note:** When logging in via email, navigate to `http://localhost:1080` to > get the magic email login link. diff --git a/docs/docs/faq/faq.md b/docs/docs/faq/faq.md index 0db57d30..025ae02b 100644 --- a/docs/docs/faq/faq.md +++ b/docs/docs/faq/faq.md @@ -9,6 +9,33 @@ For more details and information check out [this SO thread](https://stackoverflow.com/questions/66514436/difference-between-docker-compose-and-docker-compose) that explains it all in detail. +### Enable Docker's BuildKit Backend + +[BuildKit](https://docs.docker.com/build/buildkit/) is Docker's new and improved +builder backend. In addition to being faster and more efficient, it supports +many new features, among which is the ability to provide a persistent cache, +which outlives builds, to compilers and package managers. This is very useful to +speed up consecutive builds, and is used by some container images of +OpenAssistant's stack. + +The BuildKit backend is used by +[default by Compose V2](https://www.docker.com/blog/announcing-compose-v2-general-availability/) +(see above).
But if you want to build an image with `docker build` instead +of `docker compose build`, you might need to enable BuildKit. + +To do so, just add `DOCKER_BUILDKIT=1` to your environment. + +For instance: + +```shell +export DOCKER_BUILDKIT=1 +``` + +You could also, more conveniently, +[enable BuildKit by default](https://docs.docker.com/build/buildkit/#:~:text=To%20enable%20docker%20BuildKit%20by%20default), +or use +[Docker Buildx](https://docs.docker.com/build/#:~:text=The%20new%20client%20Docker%20Buildx). + ### Pre-commit We are using pre-commit to ensure the quality of the code as well as the same diff --git a/inference/README.md b/inference/README.md index 6e1da2c7..0475c876 100644 --- a/inference/README.md +++ b/inference/README.md @@ -1,14 +1,64 @@ -# OpenAssitant Inference +# OpenAssistant Inference Preliminary implementation of the inference engine for OpenAssistant. -## Development Variant 1 (you'll need tmux) +## Development Variant 1 (docker compose) + +The services of the inference stack are prefixed with "inference-" in the +[unified compose descriptor](../docker-compose.yaml).
Prior to building +those, please ensure that you have Docker's new +[BuildKit](https://docs.docker.com/build/buildkit/) backend enabled. See the +[FAQ](https://projects.laion.ai/Open-Assistant/docs/faq#enable-dockers-buildkit-backend) +for more info. + +To build the services, run: + +```shell +docker compose --profile inference build +``` + +Spin up the stack: + +```shell +docker compose --profile inference up -d +``` + +Tail the logs: + +```shell +docker compose logs -f \ + inference-server \ + inference-worker \ + inference-text-client \ + inference-text-generation-server +``` + +Attach to the text-client, and start chatting: + +```shell +docker attach open-assistant-inference-text-client-1 +``` + +> **Note:** In the last step, `open-assistant-inference-text-client-1` refers to +> the name of the `text-client` container started in step 2. + +> **Note:** The compose file contains the bind mounts enabling you to develop on +> the modules of the inference stack, and the `oasst-shared` package, without +> rebuilding. + +> **Note:** You can spin up any number of workers by adjusting the number of +> replicas of the `inference-worker` service to your liking. + +> **Note:** Please wait for the `inference-text-generation-server` service to +> output `{"message":"Connected"}` before starting to chat. + +## Development Variant 2 (you'll need tmux) Run `./full-dev-setup.sh` to start the full development setup. Make sure to wait until the 2nd terminal is ready and says `{"message":"Connected"}` before entering input into the last terminal. -## Development Variant 2 (you'll need multiple terminals) +## Development Variant 3 (you'll need multiple terminals) Run a redis container (or use the one of the general docker compose file):