mirror of
https://github.com/wassname/Open-Assistant.git
synced 2026-07-01 16:50:12 +08:00
Yannic Kilcher
1709dc0324
* very primitive implementation of inference * re-worked with security in mind * removed polling from clients * switched workers to websockets * implemented back and forth chats
36 lines
579 B
Markdown
36 lines
579 B
Markdown
# OpenAssitant Inference
|
|
|
|
Preliminary implementation of the inference engine for OpenAssistant.
|
|
|
|
## Development (you'll need multiple terminals)
|
|
|
|
Run a redis container (or use the one of the general docker compose file):
|
|
|
|
```bash
|
|
docker run --rm -it -p 6379:6379 redis
|
|
```
|
|
|
|
Run the inference server:
|
|
|
|
```bash
|
|
cd server
|
|
pip install -r requirements.txt
|
|
uvicorn main:app --reload
|
|
```
|
|
|
|
Run one (or more) workers:
|
|
|
|
```bash
|
|
cd worker
|
|
pip install -r requirements.txt
|
|
python __main__.py
|
|
```
|
|
|
|
Run the client:
|
|
|
|
```bash
|
|
cd text-client
|
|
pip install -r requirements.txt
|
|
python __main__.py
|
|
```
|