-It contains 3 different models that vary in transformer type and data it was trained on
+It contains 3 different models that vary in transformer type and data it was
+trained on
| Model name | Transformer type | Data from |
| :----------: | :---------------: | :----------------------------------------: |
@@ -12,19 +14,20 @@ It contains 3 different models that vary in transformer type and data it was tra
| unbiased | roberta-base | Unintended Bias in Toxicity Classification |
| multilingual | xlm-roberta-base | Multilingual Toxic Comment Classification |
-Unbiased and original models also have a 'small' version - but since normal models are not memory heavy, and small models perform noticably worse, they are only described in the notebook
+Unbiased and original models also have a 'small' version - but since normal
+models are not memory heavy, and small models perform noticably worse, they are
+only described in the notebook
## All tests below were ran on a 3090TI
# Inference and training times and memory usages
-Charts showing detailed memory usages and times for different sentence lengths and batch sizes are inside the notebook
-Quick overview batch size 16, sentence length 4k for training, batch size 128 sentence length 4k for inference
-| Model name | Training memory| Training speed | Inference Memory| Inference Speed|
-| :---: | :---: | :---: |:---: | :---: |
-|original| 11.8GB | 2.40s| 4.8GB|16.48s|
-|unbiased| 12GB| 1.09s| 4.8GB | 5.59s|
-|multilingual|14GB| 1.00s| 5.5GB| 4.89s|
+Charts showing detailed memory usages and times for different sentence lengths
+and batch sizes are inside the notebook Quick overview batch size 16, sentence
+length 4k for training, batch size 128 sentence length 4k for inference | Model
+name | Training memory| Training speed | Inference Memory| Inference Speed| |
+:---: | :---: | :---: |:---: | :---: | |original| 11.8GB | 2.40s| 4.8GB|16.48s|
+|unbiased| 12GB| 1.09s| 4.8GB | 5.59s| |multilingual|14GB| 1.00s| 5.5GB| 4.89s|
# Filtering quality
@@ -45,9 +48,13 @@ Detoxify was tested on 4 different types of inputs
Subjectivly 'unbiased' looks like the best performing model.
-I don't think it would do well as a security layer in a live version of open assistant unless we do some finetuning first, because it can be fooled to pass toxicity if it's presented in formal language.
+I don't think it would do well as a security layer in a live version of open
+assistant unless we do some finetuning first, because it can be fooled to pass
+toxicity if it's presented in formal language.
-With some caution it can be used to filter prompts but I would suggest also using someone for verification of messages that are marked as toxic but still below 90% confidence
+With some caution it can be used to filter prompts but I would suggest also
+using someone for verification of messages that are marked as toxic but still
+below 90% confidence
# Licensing
@@ -85,7 +92,8 @@ This is obviously not legal advice.
# Hosting
-The model is currently available on [huggingface](https://huggingface.co/unitary) and torch hub
+The model is currently available on
+[huggingface](https://huggingface.co/unitary) and torch hub
```
torch.hub.load('unitaryai/detoxify',model)
diff --git a/oasst-shared/oasst_shared/schemas/protocol.py b/oasst-shared/oasst_shared/schemas/protocol.py
index 8a6685c2..5f05adc3 100644
--- a/oasst-shared/oasst_shared/schemas/protocol.py
+++ b/oasst-shared/oasst_shared/schemas/protocol.py
@@ -5,7 +5,7 @@ from typing import Literal, Optional, Union
from uuid import UUID, uuid4
import pydantic
-from pydantic import BaseModel
+from pydantic import BaseModel, Field
class TaskRequestType(str, enum.Enum):
@@ -56,7 +56,9 @@ class TaskRequest(BaseModel):
"""The frontend asks the backend for a task."""
type: TaskRequestType = TaskRequestType.random
- user: Optional[User] = None
+ # Must use Field(..., nullable=True) to indicate to the OpenAPI schema that
+ # this is optional. https://github.com/pydantic/pydantic/issues/1270
+ user: Optional[User] = Field(None, nullable=True)
collective: bool = False
diff --git a/redis.conf b/redis.conf
new file mode 100644
index 00000000..58da1e05
--- /dev/null
+++ b/redis.conf
@@ -0,0 +1,2 @@
+maxmemory 100mb
+maxmemory-policy allkeys-lru
diff --git a/scripts/backend-development/README.md b/scripts/backend-development/README.md
index ef2ac0bf..d5b3ccc5 100644
--- a/scripts/backend-development/README.md
+++ b/scripts/backend-development/README.md
@@ -1,6 +1,12 @@
# Backend Development Setup
-In root directory, run `docker compose up backend-dev --build --attach-dependencies` to start a database. The default settings are already configured to connect to the database at `localhost:5432`.
+In root directory, run
+`docker compose up backend-dev --build --attach-dependencies` to start a
+database. The default settings are already configured to connect to the database
+at `localhost:5432`.
-Make sure you have all requirements installed. You can do this by running `pip install -r requirements.txt` inside the `backend` folder and `pip install -e .` inside the `oasst-shared` folder.
-Then, run the backend using the `run-local.sh` script. This will start the backend server at `http://localhost:8080`.
+Make sure you have all requirements installed. You can do this by running
+`pip install -r requirements.txt` inside the `backend` folder and
+`pip install -e .` inside the `oasst-shared` folder. Then, run the backend using
+the `run-local.sh` script. This will start the backend server at
+`http://localhost:8080`.
diff --git a/scripts/backend-development/start-mock-server.sh b/scripts/backend-development/start-mock-server.sh
new file mode 100755
index 00000000..35a202a6
--- /dev/null
+++ b/scripts/backend-development/start-mock-server.sh
@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+parent_path=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P )
+
+# switch to backend directory
+pushd "$parent_path/../../backend"
+
+MOCK_SERVER_PORT=8080
+OPENAPI_JSON_FILE_NAME=openapi.json
+
+echo "Generating OpenAPI schema..."
+python -m main --print-openapi-schema > $OPENAPI_JSON_FILE_NAME
+echo "Done!"
+
+# If oasst-mock-backend docker container is already running,
+# just restart it
+if [ "$(docker ps -q -f name=oasst-mock-backend)" ]; then
+ echo "oasst-mock-backend container exists, restarting..."
+ docker restart oasst-mock-backend
+else
+ echo "Creating new oasst-mock-backend container..."
+ docker run --init --rm -d \
+ --name oasst-mock-backend \
+ -p $MOCK_SERVER_PORT:4010 \
+ -v $(pwd):/tmp \
+ -P stoplight/prism:4 \
+ mock -h 0.0.0.0 "/tmp/$OPENAPI_JSON_FILE_NAME"
+fi
+
+echo "Waiting for server to be live..."
+curl --retry-all-errors --retry 5 localhost:$MOCK_SERVER_PORT
+echo ""
+
+# if return code is successful, print successful response
+if [ $? -eq 0 ]; then
+ echo "Mock server is running at localhost:$MOCK_SERVER_PORT"
+else
+ echo "Mock server failed to start"
+fi
+
+
+popd
diff --git a/scripts/backend-development/stop-mock-server.sh b/scripts/backend-development/stop-mock-server.sh
new file mode 100755
index 00000000..20248aaa
--- /dev/null
+++ b/scripts/backend-development/stop-mock-server.sh
@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+docker stop oasst-mock-backend
diff --git a/scripts/discord-bot-development/test.sh b/scripts/discord-bot-development/test.sh
new file mode 100755
index 00000000..a45adf00
--- /dev/null
+++ b/scripts/discord-bot-development/test.sh
@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+parent_path=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P )
+
+# switch to backend directory
+pushd "$parent_path/../../discord-bot"
+
+pytest .
+
+popd
diff --git a/scripts/frontend-development/README.md b/scripts/frontend-development/README.md
index 05349fb9..3ac2a258 100644
--- a/scripts/frontend-development/README.md
+++ b/scripts/frontend-development/README.md
@@ -1,5 +1,8 @@
# Frontend Development Setup
-In root directory run `docker compose up frontend-dev --build --attach-dependencies` to start a database and the backend server.
+In root directory run
+`docker compose up frontend-dev --build --attach-dependencies` to start a
+database and the backend server.
-Then, point your frontend at `http://localhost:8080` to start developing. During development, any API key will be accepted.
+Then, point your frontend at `http://localhost:8080` to start developing. During
+development, any API key will be accepted.
diff --git a/website/README.md b/website/README.md
index f70dcfce..5198a820 100644
--- a/website/README.md
+++ b/website/README.md
@@ -26,8 +26,8 @@ This website is built using:
development.
1. [Prisma](https://www.prisma.io/): An ORM to interact with a web specific
[Postgres](https://www.postgresql.org/) database.
-1. [NextAuth.js](https://next-auth.js.org/): A user authentication framework
- to ensure we handle accounts with best practices.
+1. [NextAuth.js](https://next-auth.js.org/): A user authentication framework to
+ ensure we handle accounts with best practices.
1. [TailwindCSS](https://tailwindcss.com/): A general purpose framework for
styling any component.
1. [Chakra-UI](https://chakra-ui.com/): A wide collection of pre-built UI
@@ -38,10 +38,10 @@ This website is built using:
To contribute to the website, make sure you have the following setup and
installed:
-1. [NVM](https://github.com/nvm-sh/nvm): The Node Version Manager makes it
- easy to ensure you have the right NodeJS version installed. Once installed,
- run `nvm use 16` to use Node 16.x. The website is known to be stable with
- NodeJS version 16.x. This will install both Node and NPM.
+1. [NVM](https://github.com/nvm-sh/nvm): The Node Version Manager makes it easy
+ to ensure you have the right NodeJS version installed. Once installed, run
+ `nvm use 16` to use Node 16.x. The website is known to be stable with NodeJS
+ version 16.x. This will install both Node and NPM.
1. [Docker](https://www.docker.com/): We use docker to simplify running
dependent services.
@@ -50,8 +50,8 @@ installed:
If you're doing active development we suggest the following workflow:
1. In one tab, navigate to the project root.
-1. Run `docker compose up frontend-dev --build --attach-dependencies`. You can optionally include `-d` to detach and
- later track the logs if desired.
+1. Run `docker compose up frontend-dev --build --attach-dependencies`. You can
+ optionally include `-d` to detach and later track the logs if desired.
1. In another tab navigate to `${OPEN_ASSISTANT_ROOT/website`.
1. Run `npm install`
1. Run `npx prisma db push` (This is also needed when you restart the docker
@@ -64,17 +64,25 @@ If you're doing active development we suggest the following workflow:
### Using debug user credentials
-You can use the debug credentials provider to log in without fancy emails or OAuth.
+You can use the debug credentials provider to log in without fancy emails or
+OAuth.
-1. This feature is automatically on in development mode, i.e. when you run `npm run dev`. In case you want to do the same with a production build (for example, the docker image), then run the website with environment variable `DEBUG_LOGIN=true`.
+1. This feature is automatically on in development mode, i.e. when you run
+ `npm run dev`. In case you want to do the same with a production build (for
+ example, the docker image), then run the website with environment variable
+ `DEBUG_LOGIN=true`.
1. Use the `Login` button in the top right to go to the login page.
-1. You should see a section for debug credentials. Enter any username you wish, you will be logged in as that user.
+1. You should see a section for debug credentials. Enter any username you wish,
+ you will be logged in as that user.
### Using Storybook
-To develop components using [Storybook](https://storybook.js.org/) run `npm run storybook`. Then navigate to in your browser to `http://localhost:6006`.
+To develop components using [Storybook](https://storybook.js.org/) run
+`npm run storybook`. Then navigate to in your browser to
+`http://localhost:6006`.
-To create a new story create a file named `[componentName].stories.js`. An example how such a story could look like, see `Header.stories.jsx`.
+To create a new story create a file named `[componentName].stories.js`. An
+example how such a story could look like, see `Header.stories.jsx`.
## Code Layout
@@ -82,11 +90,12 @@ To create a new story create a file named `[componentName].stories.js`. An examp
All react code is under `src/` with a few sub directories:
-1. `pages/`: All pages a user could navigate too and API URLs which are under `pages/api/`.
-1. `components/`: All re-usable React components. If something gets used
- twice we should create a component and put it here.
-1. `lib/`: A generic place to store library files that are used anywhere.
- This doesn't have much structure yet.
+1. `pages/`: All pages a user could navigate too and API URLs which are under
+ `pages/api/`.
+1. `components/`: All re-usable React components. If something gets used twice
+ we should create a component and put it here.
+1. `lib/`: A generic place to store library files that are used anywhere. This
+ doesn't have much structure yet.
NOTE: `styles/` can be ignored for now.
@@ -104,16 +113,27 @@ We're not really using CSS styles. `styles/` can be ignored.
## Testing the UI
-Cypress is used for end-to-end (e2e) and component testing and is configured in `./cypress.config.ts`. The `./cypress` folder is used for supporting configuration files etc.
+Cypress is used for end-to-end (e2e) and component testing and is configured in
+`./cypress.config.ts`. The `./cypress` folder is used for supporting
+configuration files etc.
- Store e2e tests in the `./cypress/e2e` folder.
-- Store component tests adjacent to the component being tested. If you want to wriite a test for `./src/components/Layout.tsx` then store the test file at `./src/components/Layout.cy.tsx`.
+- Store component tests adjacent to the component being tested. If you want to
+ wriite a test for `./src/components/Layout.tsx` then store the test file at
+ `./src/components/Layout.cy.tsx`.
A few npm scripts are available for convenience:
-- `npm run cypress`: Useful for development, it opens Cypress and allows you to explore, run and debug tests. It assumes you have the NextJS site running at `localhost:3000`.
-- `npm run cypress:run`: Runs all tests. Useful for a quick sanity check before sending a PR or to run in CI pipelines.
-- `npm run cypress:image-baseline`: If you have tests failing because of visual changes that was expected, this command will update the baseline images stored in `./cypress-visual-screenshots/baseline` with those from the adjacent comparison folder. More can be found in the [docs of `uktrade/cypress-image-diff`](https://github.com/uktrade/cypress-image-diff/blob/main/docs/CLI.md#update-all-baseline-images-for-failing-tests).
+- `npm run cypress`: Useful for development, it opens Cypress and allows you to
+ explore, run and debug tests. It assumes you have the NextJS site running at
+ `localhost:3000`.
+- `npm run cypress:run`: Runs all tests. Useful for a quick sanity check before
+ sending a PR or to run in CI pipelines.
+- `npm run cypress:image-baseline`: If you have tests failing because of visual
+ changes that was expected, this command will update the baseline images stored
+ in `./cypress-visual-screenshots/baseline` with those from the adjacent
+ comparison folder. More can be found in the
+ [docs of `uktrade/cypress-image-diff`](https://github.com/uktrade/cypress-image-diff/blob/main/docs/CLI.md#update-all-baseline-images-for-failing-tests).
Read more in the [./cypress README](cypress/).
@@ -125,9 +145,9 @@ When writing code for the website, we have a few best practices:
dependencies. Order them alphabetically according to the package name.
1. When trying to implement something new, check if
[Chakra-UI](https://chakra-ui.com/) has components that are close enough to
- your need. For example Sliders, Radio Buttons, Progress indicators, etc. They
- have a lot and we can save time by re-using what they have and tweaking the
- style as needed.
+ your need. For example Sliders, Radio Buttons, Progress indicators, etc.
+ They have a lot and we can save time by re-using what they have and tweaking
+ the style as needed.
1. Format everything with [Prettier](https://prettier.io/). This is done by
default with pre-submits. We currently don't have any custom settings.
1. Define functional React components (with types for all properties when
@@ -135,14 +155,15 @@ When writing code for the website, we have a few best practices:
### URL Paths
-To use stable and consistent URL paths, we recommend the following strategy for new tasks:
+To use stable and consistent URL paths, we recommend the following strategy for
+new tasks:
1. For any task that involves writing a free-form response, put the page under
`website/src/pages/create` with a page name matching the task type, such as
`summarize_story.tsx`.
1. For any task that evaluates, rates, or ranks content, put the page under
- `website/src/pages/evaluate` with a page name matching the task type such
- as `rate_summary.tsx`.
+ `website/src/pages/evaluate` with a page name matching the task type such as
+ `rate_summary.tsx`.
With this we'll be able to ensure these contribution pages are hidden from
logged out users but accessible to logged in users.
@@ -151,5 +172,6 @@ logged out users but accessible to logged in users.
To learn more about Next.js, take a look at the following resources:
-- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
+- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js
+ features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
diff --git a/website/cypress/README.md b/website/cypress/README.md
index 12a32378..7f2c5d53 100644
--- a/website/cypress/README.md
+++ b/website/cypress/README.md
@@ -1,14 +1,24 @@
# Component and e2e testing with Cypress
-[Cypress](https://www.cypress.io/) is used for both component- and end-to-end testing. Below there's a few examples for the context of this site. To learn more, the [Cypress documentation](https://docs.cypress.io/guides/getting-started/opening-the-app) has it all.
+[Cypress](https://www.cypress.io/) is used for both component- and end-to-end
+testing. Below there's a few examples for the context of this site. To learn
+more, the
+[Cypress documentation](https://docs.cypress.io/guides/getting-started/opening-the-app)
+has it all.
-Don't get scared by the commercial offerings they offer. Their core is open source, the cloud offering is not necesarry at all and can be replaced by CI tooling and [community efforts](https://sorry-cypress.dev/).
+Don't get scared by the commercial offerings they offer. Their core is open
+source, the cloud offering is not necesarry at all and can be replaced by CI
+tooling and [community efforts](https://sorry-cypress.dev/).
# Component testing
-To write a new component test, you either create a new `.tsx` adjacent to the component you want to test or you can use the guide presented yo you when running `npm run cypress` which allows you to easily create the skeleton test for an existing component.
+To write a new component test, you either create a new `.tsx` adjacent to the
+component you want to test or you can use the guide presented yo you when
+running `npm run cypress` which allows you to easily create the skeleton test
+for an existing component.
-If you have a `Button.tsx` component, create a file next to it called `Button.cy.tsx` which could look like this:
+If you have a `Button.tsx` component, create a file next to it called
+`Button.cy.tsx` which could look like this:
```typescript
import React from "react";
@@ -25,17 +35,28 @@ describe("", () => {
## What's happening here?
-First we use `cy.mount` to mount our component under test. Notive how we specify `className` and inner text - this is where we arrange our component with fake data that we could assert on later.
+First we use `cy.mount` to mount our component under test. Notive how we specify
+`className` and inner text - this is where we arrange our component with fake
+data that we could assert on later.
-In the example above, we also use `cy.get` to select the rendered `button` element. Cypress has multiple ways to [select elements](https://docs.cypress.io/guides/references/best-practices), `get` is just one of them (and often not recommended).
+In the example above, we also use `cy.get` to select the rendered `button`
+element. Cypress has multiple ways to
+[select elements](https://docs.cypress.io/guides/references/best-practices),
+`get` is just one of them (and often not recommended).
-At last, we use `captureSnapshot` which is a plugin that snaps a photo of the `button` element and compares it to a baseline located in the `./cypress-visual-screenshots/baseline/` folder. If there's too many unidentical pixels between the two, it will fail the test.
+At last, we use `captureSnapshot` which is a plugin that snaps a photo of the
+`button` element and compares it to a baseline located in the
+`./cypress-visual-screenshots/baseline/` folder. If there's too many unidentical
+pixels between the two, it will fail the test.
# End-to-end (e2e) testing
-e2e tests are stored in the `./cypress/e2e` folder and should be named `{page}.cy.ts` and located in a relative folder structure that mirrors the page under test.
+e2e tests are stored in the `./cypress/e2e` folder and should be named
+`{page}.cy.ts` and located in a relative folder structure that mirrors the page
+under test.
-When running `npm run cypress` and selecting e2e testing, we assume you have the NextJS site running at `localhost:3000`.
+When running `npm run cypress` and selecting e2e testing, we assume you have the
+NextJS site running at `localhost:3000`.
An example test from this time of writing, could look as follows:
@@ -53,10 +74,18 @@ export {};
## What's happening here?
-First we use [`cy.visit`](https://docs.cypress.io/api/commands/visit) to point the browser at the desired page. It appends relative paths to the configured `baseUrl` (found in `./cypress.config.ts`).
+First we use [`cy.visit`](https://docs.cypress.io/api/commands/visit) to point
+the browser at the desired page. It appends relative paths to the configured
+`baseUrl` (found in `./cypress.config.ts`).
-Cypress will [automatically await](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress#Timeouts) almost anything you do, but fail if the default timeout is reached.
+Cypress will
+[automatically await](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress#Timeouts)
+almost anything you do, but fail if the default timeout is reached.
-Then we get the email input field and type our email address. Notice the `{enter}` keyword, this will cause Cypress to hit the return key which we expect to submit the form.
+Then we get the email input field and type our email address. Notice the
+`{enter}` keyword, this will cause Cypress to hit the return key which we expect
+to submit the form.
-We then assert that the URL should contain `/auth/verify`. Again the timeout will make sure we are not waiting forever, and the test will fail if we do not manage to get there in a reasonable time.
+We then assert that the URL should contain `/auth/verify`. Again the timeout
+will make sure we are not waiting forever, and the test will fail if we do not
+manage to get there in a reasonable time.
diff --git a/website/next-lint.js b/website/next-lint.js
new file mode 100755
index 00000000..0b3a5c90
--- /dev/null
+++ b/website/next-lint.js
@@ -0,0 +1,24 @@
+#!/usr/bin/env node
+const { spawnSync } = require("child_process");
+async function npmLint() {
+ const spawnOption = {
+ shell: true,
+ env: process.env,
+ stdio: "inherit",
+ cwd: "./website",
+ };
+ let npmInstall;
+ let npmRunLint;
+ try {
+ npmInstall = await spawnSync("npm", ["install"], spawnOption);
+ if (npmInstall.status !== 0) {
+ process.exit(npmInstall.status);
+ }
+ npmRunLint = await spawnSync("npm", ["run lint"], spawnOption);
+ process.exit(npmRunLint.status);
+ } catch (error) {
+ console.error(error);
+ process.exit(1);
+ }
+}
+npmLint();
diff --git a/website/src/components/Footer.tsx b/website/src/components/Footer.tsx
index a07ba24a..5c774398 100644
--- a/website/src/components/Footer.tsx
+++ b/website/src/components/Footer.tsx
@@ -20,24 +20,21 @@ export function Footer() {