export and auto-commit docs/docs/api/openapi.json as part of test-api-contract workflow (#719)

- copies that openapi.json file to docs/docs/api/openapi.json as part of start-mock-server.sh.
- adds a step to the end of the test-api-contract workflow that just auto-commits changes to the docs/docs/api/openapi.json file.
- excludes docs/docs/api/openapi.json from pre-commit since its already been formatted by jq.

.github/workflows/test-api-contract.yaml

@@ -4,6 +4,7 @@ on:
   push:
     branches:
       - main
+      - add-api-docs-workflow
     paths:
       - "oasst-shared/**"
       - "backend/**"
@@ -44,3 +45,9 @@ jobs:
         run: ./scripts/frontend-development/run-contract-test.sh
 
       - run: ./scripts/backend-development/stop-mock-server.sh
+
+      - uses: stefanzweifel/git-auto-commit-action@v4
+        with:
+          file_pattern: "docs/docs/api/openapi.json"
+          commit_message:
+            update docs/docs/api/openapi.json by run ${{ github.run_id }}

.pre-commit-config.yaml

@@ -26,7 +26,7 @@
 #
 # /WARNING!
 
-exclude: build|stubs|^bot/templates/$|openassistant/templates
+exclude: build|stubs|^bot/templates/$|openassistant/templates|docs/docs/api/openapi.json
 
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks

backend/README.md

@@ -59,3 +59,6 @@ without having to actually set up and run a development backend.
 # save openapi.json to docs/docs/api
 wget localhost:8080/api/v1/openapi.json -O docs/docs/api/openapi.json
 ```
+
+Note: The api docs should be automatically updated by the
+`test-api-contract.yaml` workflow.

notebooks/knowledge_extraction/T5_closed_book_QA_generators.md

@@ -1,6 +1,8 @@
 # Generate Topics, Questions, and Answers from a paragraph of text
 
-This python code can be used to generate topics, questions, and answers from a paragraph of text. This is a good way to generate ground truth knowledge about a topic from a trusted source.
+This python code can be used to generate topics, questions, and answers from a
+paragraph of text. This is a good way to generate ground truth knowledge about a
+topic from a trusted source.
 
 The output of this is a dictionary with the following information:
 
@@ -9,8 +11,12 @@ The output of this is a dictionary with the following information:
 3. generated questions
 4. generated topic prefixes that can be prepended to the questions
 5. open book answer based only on the provided paragraph
-6. closed book answers generated by FLAN-T5-11B (uses only question and optionally question prefix to generate the answer)
+6. closed book answers generated by FLAN-T5-11B (uses only question and
+   optionally question prefix to generate the answer)
 
 ## Contributing
 
-This code is verified to work on a 24GB vram graphics card (like an RTX3090). We are working on getting it to run on google colab TPUs and also it may be possible to use smaller T5 models like the 3 billion parameter model and still get acceptable results.
+This code is verified to work on a 24GB vram graphics card (like an RTX3090). We
+are working on getting it to run on google colab TPUs and also it may be
+possible to use smaller T5 models like the 3 billion parameter model and still
+get acceptable results.

notebooks/knowledge_extraction/T5_closed_book_QA_generators.py

@@ -1,4 +1,3 @@
-# -*- coding: utf-8 -*-
 # This notebook will run on a system with a single RTX3090 (24 GB vram) GPU.
 # You need to install accelerate, bitsandbytes, and transformers
 

scripts/backend-development/start-mock-server.sh

@@ -11,6 +11,9 @@ echo "Generating OpenAPI schema..."
 python -m main --print-openapi-schema > $OPENAPI_JSON_FILE_NAME
 echo "Done!"
 
+echo "Formatting & Copying OpenAPI schema to docs directory..."
+jq . $OPENAPI_JSON_FILE_NAME > ../docs/docs/api/openapi.json
+
 # If oasst-mock-backend docker container is already running,
 # just restart it
 if [ "$(docker ps -q -f name=oasst-mock-backend)" ]; then