Commit Graph

22 Commits

Author SHA1 Message Date
Andreas Köpf ead51ff423 770 tree manager allow to specify desired task_type no prompter ranking (#775)
* only ranking assistant replies by default

* add tasks/availability endpoint allow to specify desired task

* move rank_prompter_replies option to TreeManagerConfiguration

* fix type annotation

* remove desired_task_type from _random_task_selection()

* fix typo

* Convert query_tree_size to sqlachemy, return 'full' text-labeling tasks if they were explicitly requested
2023-01-16 20:05:40 +01:00
Andreas Köpf 7f562fbbae add managed_tx_function() decorator and use it for startup db-calls 2023-01-16 09:16:28 +01:00
Andreas Köpf a42f2cd3ed add new debug option 'DEBUG_DATABASE_ECHO' (default False) 2023-01-16 08:46:45 +01:00
Yannic Kilcher 5d441b1570 prepared playbook for prod 2023-01-15 23:30:16 +01:00
Yannic Kilcher cc03376d86 added root tokens and endpoint for adding api keys (#742)
* added root tokens and endpoint for adding api keys

* Change down revision to current alembic head

* removed added_by_root_token

* refactored description

* fixed jinja errors

Co-authored-by: Andreas Köpf <andreas.koepf@provisio.com>
2023-01-15 21:24:25 +00:00
Andreas Köpf b5bb5bb7c0 Add leaderboard stats, periodic updates via fastapi-utils (#724)
* add leaderboard stats, periodic update via fastapi-utils

* count label tasks for assistant and prompter replies

* Daily stats update every 15 mins, simplify leaderboard endpoint

* add indices for some created_date columns

* make user stats update intervals configurable

* make sure intervals are positive
2023-01-15 12:04:19 +01:00
kayjay 5cd9c630fc Small typo fix + Consistent Docstrings (#698)
* Small typo fix + Consistent Docstrings

Also removed a redundant comment.

* undo comment removal

* Fixed indentation from previous commit
2023-01-14 17:24:02 +01:00
Nil Andreu a902c600fa Store Message Toxicity in database (#553)
* [NEW] MessageToxicity table

* [NEW] Alembic message Toxicity

* [NEW] Model name enum

* [NEW] Refactor Enum HF

* [NEW] Settings: DEBUT_SKIP_TOXICITY_CALCULATION

* [NEW] Store toxicity values

* [FIX] Merge conflict

* [FIX] Documentation

* [NEW] save_toxicity: function

* [FIX] Formatted string

* [NEW] DEBUG_SKIP_TOXICITY_CALCULATION=True

* [FIX] HfClassificationModel

* [FIX] Alembic merge heads

* [NEW] Refactor save_toxicity

* [NEW] Separating score/label

* [NEW] Store score and label

* [FIX] Cleaning Alembic

* [NEW] Clean HF names

* [NEW] Not type hinting

* [NEW] Update alembic versions

* [NEW] Revert the changes

* [NEW] Type hinting label & score

* Updated down_revision in migration script

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-14 12:22:55 +00:00
Andreas Koepf d0ef7c4bbc add missing doc-strings for tree-manager mandatory labels settings 2023-01-13 16:04:06 +00:00
Andreas Koepf 343853fb57 add TreeManagerConfiguration as sub-module to global pydantic settings class 2023-01-13 15:52:09 +00:00
dhug 9421488619 Add realistic seed data json and update seed_data to properly label task payloads (#613)
* modifies seed_data script to properly label taks types as prompteror assistant

* added realistic data json

* pre-commit changes

* update realistic_seed_data.json 5000 messages -> 450 messages

* pre-commit changes

* modifies seed_data script to properly label taks types as prompteror assistant

* added realistic data json

* pre-commit changes

* update realistic_seed_data.json 5000 messages -> 450 messages

* pre-commit changes

* changed DEBUG_USE_SEED_DATA_PATH to realistic_seed_data, merged with main, ran pre-commit

* Delete 2023_01_10_2052-06cc56417e69_.py

Co-authored-by: Andreas Köpf <andreas.koepf@provisio.com>
2023-01-11 18:48:14 +01:00
Andreas Köpf 14fa08e2e7 Message tree state machine (#555)
* add query_incomplete_rankings()

* Add SQL queries for TreeManager task selection

* first working version of TreeManager.next_task()

* remove old generate_task(), add mandatory_labels to text_labels task

* Add ConversationMessage list to Ranking tasks

* add more sophisticated sql queries to find extendible trees

* add TreeManager.query_extendible_parents()

* fix task validation, seed data insertion (reviewed)

* provide user for task selection in text-frontend

* enter 'growing' state

* enter 'aborted_low_grade' state

* enter 'ranking' state

* check tree 'growing' state upon relpy insertion

* exclude user from labeling their own messages (added DEBUG_ALLOW_SELF_LABELING setting)

* add DEBUG_ALLOW_SELF_LABELING to docker-compose.yaml

* fix ranking submission

* add query_tree_ranking_results()

* add ranked_message_ids to RankingReactionPayload

* fix reply_messages instead of prompt_messages

* incorment 'ranking_count' of ranked replies

* added logic to check_condition_for_scoring_state

* changes to msg_tree_state_machine

* pre-commit changes

* enter 'ready_for_scoring' state

* re-add HF embedding call (lost during merge)

* use prepare_conversation() helper for seed-data creation

* Partially add user specified task selection

Co-authored-by: Daniel Hug <danielpatrickhug@gmail.com>
2023-01-11 10:54:03 +01:00
jojopirker 11d55d572a message embeddings in Messages table 2023-01-08 12:28:38 +01:00
Ken Tsui 043b5eff5a Add DEBUG_USD_SEED_DATA_PATH in config to make seed data flexible (#395)
* Add DEBUG_USD_SEED_DATA_PATH in config to make seed data flexible

* reformat

* Copy test_data folder in Dockerfile.backend, correct DEBUG_USE_SEED_DATA_PATH in cofig

* - make DEBUG_USE_SEED_DATA_PATH to absolute path
- correct test_data path in  Dockerfile.backend
2023-01-07 14:37:30 +01:00
Nil Andreu ee50b573e1 Detoxify Rest API Client (#404)
* [NEW] utils: Endpoint Toxic Roberta

* [NEW] Constants API URL

* [NEW] Git ignore venv

* [NEW] Lint

* [NEW] Backend default args

* [NEW] HUGGINGFACE_API_ERROR

* [NEW] Requests package

* [NEW] Get Toxicity Endpoint

* [NEW] Schema:  ToxicityClassification

[NEW] Constants module

[FIX] Module

* [FIX] Test Key HF

* [NEW] settings: HUGGING_FACE_API_KEY

* [NEW] Remove requests

* [NEW] HuggingFace client

* [NEW] Cleaning code
2023-01-05 16:18:04 +01:00
kiritowu 2a2968f6d4 Initialise redis connection for Fastapi limiter 2023-01-03 00:28:05 +08:00
Gareth Davidson 7000e10bc0 apply pre-commit rules 2023-01-02 00:01:45 +00:00
Andreas Köpf 30db7254c7 rename flag to DEBUG_USE_SEED_DATA default False, update compose config 2022-12-29 15:58:35 +01:00
Andreas Köpf c2cb2562d5 add db seed-data check on backend startup 2022-12-29 15:58:35 +01:00
Yannic Kilcher 703c1129dd re-introduced ALLOW_ANY_API_KEY 2022-12-25 18:24:49 +01:00
Yannic Kilcher 21f44d2d4c removed ALLOW_ANY_API_KEY and allow skipping API key check in debug 2022-12-25 17:25:30 +01:00
Yannic Kilcher b34ef75423 started re-working python code 2022-12-19 18:00:09 +01:00