Commit Graph

43 Commits

Author SHA1 Message Date
Andreas Köpf e58ffd64fa add /api/v1/users/{user_id}/stats endpoint (#744)
* add /api/v1/users/{user_id}/stats endpoint

* return 0 stats and add /api/v1/users/{user_id}/stats/{time_frame}

* use utcnow() as modified date for 0 stats
2023-01-15 20:24:15 +00:00
Andreas Köpf b5bb5bb7c0 Add leaderboard stats, periodic updates via fastapi-utils (#724)
* add leaderboard stats, periodic update via fastapi-utils

* count label tasks for assistant and prompter replies

* Daily stats update every 15 mins, simplify leaderboard endpoint

* add indices for some created_date columns

* make user stats update intervals configurable

* make sure intervals are positive
2023-01-15 12:04:19 +01:00
Andreas Köpf 6547058778 add rank to message table and fix message_toxicity migration 2023-01-14 15:12:10 +01:00
Nil Andreu a902c600fa Store Message Toxicity in database (#553)
* [NEW] MessageToxicity table

* [NEW] Alembic message Toxicity

* [NEW] Model name enum

* [NEW] Refactor Enum HF

* [NEW] Settings: DEBUT_SKIP_TOXICITY_CALCULATION

* [NEW] Store toxicity values

* [FIX] Merge conflict

* [FIX] Documentation

* [NEW] save_toxicity: function

* [FIX] Formatted string

* [NEW] DEBUG_SKIP_TOXICITY_CALCULATION=True

* [FIX] HfClassificationModel

* [FIX] Alembic merge heads

* [NEW] Refactor save_toxicity

* [NEW] Separating score/label

* [NEW] Store score and label

* [FIX] Cleaning Alembic

* [NEW] Clean HF names

* [NEW] Not type hinting

* [NEW] Update alembic versions

* [NEW] Revert the changes

* [NEW] Type hinting label & score

* Updated down_revision in migration script

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-14 12:22:55 +00:00
Andreas Köpf 945c645818 returning list[FrontEndUser] from api/v1/frontend_users/ endpoint 2023-01-12 22:58:43 +01:00
Andreas Köpf f264b43cde fix user's notes column server_default value 2023-01-12 21:43:33 +01:00
Oliver Stanley 050d4902f3 538: Add endpoints to manage users (#601)
* Add endpoints for getting, updating, deleting users by global user ID

* Resolve formatting

* Include alembic revision script

* Updated down_revision to current alembic head

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-12 21:30:07 +01:00
Andreas Köpf b659e1cf01 remove accepted_messages from message_tree_state 2023-01-12 01:21:32 +01:00
Andreas Köpf 14fa08e2e7 Message tree state machine (#555)
* add query_incomplete_rankings()

* Add SQL queries for TreeManager task selection

* first working version of TreeManager.next_task()

* remove old generate_task(), add mandatory_labels to text_labels task

* Add ConversationMessage list to Ranking tasks

* add more sophisticated sql queries to find extendible trees

* add TreeManager.query_extendible_parents()

* fix task validation, seed data insertion (reviewed)

* provide user for task selection in text-frontend

* enter 'growing' state

* enter 'aborted_low_grade' state

* enter 'ranking' state

* check tree 'growing' state upon relpy insertion

* exclude user from labeling their own messages (added DEBUG_ALLOW_SELF_LABELING setting)

* add DEBUG_ALLOW_SELF_LABELING to docker-compose.yaml

* fix ranking submission

* add query_tree_ranking_results()

* add ranked_message_ids to RankingReactionPayload

* fix reply_messages instead of prompt_messages

* incorment 'ranking_count' of ranked replies

* added logic to check_condition_for_scoring_state

* changes to msg_tree_state_machine

* pre-commit changes

* enter 'ready_for_scoring' state

* re-add HF embedding call (lost during merge)

* use prepare_conversation() helper for seed-data creation

* Partially add user specified task selection

Co-authored-by: Daniel Hug <danielpatrickhug@gmail.com>
2023-01-11 10:54:03 +01:00
Nil-Andreu 70620520b4 [NEW] Created date 2023-01-08 21:29:12 +01:00
jojopirker a677e40cff insert embedding now to new table 2023-01-08 16:46:53 +01:00
jojopirker 11d55d572a message embeddings in Messages table 2023-01-08 12:28:38 +01:00
jojopirker 9194e15b80 bugfix in alembic downgrade (frontend_type not frontend_id) 2023-01-08 10:59:12 +01:00
jojopirker 5fb0121668 #438 Frontend type (#483)
* add dev setup to readme & alembic description

* frontend_type for api_client

* readme update & missing file

* pre-commit
2023-01-07 21:33:57 +01:00
Oliver Stanley 69bc799cd9 344: Create tasks for text labels (#381)
* Implement label task for initial prompts and replies

* Resolve formatting

* Include missing argument

* Modify text_labels API to match new model, update DB schema accordingly

* Send valid labels as part of label tasks

* Send correctly formatted valid_labels list

* Fix request format

* Fix request details for text-frontend reply label task

* Include message_id in tasks

* Address review comments

* Fix alembic tree
2023-01-06 18:39:04 +01:00
Daniel Hug 935e556cf6 Add message tree state model (#414)
* inital commit, created file for MessageTreeState table

* added initial implementation of MessageTreeState SQl model and added it to__init__

* Ran Alembic revision for migration

* reran precommit on

* removed create_data and deleted flag from model

* ran migration revision to remove unwant class variables date_created and deleted

* removed unused imports

* ran pre-commit

* Updated States definiton to be Enums instead fo Constants

* ran pre-commit formatting

* Fixed Enum class

* ran pre-commit
2023-01-05 21:32:47 +01:00
Gareth Davidson 7000e10bc0 apply pre-commit rules 2023-01-02 00:01:45 +00:00
Igor Miagkov 6d98ba1f75 fix alembic migration 2022-12-31 10:31:31 +01:00
Igor Miagkov 13d01b5a2f management api 2022-12-31 10:31:31 +01:00
Andreas Köpf 5f16a8f780 make sure alembic migration works in both directions 2022-12-31 00:18:55 +01:00
Andreas Köpf 048802fa1a add alembic migration script 2022-12-31 00:18:55 +01:00
Igor Miagkov 7867466700 added trusted api client flag 2022-12-30 00:35:30 +01:00
Yannic Kilcher 3ab2e01b21 Merge pull request #128 from GraemeHarris/add-lang-column
Added column to post table for lang with US english set to default.
2022-12-29 21:33:41 +01:00
Igor Miagkov efafc0173a added support for collective tasks 2022-12-29 21:32:17 +04:00
Graeme Harris e5c32eda21 Updates from linting 2022-12-29 19:26:30 +02:00
GraemeHarris 8046a4a15b Merge branch 'main' into add-lang-column 2022-12-29 09:52:11 +02:00
Graeme Harris 00ebf90871 Added column to post table for lang with US english set to default. Included alembic migration 2022-12-28 23:25:29 +02:00
Igor Miagkov abf2fcc1c8 Migrations for task sampling 2022-12-28 22:23:16 +01:00
Andreas Köpf f74fe68f87 add journal table and JournalWriter helper class 2022-12-27 22:37:48 +01:00
Yannic Kilcher 78a7747e05 added docstring to migration and renamed file 2022-12-25 18:58:48 +01:00
Yannic Kilcher a37bf6bf41 added text labels to the API 2022-12-25 17:08:57 +01:00
Andreas Köpf 8a48722e72 first api-interaction, fix auth_method unique-index 2022-12-22 18:41:50 +01:00
Yannic Kilcher b34ef75423 started re-working python code 2022-12-19 18:00:09 +01:00
Yannic Kilcher 1acdc66973 moved alembic 2022-12-17 23:40:40 +01:00
Andreas Köpf afb5ec3669 change alembic file_template 2022-12-17 23:31:35 +01:00
Andreas Köpf 2a56438a55 add auth_method column to person table 2022-12-17 23:01:21 +01:00
Andreas Köpf 1f31e6a499 add frontend post_id binding and post reply handing 2022-12-16 13:32:06 +01:00
Andreas Köpf b38c14233f add polymorphic JSONB payload support
update nullable fields, new auth check
2022-12-16 02:08:43 +01:00
Andreas Köpf a827b7f19e add sqlmodel classes 2022-12-15 23:51:58 +01:00
Andreas Köpf 7d88ff4a8c use JSONB instead of Text of payload columns 2022-12-15 23:51:58 +01:00
Andreas Köpf e1a7ae42aa add v1 database alembic migration script 2022-12-15 23:51:58 +01:00
Yannic Kilcher 8dd9e16c33 Merge branch 'main' of github.com:LAION-AI/Open-Chat-GPT 2022-12-13 12:39:14 +01:00
Yannic Kilcher f407887501 renamed alemic to alembic 2022-12-13 11:57:54 +01:00