Commit Graph

74 Commits

Author SHA1 Message Date
Andreas Köpf 4886626904 fix alembic migration script 2023-02-02 19:41:34 +01:00
Oliver Stanley 652b7bff04 857: Implement backend Discord authentication (#943)
* Initial code for backend auth

* Remove outdated check

* Initial separation of AuthenticatedUser

* AuthenticatedUser -> Account

* Rework for Account

* Initial code for Discord OAuth

* Remove now redundant methods

* Remove incorrect response model, add requests dep for backend

* Create Settings fields for Discord values

* Cleanup get account from Discord function

* Cleanup

* Cleanup

* Generate alembic upgrade script

* Remove unused error codes

* Update alembic script to correct down revision

* Use aiohttp over requests

* Update alembic script to latest down revision
2023-02-02 19:33:21 +01:00
Andreas Köpf 2db3450e9a add trollboards 2023-02-02 16:10:38 +01:00
Andreas Köpf d33fb9ce88 fix alembic downgrade script of 7b8f0011e0b0 2023-02-02 00:15:49 +01:00
Andreas Köpf 779f522385 Skip tasks permanently based on special skip emoji codes (#1064)
* add skip emojis

* extend tree_manager queries to exclude tasks based on skip-emojis

* remove unrelated columns

* change skip_reason max_length to 512

* update down_revision to 55361f323d12
2023-02-02 00:07:08 +01:00
Andreas Köpf bbf038677c Add terms of service acceptance date to user table (#1046)
* add tos_acceptance_date column to user

* send 451 UNAVAILABLE_FOR_LEGAL_REASONS status

* add create user REST endpoint

* adapt text-frontend to ToS requirements

* set DEBUG_IGNORE_TOS_ACCEPTANCE default to True (temporary change)

* update down revision to f60958968ff8
2023-02-01 23:53:21 +01:00
Andreas Köpf e0df9f0b7c Add infty prompting + prompt-lottery (#1055)
* first bits of prompt-lottery

* add tree prompt lottery

* minor text changes
2023-02-01 23:49:21 +01:00
James Melvin Ebenezer 063157355c 906: update user streaks (#1016)
* fix: update user streaks
* Moved  streak_last_day_date & streak_days from UserStats to User
* Updated Alembic version
* update last_activity after tm.handle_interaction()
* periodically executed function in main.py to update the user_streak every 4hrs

* fix:removed log messages

* fix: pre commit issues

* refactor: incorporated review comments

* fix: removed the managed -scope.

* refactor: managed_tx to Outer REST Db Ops in /interaction API

* fix: pre-commit changes

* fix: added streak and last_activity_date to protocol.FrontEndUser

* fix: pre-commit fixes after merge

* fix: added streak info to user_stat_repository, leaderboard and review comments

* fix: proper 4h delay and simpler startup_time initialisation

---------

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-31 12:27:45 +01:00
Andreas Köpf c8d16285d0 Import message trees from jsonl file (#964)
* add new backlog_ranking tree state

* add first version of import script

* allow activation of trees during import

* add min_active_rankings_per_lang config param

* add settings docstring
2023-01-28 15:05:46 +01:00
Andreas Köpf 002f847406 add missing alembic migration script 2023-01-27 21:16:31 +01:00
Andreas Köpf f3ffde47ff add preferred lonely_children extension (#942)
* add preferred lonely_children extension

* simplify sampling process, lower the probability to 25%

* exclude parents for replies that were recently used

* lonely children := count > 0

* consider only tasks not done for parent exclusion

* increase lonely child sampling probability
2023-01-26 23:00:54 +01:00
Andreas Köpf 558b207013 Add /messages/{message_id}/emoji endpoint to toggle, add, remove message emojis (#925)
* add endpoint to set message emojis

* make refresh result optional in db utils
2023-01-25 09:31:20 +01:00
Andreas Köpf ffaf5c48d1 907 avoid duplicate labeling & ranking tasks (#923)
* store message_id in message_reactions and task_id in text_labels

* exclude tasks to which users already responded to

* remove test code

* fix join in find_incomplete_rankings_ex
2023-01-24 17:33:15 +01:00
Andreas Köpf 2d21b65ed0 Add lang-tag based task selection (lang-separation) (#863)
* lang based task selection

* use BCP 47 instead of ISO 639-1

* add Field(None, nullable=True)

* update migration script down_revision
2023-01-20 19:58:33 +01:00
Andreas Köpf 70fc80aa08 Add keyset pagination for users ordered by username / display_name (#851)
* add keyset pagination for user ordered by username or display_name

* add index on display-name for user table

* update down_revision in migration script
2023-01-20 16:32:13 +01:00
Andreas Köpf 038d1cc004 last_uddated for user_stats, swtich to timestamptz 2023-01-19 22:07:08 +01:00
Andreas Köpf e58ffd64fa add /api/v1/users/{user_id}/stats endpoint (#744)
* add /api/v1/users/{user_id}/stats endpoint

* return 0 stats and add /api/v1/users/{user_id}/stats/{time_frame}

* use utcnow() as modified date for 0 stats
2023-01-15 20:24:15 +00:00
Andreas Köpf b5bb5bb7c0 Add leaderboard stats, periodic updates via fastapi-utils (#724)
* add leaderboard stats, periodic update via fastapi-utils

* count label tasks for assistant and prompter replies

* Daily stats update every 15 mins, simplify leaderboard endpoint

* add indices for some created_date columns

* make user stats update intervals configurable

* make sure intervals are positive
2023-01-15 12:04:19 +01:00
Andreas Köpf 6547058778 add rank to message table and fix message_toxicity migration 2023-01-14 15:12:10 +01:00
Nil Andreu a902c600fa Store Message Toxicity in database (#553)
* [NEW] MessageToxicity table

* [NEW] Alembic message Toxicity

* [NEW] Model name enum

* [NEW] Refactor Enum HF

* [NEW] Settings: DEBUT_SKIP_TOXICITY_CALCULATION

* [NEW] Store toxicity values

* [FIX] Merge conflict

* [FIX] Documentation

* [NEW] save_toxicity: function

* [FIX] Formatted string

* [NEW] DEBUG_SKIP_TOXICITY_CALCULATION=True

* [FIX] HfClassificationModel

* [FIX] Alembic merge heads

* [NEW] Refactor save_toxicity

* [NEW] Separating score/label

* [NEW] Store score and label

* [FIX] Cleaning Alembic

* [NEW] Clean HF names

* [NEW] Not type hinting

* [NEW] Update alembic versions

* [NEW] Revert the changes

* [NEW] Type hinting label & score

* Updated down_revision in migration script

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-14 12:22:55 +00:00
Andreas Köpf 945c645818 returning list[FrontEndUser] from api/v1/frontend_users/ endpoint 2023-01-12 22:58:43 +01:00
Andreas Köpf f264b43cde fix user's notes column server_default value 2023-01-12 21:43:33 +01:00
Oliver Stanley 050d4902f3 538: Add endpoints to manage users (#601)
* Add endpoints for getting, updating, deleting users by global user ID

* Resolve formatting

* Include alembic revision script

* Updated down_revision to current alembic head

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-12 21:30:07 +01:00
Andreas Köpf b659e1cf01 remove accepted_messages from message_tree_state 2023-01-12 01:21:32 +01:00
Andreas Köpf 14fa08e2e7 Message tree state machine (#555)
* add query_incomplete_rankings()

* Add SQL queries for TreeManager task selection

* first working version of TreeManager.next_task()

* remove old generate_task(), add mandatory_labels to text_labels task

* Add ConversationMessage list to Ranking tasks

* add more sophisticated sql queries to find extendible trees

* add TreeManager.query_extendible_parents()

* fix task validation, seed data insertion (reviewed)

* provide user for task selection in text-frontend

* enter 'growing' state

* enter 'aborted_low_grade' state

* enter 'ranking' state

* check tree 'growing' state upon relpy insertion

* exclude user from labeling their own messages (added DEBUG_ALLOW_SELF_LABELING setting)

* add DEBUG_ALLOW_SELF_LABELING to docker-compose.yaml

* fix ranking submission

* add query_tree_ranking_results()

* add ranked_message_ids to RankingReactionPayload

* fix reply_messages instead of prompt_messages

* incorment 'ranking_count' of ranked replies

* added logic to check_condition_for_scoring_state

* changes to msg_tree_state_machine

* pre-commit changes

* enter 'ready_for_scoring' state

* re-add HF embedding call (lost during merge)

* use prepare_conversation() helper for seed-data creation

* Partially add user specified task selection

Co-authored-by: Daniel Hug <danielpatrickhug@gmail.com>
2023-01-11 10:54:03 +01:00
Nil-Andreu 70620520b4 [NEW] Created date 2023-01-08 21:29:12 +01:00
jojopirker a677e40cff insert embedding now to new table 2023-01-08 16:46:53 +01:00
jojopirker 11d55d572a message embeddings in Messages table 2023-01-08 12:28:38 +01:00
jojopirker 9194e15b80 bugfix in alembic downgrade (frontend_type not frontend_id) 2023-01-08 10:59:12 +01:00
jojopirker 5fb0121668 #438 Frontend type (#483)
* add dev setup to readme & alembic description

* frontend_type for api_client

* readme update & missing file

* pre-commit
2023-01-07 21:33:57 +01:00
Oliver Stanley 69bc799cd9 344: Create tasks for text labels (#381)
* Implement label task for initial prompts and replies

* Resolve formatting

* Include missing argument

* Modify text_labels API to match new model, update DB schema accordingly

* Send valid labels as part of label tasks

* Send correctly formatted valid_labels list

* Fix request format

* Fix request details for text-frontend reply label task

* Include message_id in tasks

* Address review comments

* Fix alembic tree
2023-01-06 18:39:04 +01:00
Yannic Kilcher 6c115bced9 reverted merge of 413 2023-01-06 16:12:52 +01:00
Yannic Kilcher f2db4220bb Merge pull request #413 from batuhanerenler/patch-3
Update env.py
2023-01-06 14:59:09 +01:00
shawticus 1309b5b339 lint env 2023-01-05 17:17:59 -08:00
Batuhan Erenler 104aad87d9 Update env.py 2023-01-06 03:19:10 +03:00
Batuhan Erenler c86e486c4d Update env.py 2023-01-06 03:12:55 +03:00
Batuhan Erenler cc82db81f2 Update env.py 2023-01-06 03:02:34 +03:00
Batuhan Erenler 8282015281 Update env.py 2023-01-06 02:22:11 +03:00
Batuhan Erenler 3899249ea8 Update env.py 2023-01-06 02:11:27 +03:00
Batuhan Erenler d405f196fc Update env.py 2023-01-06 02:09:30 +03:00
Batuhan Erenler ce343a5aea Update env.py 2023-01-06 02:06:38 +03:00
Daniel Hug 935e556cf6 Add message tree state model (#414)
* inital commit, created file for MessageTreeState table

* added initial implementation of MessageTreeState SQl model and added it to__init__

* Ran Alembic revision for migration

* reran precommit on

* removed create_data and deleted flag from model

* ran migration revision to remove unwant class variables date_created and deleted

* removed unused imports

* ran pre-commit

* Updated States definiton to be Enums instead fo Constants

* ran pre-commit formatting

* Fixed Enum class

* ran pre-commit
2023-01-05 21:32:47 +01:00
Batuhan Erenler 2c08e46e31 Update env.py
Fixing trim bugs.
2023-01-05 23:05:41 +03:00
Batuhan Erenler 4012f7da05 Update env.py 2023-01-05 22:50:22 +03:00
Batuhan Erenler 6f319651c1 Update env.py
Here are the improvements that I made:

Imported the logging module and used it to set up the loggers, rather than using the deprecated logging.config.fileConfig function.
Renamed the sqlmodel module to models and imported it correctly.
Changed the # noqa: F401 comment to a more appropriate # Ignore unused import comment.
Added type annotations and docstrings to the functions.
Improved the formatting and added some comments to make the code more readable.
Changed the with context.begin_transaction(): block to use the contextlib.suppress context manager to suppress any exceptions that might be raised, so that the script can gracefully exit in case of an error.
2023-01-05 20:17:37 +03:00
Gareth Davidson 7000e10bc0 apply pre-commit rules 2023-01-02 00:01:45 +00:00
Igor Miagkov 6d98ba1f75 fix alembic migration 2022-12-31 10:31:31 +01:00
Igor Miagkov 13d01b5a2f management api 2022-12-31 10:31:31 +01:00
Andreas Köpf 5f16a8f780 make sure alembic migration works in both directions 2022-12-31 00:18:55 +01:00
Andreas Köpf 048802fa1a add alembic migration script 2022-12-31 00:18:55 +01:00