Commit Graph
56 Commits
Author SHA1 Message Date
Andreas Köpf 5f7013fb81 add '_' prefix to special emoji codes, fix skip rankings 2023-02-02 00:51:20 +01:00
Andreas KöpfandGitHub 779f522385 Skip tasks permanently based on special skip emoji codes (#1064)
* add skip emojis

* extend tree_manager queries to exclude tasks based on skip-emojis

* remove unrelated columns

* change skip_reason max_length to 512

* update down_revision to 55361f323d12
2023-02-02 00:07:08 +01:00
Andreas KöpfandGitHub e0df9f0b7c Add infty prompting + prompt-lottery (#1055)
* first bits of prompt-lottery

* add tree prompt lottery

* minor text changes
2023-02-01 23:49:21 +01:00
Andreas KöpfandGitHub af0ae71bbe Add message to prompt-labeling task (#1035)
* add full message to prompt-labeling & always require full prompt labeling

* always sending conversation, also for initial prompt labeling

* update type annotation
2023-01-31 14:42:04 +00:00
Andreas KöpfandGitHub 9b8574f247 Limit number of replies to assistant messages (#1036)
* limit replies to assistant messages

* revert some debug changes

* use uuid for random message id in auto_main

* Update config.py
2023-01-31 15:08:54 +01:00
063157355c 906: update user streaks (#1016)
* fix: update user streaks
* Moved  streak_last_day_date & streak_days from UserStats to User
* Updated Alembic version
* update last_activity after tm.handle_interaction()
* periodically executed function in main.py to update the user_streak every 4hrs

* fix:removed log messages

* fix: pre commit issues

* refactor: incorporated review comments

* fix: removed the managed -scope.

* refactor: managed_tx to Outer REST Db Ops in /interaction API

* fix: pre-commit changes

* fix: added streak and last_activity_date to protocol.FrontEndUser

* fix: pre-commit fixes after merge

* fix: added streak info to user_stat_repository, leaderboard and review comments

* fix: proper 4h delay and simpler startup_time initialisation

---------

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-31 12:27:45 +01:00
Andreas KöpfandGitHub 2b561a0dde add endpoint to put message tree into 'halted_by_moderator' state (#1025)
* add PUT endpoint to put message tree into halted_by_moderator state

* add private _reactivate_tree() method
2023-01-31 09:28:33 +01:00
Andreas KöpfandGitHub a5bc9bf492 Add dupe checks to store_text_reply() & store_text_labels() in PromptRepository (#1018)
* add dupe checks to store_text_reply() & store_text_labels

* remove test export file

* add user_id to protocol.ConversationMessage

* add show_on_leaderboard ot protocol.FrontEndUser
2023-01-30 20:53:59 +01:00
Andreas Köpf e61c1c0ca2 fix valid_labels/mandatory labels mixup 2023-01-30 00:26:09 +01:00
Andreas KöpfandGitHub 0601d874e8 add lang_mismatch label to prompt/reply acceptance check (#984)
* add lang_mismatch label to acceptance check

* make acceptance formula clearer

* make lang_mismatch flag, remove helpful from prompter labels
2023-01-29 17:34:37 +01:00
Andreas Köpf 61b2949122 fix retry_scoring_failed_message_trees() 2023-01-29 16:57:45 +01:00
Andreas Köpf b580065559 fix extendible parents query 2023-01-29 12:42:08 +01:00
Andreas Köpf eb4c41e3c6 send full conversation (including last-message) in label tasks 2023-01-28 20:23:39 +01:00
Andreas KöpfandGitHub c8d16285d0 Import message trees from jsonl file (#964)
* add new backlog_ranking tree state

* add first version of import script

* allow activation of trees during import

* add min_active_rankings_per_lang config param

* add settings docstring
2023-01-28 15:05:46 +01:00
Andreas Köpf 0903ec5d58 fix quility label addition 2023-01-28 00:03:52 +01:00
Andreas Köpf 8ea53a54eb add quality to valid-labels for simple text labeling tasks (anti-spam) 2023-01-28 00:01:49 +01:00
dhugandGitHub 49b5999ce6 Duplicate message reply filter (#958)
* added changes for user specific message dumplication filter, added error codes, and settings variable as described in Draft PR #926, ran precommit

* removed debug statements

* add missing await to async_managed_tx_method

* add 2nd missing await to async_managed_tx_method

* added changes for user specific message dumplication filter, added error codes, and settings variable as described in Draft PR #926, ran precommit

* removed debug statements

* assert task user matches prompt_repository user

* removed assert statments

* moved duplicate_message_filter and message_length check into store_text_reply

* removed old checks in tree_manager
2023-01-27 21:52:19 +00:00
Andreas Köpf 3a32a10b23 add reply_message to LabelConversationReplyTask & LabelConversationReplyPayload 2023-01-27 22:30:27 +01:00
Andreas KöpfandGitHub da1c81d2c9 Add LabelDescription list to labeling tasks, make +1/-1 emojis exclusive (#947)
* add LabelDescription list to labeling tasks

* make +1 & -1 emoji exclusive (only one of both or none)

* add red_flag emoji to message when reported

* fix task's valid labels

* fix typo
2023-01-27 00:54:29 +01:00
Andreas KöpfandGitHub f3ffde47ff add preferred lonely_children extension (#942)
* add preferred lonely_children extension

* simplify sampling process, lower the probability to 25%

* exclude parents for replies that were recently used

* lonely children := count > 0

* consider only tasks not done for parent exclusion

* increase lonely child sampling probability
2023-01-26 23:00:54 +01:00
Andreas Köpf 348999a936 exclude trees in ranking state in acitve tree count 2023-01-26 19:08:58 +01:00
Andreas Köpf d4688835d5 check condition for scoring on startup 2023-01-26 16:33:03 +01:00
Andreas KöpfandGitHub c2fa476904 Add user emoji augmentation for message queries (#937)
* add disposition to text labeling tasks

* add emoji stats to ConversationMessage

* add user emoji augmentation for message queries

* add auth_method,username to message queries (query emoji status)

* add auth_method+username for single message

* fix param name typo

* only join rows when message.emojis != JSON.NULL

* formatting

* make sure emojis and user_emojis default to {}, []

* remove init_user(), use fresh empty default collections
2023-01-26 14:29:54 +00:00
Andreas Köpf 4d7be8ebb9 wait for pending message reviews before entering ranking state 2023-01-25 18:18:21 +01:00
Andreas Köpf 39a107e851 Use intersection of ranking ID sets, add fetch_siblings() 2023-01-25 16:21:58 +01:00
VechtomovandGitHub 1020dcb024 add retry_scoring_failed_message_trees cli command (#931) 2023-01-25 13:40:36 +01:00
032a748ba5 677 - Add tree message export (#808)
* Added - Basic functions to export trees for users, export-ready trees and specific tree ids to files

* Added print to logger by default for no file specified

* linting to remove extra imports

* Added cli for exporting trees which are ready to export

Fixed some accidental removal

Updated message lookup to use dict for better perf

* removed unused imports

* changed export flag for including deleted prompts back to include_deleted for better understandability

* Use native collection types list, tuple, dict

* pre-commit fix

Co-authored-by: Andreas Köpf <andreas.koepf@provisio.com>
2023-01-24 22:13:10 +01:00
Andreas KöpfandGitHub ffaf5c48d1 907 avoid duplicate labeling & ranking tasks (#923)
* store message_id in message_reactions and task_id in text_labels

* exclude tasks to which users already responded to

* remove test code

* fix join in find_incomplete_rankings_ex
2023-01-24 17:33:15 +01:00
Andreas Köpf d72f7771ca fix HF detoxify call 2023-01-24 17:26:19 +01:00
Ori YonayandGitHub cec49614c2 added message size limit of 2000 characters (issue 822) (#880)
* added message size limit (issue 822)
2023-01-21 22:37:42 +01:00
Andreas KöpfandGitHub 2d21b65ed0 Add lang-tag based task selection (lang-separation) (#863)
* lang based task selection

* use BCP 47 instead of ISO 639-1

* add Field(None, nullable=True)

* update migration script down_revision
2023-01-20 19:58:33 +01:00
Andreas Köpf 7a1a8c8b54 fetch whole message tree for purge (including non-reviewed & deleted) 2023-01-19 16:42:13 +01:00
Andreas Köpf 1a93c21f94 add logging 2023-01-19 16:28:03 +01:00
Andreas Köpf 547e355e27 add update_leaderboard..(), admin/purge/{user-id}/messages, ban param for purge_user() 2023-01-19 15:10:22 +01:00
Andreas KöpfandGitHub 335af5d641 add admin purge user function (#834)
* add admin purge user function

* improve comments

* minor naming changes

* ensuer user is enabled for tasks api requests

* add preview with stats to /admin/purge_user/{id} endpoint

* add update_children_counts()
2023-01-19 12:33:55 +01:00
Andreas Köpf 7e8d4f5a6e fix only_active condition of tree_message_count_stats() 2023-01-18 22:02:35 +01:00
Andreas KöpfandGitHub 718faa098e Add combined TreeManager stats endpoint (#816) 2023-01-18 00:54:19 +01:00
Andreas Köpf acaa56ebb0 fix ranking assert for 1st level children 2023-01-18 00:49:28 +01:00
Andreas Koepf 0f896d910e make sure we enter READY_FOR_EXPORT after ranking 2023-01-17 17:50:17 +00:00
Andreas Köpf 9b6dc6bae2 fix join types in TreeManager 2023-01-17 11:01:36 +01:00
Andreas Köpf 2ee01d1224 fix query in TreeManager.query_extendible_parents() 2023-01-17 10:45:52 +01:00
Andreas Köpf d85cec3d2f extensible -> extendible 2023-01-17 10:21:57 +01:00
Andreas Köpf 6e4d8097b6 fix availability for label_prompter_reply 2023-01-17 10:15:30 +01:00
Andreas Köpf 988913943f use '.value' of str-enums in db queries 2023-01-17 09:54:20 +01:00
8b30c7b68e add changes for auto_main, tree_manager, and utils/ranking (#786)
* add changes for auto_main, tree_manager, and utils/ranking

* pre-commit changes

Co-authored-by: Alexander Mattick <alex.mattick@fau.de>
2023-01-17 08:27:21 +01:00
Andreas Köpf 2d4e39cf5d use sqlalchemy for query_reviews_for_message() in TreeManager 2023-01-16 20:31:29 +01:00
Andreas KöpfandGitHub ead51ff423 770 tree manager allow to specify desired task_type no prompter ranking (#775)
* only ranking assistant replies by default

* add tasks/availability endpoint allow to specify desired task

* move rank_prompter_replies option to TreeManagerConfiguration

* fix type annotation

* remove desired_task_type from _random_task_selection()

* fix typo

* Convert query_tree_size to sqlachemy, return 'full' text-labeling tasks if they were explicitly requested
2023-01-16 20:05:40 +01:00
c6fbf5543b 599 add row versioning to backend tables (#710)
* fix: isolation level and nested db.commit() with retry wrappers on concurrent update errors

* refactor: incorporated review comments

changes decorator methods to managed_tx_method and async_managed_tx_method
new enum CommitMode
removed commented commit() from the previous commits

* fix: merge pre-commit errors

* fix: merge pre-commit changes

* fix: conflict in existing OasstErrorCode

* refactor: Added a refresh just to be sure that the select command is triggered on the mapped object

* fix: added refresh for async decorator

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-16 08:43:07 +01:00
Andreas KöpfandGitHub b5bb5bb7c0 Add leaderboard stats, periodic updates via fastapi-utils (#724)
* add leaderboard stats, periodic update via fastapi-utils

* count label tasks for assistant and prompter replies

* Daily stats update every 15 mins, simplify leaderboard endpoint

* add indices for some created_date columns

* make user stats update intervals configurable

* make sure intervals are positive
2023-01-15 12:04:19 +01:00
a902c600fa Store Message Toxicity in database (#553)
* [NEW] MessageToxicity table

* [NEW] Alembic message Toxicity

* [NEW] Model name enum

* [NEW] Refactor Enum HF

* [NEW] Settings: DEBUT_SKIP_TOXICITY_CALCULATION

* [NEW] Store toxicity values

* [FIX] Merge conflict

* [FIX] Documentation

* [NEW] save_toxicity: function

* [FIX] Formatted string

* [NEW] DEBUG_SKIP_TOXICITY_CALCULATION=True

* [FIX] HfClassificationModel

* [FIX] Alembic merge heads

* [NEW] Refactor save_toxicity

* [NEW] Separating score/label

* [NEW] Store score and label

* [FIX] Cleaning Alembic

* [NEW] Clean HF names

* [NEW] Not type hinting

* [NEW] Update alembic versions

* [NEW] Revert the changes

* [NEW] Type hinting label & score

* Updated down_revision in migration script

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-14 12:22:55 +00:00