Commit Graph

336 Commits

Author SHA1 Message Date
Andreas Köpf 4886626904 fix alembic migration script 2023-02-02 19:41:34 +01:00
Oliver Stanley 652b7bff04 857: Implement backend Discord authentication (#943)
* Initial code for backend auth

* Remove outdated check

* Initial separation of AuthenticatedUser

* AuthenticatedUser -> Account

* Rework for Account

* Initial code for Discord OAuth

* Remove now redundant methods

* Remove incorrect response model, add requests dep for backend

* Create Settings fields for Discord values

* Cleanup get account from Discord function

* Cleanup

* Cleanup

* Generate alembic upgrade script

* Remove unused error codes

* Update alembic script to correct down revision

* Use aiohttp over requests

* Update alembic script to latest down revision
2023-02-02 19:33:21 +01:00
Andreas Köpf 2db3450e9a add trollboards 2023-02-02 16:10:38 +01:00
Andreas Köpf 5f7013fb81 add '_' prefix to special emoji codes, fix skip rankings 2023-02-02 00:51:20 +01:00
Andreas Köpf f4b534eb17 no task ack check for skip 2023-02-02 00:27:27 +01:00
Andreas Köpf d33fb9ce88 fix alembic downgrade script of 7b8f0011e0b0 2023-02-02 00:15:49 +01:00
Andreas Köpf 779f522385 Skip tasks permanently based on special skip emoji codes (#1064)
* add skip emojis

* extend tree_manager queries to exclude tasks based on skip-emojis

* remove unrelated columns

* change skip_reason max_length to 512

* update down_revision to 55361f323d12
2023-02-02 00:07:08 +01:00
Andreas Köpf bbf038677c Add terms of service acceptance date to user table (#1046)
* add tos_acceptance_date column to user

* send 451 UNAVAILABLE_FOR_LEGAL_REASONS status

* add create user REST endpoint

* adapt text-frontend to ToS requirements

* set DEBUG_IGNORE_TOS_ACCEPTANCE default to True (temporary change)

* update down revision to f60958968ff8
2023-02-01 23:53:21 +01:00
Andreas Köpf e0df9f0b7c Add infty prompting + prompt-lottery (#1055)
* first bits of prompt-lottery

* add tree prompt lottery

* minor text changes
2023-02-01 23:49:21 +01:00
Andreas Köpf af0ae71bbe Add message to prompt-labeling task (#1035)
* add full message to prompt-labeling & always require full prompt labeling

* always sending conversation, also for initial prompt labeling

* update type annotation
2023-01-31 14:42:04 +00:00
Andreas Köpf 9b8574f247 Limit number of replies to assistant messages (#1036)
* limit replies to assistant messages

* revert some debug changes

* use uuid for random message id in auto_main

* Update config.py
2023-01-31 15:08:54 +01:00
Andreas Köpf b6bdb84019 Add endpoint to query nearby leaderboard rows (#1038)
* add is_current_user bool

* add user leaderboard surrounding window function
2023-01-31 15:05:05 +01:00
Andreas Köpf 65d69dac66 fix steak interaction changes 2023-01-31 13:08:13 +01:00
James Melvin Ebenezer 063157355c 906: update user streaks (#1016)
* fix: update user streaks
* Moved  streak_last_day_date & streak_days from UserStats to User
* Updated Alembic version
* update last_activity after tm.handle_interaction()
* periodically executed function in main.py to update the user_streak every 4hrs

* fix:removed log messages

* fix: pre commit issues

* refactor: incorporated review comments

* fix: removed the managed -scope.

* refactor: managed_tx to Outer REST Db Ops in /interaction API

* fix: pre-commit changes

* fix: added streak and last_activity_date to protocol.FrontEndUser

* fix: pre-commit fixes after merge

* fix: added streak info to user_stat_repository, leaderboard and review comments

* fix: proper 4h delay and simpler startup_time initialisation

---------

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-31 12:27:45 +01:00
Andreas Köpf 2b561a0dde add endpoint to put message tree into 'halted_by_moderator' state (#1025)
* add PUT endpoint to put message tree into halted_by_moderator state

* add private _reactivate_tree() method
2023-01-31 09:28:33 +01:00
Andreas Köpf a5bc9bf492 Add dupe checks to store_text_reply() & store_text_labels() in PromptRepository (#1018)
* add dupe checks to store_text_reply() & store_text_labels

* remove test export file

* add user_id to protocol.ConversationMessage

* add show_on_leaderboard ot protocol.FrontEndUser
2023-01-30 20:53:59 +01:00
Andreas Köpf 6e91c1a4bd export messages ordered by rank 2023-01-30 10:36:58 +01:00
Andreas Köpf e91a645836 fix rank123 scores 2023-01-30 09:46:03 +01:00
Andreas Köpf e61c1c0ca2 fix valid_labels/mandatory labels mixup 2023-01-30 00:26:09 +01:00
Andreas Köpf eda275bef1 fix export & add emojis (#1004) 2023-01-29 23:28:47 +01:00
Andreas Köpf 0601d874e8 add lang_mismatch label to prompt/reply acceptance check (#984)
* add lang_mismatch label to acceptance check

* make acceptance formula clearer

* make lang_mismatch flag, remove helpful from prompter labels
2023-01-29 17:34:37 +01:00
Andreas Köpf 61b2949122 fix retry_scoring_failed_message_trees() 2023-01-29 16:57:45 +01:00
Andreas Köpf b580065559 fix extendible parents query 2023-01-29 12:42:08 +01:00
Keith Stevens cc1a4bdcb4 Merge pull request #920 from LAION-AI/857-backend-token-decrypt
857 backend token decrypt
2023-01-29 19:31:16 +09:00
Andreas Köpf 8db2cd6413 change default values 2023-01-29 11:10:06 +01:00
Andreas Köpf eb4c41e3c6 send full conversation (including last-message) in label tasks 2023-01-28 20:23:39 +01:00
Andreas Köpf 54503b7e1b reduce userstats cron defaults, fix reference error 2023-01-28 19:10:03 +01:00
Andreas Köpf 19116f7251 add optional message_id query param to text_labels/valid_labels endpoint 2023-01-28 15:29:38 +01:00
Andreas Köpf 264e914225 exclude fails_task from default valid labels 2023-01-28 15:07:46 +01:00
Andreas Köpf c8d16285d0 Import message trees from jsonl file (#964)
* add new backlog_ranking tree state

* add first version of import script

* allow activation of trees during import

* add min_active_rankings_per_lang config param

* add settings docstring
2023-01-28 15:05:46 +01:00
Keith Stevens 3197b6088b Using more pydantic features in the backend and fixing env issues on the website 2023-01-28 19:35:40 +09:00
Keith Stevens c333ecd27a Merging from main 2023-01-28 18:05:56 +09:00
Keith Stevens 62d00d5bd4 Adding a minimal backend API route that decrypts the web's JWE and returns the email 2023-01-28 16:56:34 +09:00
MattAlexMiracle 994dce7c76 language classifier (#963)
Co-authored-by: Alexander Mattick <alex.mattick@fau.de>
2023-01-28 00:56:56 +01:00
Andreas Köpf 0903ec5d58 fix quility label addition 2023-01-28 00:03:52 +01:00
Andreas Köpf 8ea53a54eb add quality to valid-labels for simple text labeling tasks (anti-spam) 2023-01-28 00:01:49 +01:00
dhug 49b5999ce6 Duplicate message reply filter (#958)
* added changes for user specific message dumplication filter, added error codes, and settings variable as described in Draft PR #926, ran precommit

* removed debug statements

* add missing await to async_managed_tx_method

* add 2nd missing await to async_managed_tx_method

* added changes for user specific message dumplication filter, added error codes, and settings variable as described in Draft PR #926, ran precommit

* removed debug statements

* assert task user matches prompt_repository user

* removed assert statments

* moved duplicate_message_filter and message_length check into store_text_reply

* removed old checks in tree_manager
2023-01-27 21:52:19 +00:00
Andreas Köpf 3a32a10b23 add reply_message to LabelConversationReplyTask & LabelConversationReplyPayload 2023-01-27 22:30:27 +01:00
Andreas Köpf 4b3ce24c72 add parsing of additional x-oasst-user header field (#957)
* add parsing of additional x-oasst-user header field

* simplify passing FrontendUserId to PromptRepository, use frontend_user_id on more message enpoints

* remove unrelated change
2023-01-27 21:28:57 +01:00
Andreas Köpf 002f847406 add missing alembic migration script 2023-01-27 21:16:31 +01:00
Andreas Köpf 9bad3d6e74 Exclude certain users from leaderboard stats (#961)
Co-authored-by: Akhil Datla <66145155+akhil-datla@users.noreply.github.com>
2023-01-27 21:04:55 +01:00
Andreas Köpf c7692b9049 add 2nd missing await to async_managed_tx_method 2023-01-27 19:44:48 +01:00
Andreas Köpf d165987256 add missing await to async_managed_tx_method 2023-01-27 19:43:08 +01:00
James Melvin Ebenezer 3b04080d7b 949_transaction error handling (#950)
* fix: transaction error handling

* refactor: retry handling for all decorators as per review comments

* fix: raising retry exhausted error

* fix: avoid auto refresh on RollBack and review comments

* removed refresh_result param from managed_tx_function

---------

Co-authored-by: James Melvin <melvin@gameface.ai>
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-27 18:06:25 +01:00
Andreas Köpf da1c81d2c9 Add LabelDescription list to labeling tasks, make +1/-1 emojis exclusive (#947)
* add LabelDescription list to labeling tasks

* make +1 & -1 emoji exclusive (only one of both or none)

* add red_flag emoji to message when reported

* fix task's valid labels

* fix typo
2023-01-27 00:54:29 +01:00
Andreas Köpf f3ffde47ff add preferred lonely_children extension (#942)
* add preferred lonely_children extension

* simplify sampling process, lower the probability to 25%

* exclude parents for replies that were recently used

* lonely children := count > 0

* consider only tasks not done for parent exclusion

* increase lonely child sampling probability
2023-01-26 23:00:54 +01:00
Andreas Köpf 348999a936 exclude trees in ranking state in acitve tree count 2023-01-26 19:08:58 +01:00
Andreas Köpf d4688835d5 check condition for scoring on startup 2023-01-26 16:33:03 +01:00
Andreas Köpf c2fa476904 Add user emoji augmentation for message queries (#937)
* add disposition to text labeling tasks

* add emoji stats to ConversationMessage

* add user emoji augmentation for message queries

* add auth_method,username to message queries (query emoji status)

* add auth_method+username for single message

* fix param name typo

* only join rows when message.emojis != JSON.NULL

* formatting

* make sure emojis and user_emojis default to {}, []

* remove init_user(), use fresh empty default collections
2023-01-26 14:29:54 +00:00
MattAlexMiracle 5d4f74f9d6 Ranked pairs (#933)
* commented out legacy numerical solver

* added comments and task_scheduling for selecting which task to serve to users

* removed standalone task weighting

* pre-commit hook rerun

* fixed ranking

* fix index error

* ranking fix

* fix typo

Co-authored-by: Alexander Mattick <alex.mattick@fau.de>
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-26 09:50:25 +00:00