Commit Graph

128 Commits

Author SHA1 Message Date
Graeme Harris 66b7ed2b9b 968 add flagged message table and endpoints (#1325)
* Added flagged message table

* Added alembic migration and updated imports to match style

* Added GET endpoint to query all flagged messages

* Updates from linter

* Added POST endpoint for processing flagged messages

* Added pydantic interface model and fixed limit update bug

* fixed session in admin endpoint and added require session refresh for returned update

* removed unused import
2023-02-07 22:39:24 +01:00
Andreas Köpf 363a3a1244 Ignore up/down votes (emojis) for own messages (#1302)
* ignore up/down votes (emojis) for own messages

* add 'user_is_author' bool porperty to protocol.ConversationMessage
2023-02-07 16:40:25 +01:00
Kian-Meng Ang 1e321a6fca Fix typos (#1143)
Found via `codespell -S .mypy_cache,yarn.lock,*.json,*.ipynb -L
rouge,nam,vie`
2023-02-05 20:18:03 +01:00
Andreas Köpf f02543b36f reduce INFO logging 2023-02-04 20:26:20 +01:00
Oliver Stanley 652b7bff04 857: Implement backend Discord authentication (#943)
* Initial code for backend auth

* Remove outdated check

* Initial separation of AuthenticatedUser

* AuthenticatedUser -> Account

* Rework for Account

* Initial code for Discord OAuth

* Remove now redundant methods

* Remove incorrect response model, add requests dep for backend

* Create Settings fields for Discord values

* Cleanup get account from Discord function

* Cleanup

* Cleanup

* Generate alembic upgrade script

* Remove unused error codes

* Update alembic script to correct down revision

* Use aiohttp over requests

* Update alembic script to latest down revision
2023-02-02 19:33:21 +01:00
Andreas Köpf 2db3450e9a add trollboards 2023-02-02 16:10:38 +01:00
Andreas Köpf 779f522385 Skip tasks permanently based on special skip emoji codes (#1064)
* add skip emojis

* extend tree_manager queries to exclude tasks based on skip-emojis

* remove unrelated columns

* change skip_reason max_length to 512

* update down_revision to 55361f323d12
2023-02-02 00:07:08 +01:00
Andreas Köpf bbf038677c Add terms of service acceptance date to user table (#1046)
* add tos_acceptance_date column to user

* send 451 UNAVAILABLE_FOR_LEGAL_REASONS status

* add create user REST endpoint

* adapt text-frontend to ToS requirements

* set DEBUG_IGNORE_TOS_ACCEPTANCE default to True (temporary change)

* update down revision to f60958968ff8
2023-02-01 23:53:21 +01:00
Andreas Köpf 9b8574f247 Limit number of replies to assistant messages (#1036)
* limit replies to assistant messages

* revert some debug changes

* use uuid for random message id in auto_main

* Update config.py
2023-01-31 15:08:54 +01:00
Andreas Köpf b6bdb84019 Add endpoint to query nearby leaderboard rows (#1038)
* add is_current_user bool

* add user leaderboard surrounding window function
2023-01-31 15:05:05 +01:00
Andreas Köpf 65d69dac66 fix steak interaction changes 2023-01-31 13:08:13 +01:00
James Melvin Ebenezer 063157355c 906: update user streaks (#1016)
* fix: update user streaks
* Moved  streak_last_day_date & streak_days from UserStats to User
* Updated Alembic version
* update last_activity after tm.handle_interaction()
* periodically executed function in main.py to update the user_streak every 4hrs

* fix:removed log messages

* fix: pre commit issues

* refactor: incorporated review comments

* fix: removed the managed -scope.

* refactor: managed_tx to Outer REST Db Ops in /interaction API

* fix: pre-commit changes

* fix: added streak and last_activity_date to protocol.FrontEndUser

* fix: pre-commit fixes after merge

* fix: added streak info to user_stat_repository, leaderboard and review comments

* fix: proper 4h delay and simpler startup_time initialisation

---------

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-31 12:27:45 +01:00
Andreas Köpf 2b561a0dde add endpoint to put message tree into 'halted_by_moderator' state (#1025)
* add PUT endpoint to put message tree into halted_by_moderator state

* add private _reactivate_tree() method
2023-01-31 09:28:33 +01:00
Andreas Köpf a5bc9bf492 Add dupe checks to store_text_reply() & store_text_labels() in PromptRepository (#1018)
* add dupe checks to store_text_reply() & store_text_labels

* remove test export file

* add user_id to protocol.ConversationMessage

* add show_on_leaderboard ot protocol.FrontEndUser
2023-01-30 20:53:59 +01:00
Keith Stevens cc1a4bdcb4 Merge pull request #920 from LAION-AI/857-backend-token-decrypt
857 backend token decrypt
2023-01-29 19:31:16 +09:00
Andreas Köpf 19116f7251 add optional message_id query param to text_labels/valid_labels endpoint 2023-01-28 15:29:38 +01:00
Andreas Köpf 264e914225 exclude fails_task from default valid labels 2023-01-28 15:07:46 +01:00
Andreas Köpf c8d16285d0 Import message trees from jsonl file (#964)
* add new backlog_ranking tree state

* add first version of import script

* allow activation of trees during import

* add min_active_rankings_per_lang config param

* add settings docstring
2023-01-28 15:05:46 +01:00
Keith Stevens 3197b6088b Using more pydantic features in the backend and fixing env issues on the website 2023-01-28 19:35:40 +09:00
Keith Stevens c333ecd27a Merging from main 2023-01-28 18:05:56 +09:00
Keith Stevens 62d00d5bd4 Adding a minimal backend API route that decrypts the web's JWE and returns the email 2023-01-28 16:56:34 +09:00
Andreas Köpf 3a32a10b23 add reply_message to LabelConversationReplyTask & LabelConversationReplyPayload 2023-01-27 22:30:27 +01:00
Andreas Köpf 4b3ce24c72 add parsing of additional x-oasst-user header field (#957)
* add parsing of additional x-oasst-user header field

* simplify passing FrontendUserId to PromptRepository, use frontend_user_id on more message enpoints

* remove unrelated change
2023-01-27 21:28:57 +01:00
Andreas Köpf 9bad3d6e74 Exclude certain users from leaderboard stats (#961)
Co-authored-by: Akhil Datla <66145155+akhil-datla@users.noreply.github.com>
2023-01-27 21:04:55 +01:00
Andreas Köpf da1c81d2c9 Add LabelDescription list to labeling tasks, make +1/-1 emojis exclusive (#947)
* add LabelDescription list to labeling tasks

* make +1 & -1 emoji exclusive (only one of both or none)

* add red_flag emoji to message when reported

* fix task's valid labels

* fix typo
2023-01-27 00:54:29 +01:00
Andreas Köpf c2fa476904 Add user emoji augmentation for message queries (#937)
* add disposition to text labeling tasks

* add emoji stats to ConversationMessage

* add user emoji augmentation for message queries

* add auth_method,username to message queries (query emoji status)

* add auth_method+username for single message

* fix param name typo

* only join rows when message.emojis != JSON.NULL

* formatting

* make sure emojis and user_emojis default to {}, []

* remove init_user(), use fresh empty default collections
2023-01-26 14:29:54 +00:00
Keith Stevens 76ef08dfe4 decoding 2023-01-25 18:12:44 +09:00
Andreas Köpf 558b207013 Add /messages/{message_id}/emoji endpoint to toggle, add, remove message emojis (#925)
* add endpoint to set message emojis

* make refresh result optional in db utils
2023-01-25 09:31:20 +01:00
Jordi Smit 8c632f9ef9 add lang filter option to message endpoints (#902)
* add lang filter option to message endpoints

* add lang filter option to frontend_users endpoints

* move lang param before api_client

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-23 21:46:59 +00:00
Keith Stevens 3c2d1086b8 Adding a new auth route 2023-01-23 18:13:33 +09:00
Andreas Köpf 43e7d4846b lt->before, gt->after for cursor endpoints 2023-01-22 18:43:38 +01:00
Andreas Köpf 0b8ed52102 fix desc ordering of message cursor endpoints 2023-01-22 18:37:09 +01:00
Keith Stevens ec5bdef719 Not sure this will work 2023-01-22 21:36:52 +09:00
James Melvin Ebenezer c0391a6df9 fix: redundant row updates with no Task id in text_labels table (#876)
* fix: redundant row updates with no Task id in text_labels table

* fix: review comments incorporated

* fix: better error handling and function name

* fix: review comments

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-22 11:08:02 +01:00
Andreas Köpf 3b5b6669a5 move lt-desc order to users-cursor function 2023-01-22 08:42:38 +01:00
Andreas Köpf 76b5bdc5d7 fix cursor queries query_users_ordered_by_username/query_users_ordered_by_display_name 2023-01-22 00:58:03 +01:00
Ori Yonay cec49614c2 added message size limit of 2000 characters (issue 822) (#880)
* added message size limit (issue 822)
2023-01-21 22:37:42 +01:00
Andreas Köpf e08b36e675 imporve prev/next status for user cursor 2023-01-21 16:04:43 +01:00
Andreas Köpf 186aabe3a5 improve prev,next cursor values 2023-01-21 12:19:45 +01:00
Andreas Köpf 94e5d50537 add users/cursor keyset_pagination endpoint (#866)
* add users/cursor endpoint

* add messages/cursor endpoint

* add user/{user_id}/messages/cursor, frontend_user/{auth_method}/{username}/messages/cursor

* user regex to parse cursor value
2023-01-21 00:29:53 +01:00
Andreas Köpf 2d21b65ed0 Add lang-tag based task selection (lang-separation) (#863)
* lang based task selection

* use BCP 47 instead of ISO 639-1

* add Field(None, nullable=True)

* update migration script down_revision
2023-01-20 19:58:33 +01:00
Andreas Köpf 70fc80aa08 Add keyset pagination for users ordered by username / display_name (#851)
* add keyset pagination for user ordered by username or display_name

* add index on display-name for user table

* update down_revision in migration script
2023-01-20 16:32:13 +01:00
Andreas Köpf 7a1a8c8b54 fetch whole message tree for purge (including non-reviewed & deleted) 2023-01-19 16:42:13 +01:00
Andreas Köpf 2d4e38a272 remove ensure_user_is_enabled() from ACK/NACK 2023-01-19 15:36:34 +01:00
Andreas Köpf 488842c3cb purge -> purge_user in path 2023-01-19 15:12:37 +01:00
Andreas Köpf 547e355e27 add update_leaderboard..(), admin/purge/{user-id}/messages, ban param for purge_user() 2023-01-19 15:10:22 +01:00
Andreas Köpf ef8a00e682 add GET /api/v1/admin/backend_settings/{full/public} endpoint for api-clients (#830)
* add GET /api/v1/admin/backend_settings endpoint for trusted api-clients

* add backend_settings/public for untrusted api_clients
2023-01-19 11:48:26 +00:00
Andreas Köpf 335af5d641 add admin purge user function (#834)
* add admin purge user function

* improve comments

* minor naming changes

* ensuer user is enabled for tasks api requests

* add preview with stats to /admin/purge_user/{id} endpoint

* add update_children_counts()
2023-01-19 12:33:55 +01:00
Andreas Köpf 718faa098e Add combined TreeManager stats endpoint (#816) 2023-01-18 00:54:19 +01:00
Andreas Köpf ead51ff423 770 tree manager allow to specify desired task_type no prompter ranking (#775)
* only ranking assistant replies by default

* add tasks/availability endpoint allow to specify desired task

* move rank_prompter_replies option to TreeManagerConfiguration

* fix type annotation

* remove desired_task_type from _random_task_selection()

* fix typo

* Convert query_tree_size to sqlachemy, return 'full' text-labeling tasks if they were explicitly requested
2023-01-16 20:05:40 +01:00