Commit Graph

354 Commits

Author SHA1 Message Date
Oliver Stanley cf4c8e70ce 1184: Add dedicated export.py (#1270)
* Add export script
2023-02-06 21:29:30 +00:00
Andreas Köpf 2434d44c01 Exclude extendible parents with young reply tasks (#1196)
* exclude extendible parents with young reply tasks

* fix typos
2023-02-05 20:22:45 +01:00
Kian-Meng Ang 1e321a6fca Fix typos (#1143)
Found via `codespell -S .mypy_cache,yarn.lock,*.json,*.ipynb -L
rouge,nam,vie`
2023-02-05 20:18:03 +01:00
Andreas Köpf dfda94f178 add some sql snippets for db state inspection 2023-02-05 17:54:27 +01:00
Oliver Stanley 9c7ced59bb Update get user stats to include necessary fields (#1177) 2023-02-05 16:29:46 +01:00
Andreas Köpf 263edbaefd move prompt lottery tree activation into separate transaction scope 2023-02-05 01:16:34 +01:00
Andreas Köpf 59e3252190 limit number of trees activated during _prompt_lottery() calls 2023-02-05 00:18:05 +01:00
Andreas Köpf 100ecbe2d5 Limit initial prompts, ensure max_active_trees = growing trees 2023-02-04 23:23:16 +01:00
Andreas Köpf f02543b36f reduce INFO logging 2023-02-04 20:26:20 +01:00
Andreas Köpf effbde407a increase db pool size 2023-02-04 18:28:17 +01:00
Andreas Köpf af0711e505 include initial prompt review in user stats 2023-02-04 14:36:29 +01:00
Andreas Köpf f547aa07fa retain completed tasks after expiry 2023-02-04 14:02:54 +01:00
Andreas Köpf ad6c39b81a Auto-moderation & remaining v1 fixes (#1089)
* add expiry date for tasks and periodic removal, fix purge user messages sibling ranking counts

* add auto-moderation feature

* fix doc strings

* fix bad message query

* add debug log on insert message

* fix >= comparison
2023-02-03 23:48:45 +01:00
Andreas Köpf 49e4a38191 fix incomplete rankings query 2023-02-03 14:27:34 +00:00
Andreas Köpf 37d4fac904 change formatting 2023-02-03 10:19:04 +00:00
Andreas Köpf af3ec08f71 fix: not timeout for sikpped tasks 2023-02-03 09:43:19 +00:00
Andreas Köpf 6681aedd01 add P_LONELY_CHILD_EXTENSION to workflow vars 2023-02-03 09:36:44 +00:00
Andreas Köpf 715127e60e change backend default config 2023-02-03 07:47:43 +00:00
Andreas Köpf 4886626904 fix alembic migration script 2023-02-02 19:41:34 +01:00
Oliver Stanley 652b7bff04 857: Implement backend Discord authentication (#943)
* Initial code for backend auth

* Remove outdated check

* Initial separation of AuthenticatedUser

* AuthenticatedUser -> Account

* Rework for Account

* Initial code for Discord OAuth

* Remove now redundant methods

* Remove incorrect response model, add requests dep for backend

* Create Settings fields for Discord values

* Cleanup get account from Discord function

* Cleanup

* Cleanup

* Generate alembic upgrade script

* Remove unused error codes

* Update alembic script to correct down revision

* Use aiohttp over requests

* Update alembic script to latest down revision
2023-02-02 19:33:21 +01:00
Andreas Köpf 2db3450e9a add trollboards 2023-02-02 16:10:38 +01:00
Andreas Köpf 5f7013fb81 add '_' prefix to special emoji codes, fix skip rankings 2023-02-02 00:51:20 +01:00
Andreas Köpf f4b534eb17 no task ack check for skip 2023-02-02 00:27:27 +01:00
Andreas Köpf d33fb9ce88 fix alembic downgrade script of 7b8f0011e0b0 2023-02-02 00:15:49 +01:00
Andreas Köpf 779f522385 Skip tasks permanently based on special skip emoji codes (#1064)
* add skip emojis

* extend tree_manager queries to exclude tasks based on skip-emojis

* remove unrelated columns

* change skip_reason max_length to 512

* update down_revision to 55361f323d12
2023-02-02 00:07:08 +01:00
Andreas Köpf bbf038677c Add terms of service acceptance date to user table (#1046)
* add tos_acceptance_date column to user

* send 451 UNAVAILABLE_FOR_LEGAL_REASONS status

* add create user REST endpoint

* adapt text-frontend to ToS requirements

* set DEBUG_IGNORE_TOS_ACCEPTANCE default to True (temporary change)

* update down revision to f60958968ff8
2023-02-01 23:53:21 +01:00
Andreas Köpf e0df9f0b7c Add infty prompting + prompt-lottery (#1055)
* first bits of prompt-lottery

* add tree prompt lottery

* minor text changes
2023-02-01 23:49:21 +01:00
Andreas Köpf af0ae71bbe Add message to prompt-labeling task (#1035)
* add full message to prompt-labeling & always require full prompt labeling

* always sending conversation, also for initial prompt labeling

* update type annotation
2023-01-31 14:42:04 +00:00
Andreas Köpf 9b8574f247 Limit number of replies to assistant messages (#1036)
* limit replies to assistant messages

* revert some debug changes

* use uuid for random message id in auto_main

* Update config.py
2023-01-31 15:08:54 +01:00
Andreas Köpf b6bdb84019 Add endpoint to query nearby leaderboard rows (#1038)
* add is_current_user bool

* add user leaderboard surrounding window function
2023-01-31 15:05:05 +01:00
Andreas Köpf 65d69dac66 fix steak interaction changes 2023-01-31 13:08:13 +01:00
James Melvin Ebenezer 063157355c 906: update user streaks (#1016)
* fix: update user streaks
* Moved  streak_last_day_date & streak_days from UserStats to User
* Updated Alembic version
* update last_activity after tm.handle_interaction()
* periodically executed function in main.py to update the user_streak every 4hrs

* fix:removed log messages

* fix: pre commit issues

* refactor: incorporated review comments

* fix: removed the managed -scope.

* refactor: managed_tx to Outer REST Db Ops in /interaction API

* fix: pre-commit changes

* fix: added streak and last_activity_date to protocol.FrontEndUser

* fix: pre-commit fixes after merge

* fix: added streak info to user_stat_repository, leaderboard and review comments

* fix: proper 4h delay and simpler startup_time initialisation

---------

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-31 12:27:45 +01:00
Andreas Köpf 2b561a0dde add endpoint to put message tree into 'halted_by_moderator' state (#1025)
* add PUT endpoint to put message tree into halted_by_moderator state

* add private _reactivate_tree() method
2023-01-31 09:28:33 +01:00
Andreas Köpf a5bc9bf492 Add dupe checks to store_text_reply() & store_text_labels() in PromptRepository (#1018)
* add dupe checks to store_text_reply() & store_text_labels

* remove test export file

* add user_id to protocol.ConversationMessage

* add show_on_leaderboard ot protocol.FrontEndUser
2023-01-30 20:53:59 +01:00
Andreas Köpf 6e91c1a4bd export messages ordered by rank 2023-01-30 10:36:58 +01:00
Andreas Köpf e91a645836 fix rank123 scores 2023-01-30 09:46:03 +01:00
Andreas Köpf e61c1c0ca2 fix valid_labels/mandatory labels mixup 2023-01-30 00:26:09 +01:00
Andreas Köpf eda275bef1 fix export & add emojis (#1004) 2023-01-29 23:28:47 +01:00
Andreas Köpf 0601d874e8 add lang_mismatch label to prompt/reply acceptance check (#984)
* add lang_mismatch label to acceptance check

* make acceptance formula clearer

* make lang_mismatch flag, remove helpful from prompter labels
2023-01-29 17:34:37 +01:00
Andreas Köpf 61b2949122 fix retry_scoring_failed_message_trees() 2023-01-29 16:57:45 +01:00
Andreas Köpf b580065559 fix extendible parents query 2023-01-29 12:42:08 +01:00
Keith Stevens cc1a4bdcb4 Merge pull request #920 from LAION-AI/857-backend-token-decrypt
857 backend token decrypt
2023-01-29 19:31:16 +09:00
Andreas Köpf 8db2cd6413 change default values 2023-01-29 11:10:06 +01:00
Andreas Köpf eb4c41e3c6 send full conversation (including last-message) in label tasks 2023-01-28 20:23:39 +01:00
Andreas Köpf 54503b7e1b reduce userstats cron defaults, fix reference error 2023-01-28 19:10:03 +01:00
Andreas Köpf 19116f7251 add optional message_id query param to text_labels/valid_labels endpoint 2023-01-28 15:29:38 +01:00
Andreas Köpf 264e914225 exclude fails_task from default valid labels 2023-01-28 15:07:46 +01:00
Andreas Köpf c8d16285d0 Import message trees from jsonl file (#964)
* add new backlog_ranking tree state

* add first version of import script

* allow activation of trees during import

* add min_active_rankings_per_lang config param

* add settings docstring
2023-01-28 15:05:46 +01:00
Keith Stevens 3197b6088b Using more pydantic features in the backend and fixing env issues on the website 2023-01-28 19:35:40 +09:00
Keith Stevens c333ecd27a Merging from main 2023-01-28 18:05:56 +09:00