Commit Graph

289 Commits

Author SHA1 Message Date
Andreas Köpf da1c81d2c9 Add LabelDescription list to labeling tasks, make +1/-1 emojis exclusive (#947)
* add LabelDescription list to labeling tasks

* make +1 & -1 emoji exclusive (only one of both or none)

* add red_flag emoji to message when reported

* fix task's valid labels

* fix typo
2023-01-27 00:54:29 +01:00
Andreas Köpf f3ffde47ff add preferred lonely_children extension (#942)
* add preferred lonely_children extension

* simplify sampling process, lower the probability to 25%

* exclude parents for replies that were recently used

* lonely children := count > 0

* consider only tasks not done for parent exclusion

* increase lonely child sampling probability
2023-01-26 23:00:54 +01:00
Andreas Köpf 348999a936 exclude trees in ranking state in acitve tree count 2023-01-26 19:08:58 +01:00
Andreas Köpf d4688835d5 check condition for scoring on startup 2023-01-26 16:33:03 +01:00
Andreas Köpf c2fa476904 Add user emoji augmentation for message queries (#937)
* add disposition to text labeling tasks

* add emoji stats to ConversationMessage

* add user emoji augmentation for message queries

* add auth_method,username to message queries (query emoji status)

* add auth_method+username for single message

* fix param name typo

* only join rows when message.emojis != JSON.NULL

* formatting

* make sure emojis and user_emojis default to {}, []

* remove init_user(), use fresh empty default collections
2023-01-26 14:29:54 +00:00
MattAlexMiracle 5d4f74f9d6 Ranked pairs (#933)
* commented out legacy numerical solver

* added comments and task_scheduling for selecting which task to serve to users

* removed standalone task weighting

* pre-commit hook rerun

* fixed ranking

* fix index error

* ranking fix

* fix typo

Co-authored-by: Alexander Mattick <alex.mattick@fau.de>
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-26 09:50:25 +00:00
Andreas Köpf 4d7be8ebb9 wait for pending message reviews before entering ranking state 2023-01-25 18:18:21 +01:00
Andreas Köpf 39a107e851 Use intersection of ranking ID sets, add fetch_siblings() 2023-01-25 16:21:58 +01:00
Vechtomov 1020dcb024 add retry_scoring_failed_message_trees cli command (#931) 2023-01-25 13:40:36 +01:00
Andreas Köpf 558b207013 Add /messages/{message_id}/emoji endpoint to toggle, add, remove message emojis (#925)
* add endpoint to set message emojis

* make refresh result optional in db utils
2023-01-25 09:31:20 +01:00
Graeme Harris 032a748ba5 677 - Add tree message export (#808)
* Added - Basic functions to export trees for users, export-ready trees and specific tree ids to files

* Added print to logger by default for no file specified

* linting to remove extra imports

* Added cli for exporting trees which are ready to export

Fixed some accidental removal

Updated message lookup to use dict for better perf

* removed unused imports

* changed export flag for including deleted prompts back to include_deleted for better understandability

* Use native collection types list, tuple, dict

* pre-commit fix

Co-authored-by: Andreas Köpf <andreas.koepf@provisio.com>
2023-01-24 22:13:10 +01:00
Andreas Köpf ffaf5c48d1 907 avoid duplicate labeling & ranking tasks (#923)
* store message_id in message_reactions and task_id in text_labels

* exclude tasks to which users already responded to

* remove test code

* fix join in find_incomplete_rankings_ex
2023-01-24 17:33:15 +01:00
Andreas Köpf d72f7771ca fix HF detoxify call 2023-01-24 17:26:19 +01:00
Andreas Köpf 50421dfada retry user lookup in case of UniqueViolation (ix_user_username conflict) 2023-01-24 09:57:45 +01:00
Jordi Smit 8c632f9ef9 add lang filter option to message endpoints (#902)
* add lang filter option to message endpoints

* add lang filter option to frontend_users endpoints

* move lang param before api_client

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-23 21:46:59 +00:00
Andreas Köpf b7fab83d36 remove unused crud baseclass 2023-01-23 20:49:50 +01:00
Andreas Köpf 43e7d4846b lt->before, gt->after for cursor endpoints 2023-01-22 18:43:38 +01:00
Andreas Köpf 0b8ed52102 fix desc ordering of message cursor endpoints 2023-01-22 18:37:09 +01:00
notmd 7025b174d2 Merge branch 'main' into 766_admin_enhancement 2023-01-22 17:22:31 +07:00
James Melvin Ebenezer c0391a6df9 fix: redundant row updates with no Task id in text_labels table (#876)
* fix: redundant row updates with no Task id in text_labels table

* fix: review comments incorporated

* fix: better error handling and function name

* fix: review comments

Co-authored-by: James Melvin <melvin@gameface.ai>
2023-01-22 11:08:02 +01:00
Andreas Köpf 28089d9ecf fix username+auth combo check 2023-01-22 09:29:21 +01:00
notmd d9205bec52 Merge branch 'main' into 766_admin_enhancement 2023-01-22 14:58:15 +07:00
Andreas Köpf 3b5b6669a5 move lt-desc order to users-cursor function 2023-01-22 08:42:38 +01:00
notmd 101f2c536a revert change in user_repository 2023-01-22 14:20:39 +07:00
notmd 6945cc5fe7 remove reverse method 2023-01-22 14:14:41 +07:00
notmd fa5702ac9e Merge branch 'main' into 766_admin_enhancement 2023-01-22 14:07:19 +07:00
Andreas Köpf 76b5bdc5d7 fix cursor queries query_users_ordered_by_username/query_users_ordered_by_display_name 2023-01-22 00:58:03 +01:00
Ori Yonay cec49614c2 added message size limit of 2000 characters (issue 822) (#880)
* added message size limit (issue 822)
2023-01-21 22:37:42 +01:00
notmd 0cc6b3b8ae Merge branch 'main' into 766_admin_enhancement 2023-01-21 22:10:15 +07:00
Andreas Köpf e08b36e675 imporve prev/next status for user cursor 2023-01-21 16:04:43 +01:00
notmd 27e1e549c4 fix query in backward direction 2023-01-21 20:10:19 +07:00
Andreas Köpf 186aabe3a5 improve prev,next cursor values 2023-01-21 12:19:45 +01:00
Andreas Köpf 94e5d50537 add users/cursor keyset_pagination endpoint (#866)
* add users/cursor endpoint

* add messages/cursor endpoint

* add user/{user_id}/messages/cursor, frontend_user/{auth_method}/{username}/messages/cursor

* user regex to parse cursor value
2023-01-21 00:29:53 +01:00
Andreas Köpf 2d21b65ed0 Add lang-tag based task selection (lang-separation) (#863)
* lang based task selection

* use BCP 47 instead of ISO 639-1

* add Field(None, nullable=True)

* update migration script down_revision
2023-01-20 19:58:33 +01:00
Andreas Köpf 70fc80aa08 Add keyset pagination for users ordered by username / display_name (#851)
* add keyset pagination for user ordered by username or display_name

* add index on display-name for user table

* update down_revision in migration script
2023-01-20 16:32:13 +01:00
Andreas Köpf 038d1cc004 last_uddated for user_stats, swtich to timestamptz 2023-01-19 22:07:08 +01:00
Andreas Köpf 7a1a8c8b54 fetch whole message tree for purge (including non-reviewed & deleted) 2023-01-19 16:42:13 +01:00
Andreas Köpf 1a93c21f94 add logging 2023-01-19 16:28:03 +01:00
Andreas Köpf 2d4e38a272 remove ensure_user_is_enabled() from ACK/NACK 2023-01-19 15:36:34 +01:00
Andreas Köpf 488842c3cb purge -> purge_user in path 2023-01-19 15:12:37 +01:00
Andreas Köpf 547e355e27 add update_leaderboard..(), admin/purge/{user-id}/messages, ban param for purge_user() 2023-01-19 15:10:22 +01:00
Andreas Köpf ef8a00e682 add GET /api/v1/admin/backend_settings/{full/public} endpoint for api-clients (#830)
* add GET /api/v1/admin/backend_settings endpoint for trusted api-clients

* add backend_settings/public for untrusted api_clients
2023-01-19 11:48:26 +00:00
Andreas Köpf 335af5d641 add admin purge user function (#834)
* add admin purge user function

* improve comments

* minor naming changes

* ensuer user is enabled for tasks api requests

* add preview with stats to /admin/purge_user/{id} endpoint

* add update_children_counts()
2023-01-19 12:33:55 +01:00
Andreas Köpf 7e8d4f5a6e fix only_active condition of tree_message_count_stats() 2023-01-18 22:02:35 +01:00
MattAlexMiracle 530194de08 Fix to ranking algorithm (#826)
* fixed ranking

* fix index error

Co-authored-by: Alexander Mattick <alex.mattick@fau.de>
2023-01-18 21:27:16 +01:00
Vechtomov 9b98056001 change leaderboard ordering (#825) 2023-01-18 21:17:16 +01:00
Andreas Köpf 718faa098e Add combined TreeManager stats endpoint (#816) 2023-01-18 00:54:19 +01:00
Andreas Köpf acaa56ebb0 fix ranking assert for 1st level children 2023-01-18 00:49:28 +01:00
dhug 6b82b2cd13 Add api auth to user stat repo (#805)
* added api_auth to user_stats_repository

* pre_commit changes

* Remove call with dev uuid

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-17 20:07:39 +00:00
Andreas Koepf 0f896d910e make sure we enter READY_FOR_EXPORT after ranking 2023-01-17 17:50:17 +00:00