76b5bdc5d7
fix cursor queries query_users_ordered_by_username/query_users_ordered_by_display_name
2023-01-22 00:58:03 +01:00
cec49614c2
added message size limit of 2000 characters (issue 822) ( #880 )
...
* added message size limit (issue 822)
2023-01-21 22:37:42 +01:00
e08b36e675
imporve prev/next status for user cursor
2023-01-21 16:04:43 +01:00
186aabe3a5
improve prev,next cursor values
2023-01-21 12:19:45 +01:00
94e5d50537
add users/cursor keyset_pagination endpoint ( #866 )
...
* add users/cursor endpoint
* add messages/cursor endpoint
* add user/{user_id}/messages/cursor, frontend_user/{auth_method}/{username}/messages/cursor
* user regex to parse cursor value
2023-01-21 00:29:53 +01:00
2d21b65ed0
Add lang-tag based task selection (lang-separation) ( #863 )
...
* lang based task selection
* use BCP 47 instead of ISO 639-1
* add Field(None, nullable=True)
* update migration script down_revision
2023-01-20 19:58:33 +01:00
70fc80aa08
Add keyset pagination for users ordered by username / display_name ( #851 )
...
* add keyset pagination for user ordered by username or display_name
* add index on display-name for user table
* update down_revision in migration script
2023-01-20 16:32:13 +01:00
038d1cc004
last_uddated for user_stats, swtich to timestamptz
2023-01-19 22:07:08 +01:00
7a1a8c8b54
fetch whole message tree for purge (including non-reviewed & deleted)
2023-01-19 16:42:13 +01:00
1a93c21f94
add logging
2023-01-19 16:28:03 +01:00
2d4e38a272
remove ensure_user_is_enabled() from ACK/NACK
2023-01-19 15:36:34 +01:00
488842c3cb
purge -> purge_user in path
2023-01-19 15:12:37 +01:00
547e355e27
add update_leaderboard..(), admin/purge/{user-id}/messages, ban param for purge_user()
2023-01-19 15:10:22 +01:00
ef8a00e682
add GET /api/v1/admin/backend_settings/{full/public} endpoint for api-clients ( #830 )
...
* add GET /api/v1/admin/backend_settings endpoint for trusted api-clients
* add backend_settings/public for untrusted api_clients
2023-01-19 11:48:26 +00:00
335af5d641
add admin purge user function ( #834 )
...
* add admin purge user function
* improve comments
* minor naming changes
* ensuer user is enabled for tasks api requests
* add preview with stats to /admin/purge_user/{id} endpoint
* add update_children_counts()
2023-01-19 12:33:55 +01:00
7e8d4f5a6e
fix only_active condition of tree_message_count_stats()
2023-01-18 22:02:35 +01:00
530194de08
Fix to ranking algorithm ( #826 )
...
* fixed ranking
* fix index error
Co-authored-by: Alexander Mattick <alex.mattick@fau.de >
2023-01-18 21:27:16 +01:00
9b98056001
change leaderboard ordering ( #825 )
2023-01-18 21:17:16 +01:00
718faa098e
Add combined TreeManager stats endpoint ( #816 )
2023-01-18 00:54:19 +01:00
acaa56ebb0
fix ranking assert for 1st level children
2023-01-18 00:49:28 +01:00
6b82b2cd13
Add api auth to user stat repo ( #805 )
...
* added api_auth to user_stats_repository
* pre_commit changes
* Remove call with dev uuid
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com >
2023-01-17 20:07:39 +00:00
0f896d910e
make sure we enter READY_FOR_EXPORT after ranking
2023-01-17 17:50:17 +00:00
9b6dc6bae2
fix join types in TreeManager
2023-01-17 11:01:36 +01:00
2ee01d1224
fix query in TreeManager.query_extendible_parents()
2023-01-17 10:45:52 +01:00
d85cec3d2f
extensible -> extendible
2023-01-17 10:21:57 +01:00
6e4d8097b6
fix availability for label_prompter_reply
2023-01-17 10:15:30 +01:00
988913943f
use '.value' of str-enums in db queries
2023-01-17 09:54:20 +01:00
8b30c7b68e
add changes for auto_main, tree_manager, and utils/ranking ( #786 )
...
* add changes for auto_main, tree_manager, and utils/ranking
* pre-commit changes
Co-authored-by: Alexander Mattick <alex.mattick@fau.de >
2023-01-17 08:27:21 +01:00
2d4e39cf5d
use sqlalchemy for query_reviews_for_message() in TreeManager
2023-01-16 20:31:29 +01:00
ead51ff423
770 tree manager allow to specify desired task_type no prompter ranking ( #775 )
...
* only ranking assistant replies by default
* add tasks/availability endpoint allow to specify desired task
* move rank_prompter_replies option to TreeManagerConfiguration
* fix type annotation
* remove desired_task_type from _random_task_selection()
* fix typo
* Convert query_tree_size to sqlachemy, return 'full' text-labeling tasks if they were explicitly requested
2023-01-16 20:05:40 +01:00
7f562fbbae
add managed_tx_function() decorator and use it for startup db-calls
2023-01-16 09:16:28 +01:00
a42f2cd3ed
add new debug option 'DEBUG_DATABASE_ECHO' (default False)
2023-01-16 08:46:45 +01:00
c6fbf5543b
599 add row versioning to backend tables ( #710 )
...
* fix: isolation level and nested db.commit() with retry wrappers on concurrent update errors
* refactor: incorporated review comments
changes decorator methods to managed_tx_method and async_managed_tx_method
new enum CommitMode
removed commented commit() from the previous commits
* fix: merge pre-commit errors
* fix: merge pre-commit changes
* fix: conflict in existing OasstErrorCode
* refactor: Added a refresh just to be sure that the select command is triggered on the mapped object
* fix: added refresh for async decorator
Co-authored-by: James Melvin <melvin@gameface.ai >
2023-01-16 08:43:07 +01:00
72a58ca2d3
add 'cache_ok = True' to PayloadJSONBType type decorator
2023-01-16 08:37:24 +01:00
5e814eaa4c
Merge pull request #732 from LAION-AI/673-admin-page-pagination
...
Supporting user pagination on the web admin view
2023-01-16 08:39:15 +09:00
bf80820d34
fix realistic seed data (remove 'Assistant: ' prefixes)
2023-01-16 00:26:27 +01:00
de28d67031
infer role from task in store_text_reply()
2023-01-16 00:19:30 +01:00
c0cb15bbb0
fixing text frontend and startup creation
2023-01-16 00:11:20 +01:00
5d441b1570
prepared playbook for prod
2023-01-15 23:30:16 +01:00
cc03376d86
added root tokens and endpoint for adding api keys ( #742 )
...
* added root tokens and endpoint for adding api keys
* Change down revision to current alembic head
* removed added_by_root_token
* refactored description
* fixed jinja errors
Co-authored-by: Andreas Köpf <andreas.koepf@provisio.com >
2023-01-15 21:24:25 +00:00
e58ffd64fa
add /api/v1/users/{user_id}/stats endpoint ( #744 )
...
* add /api/v1/users/{user_id}/stats endpoint
* return 0 stats and add /api/v1/users/{user_id}/stats/{time_frame}
* use utcnow() as modified date for 0 stats
2023-01-15 20:24:15 +00:00
cd49080c63
add response_model param to leaderboards endpoint
2023-01-15 16:28:46 +01:00
b5bb5bb7c0
Add leaderboard stats, periodic updates via fastapi-utils ( #724 )
...
* add leaderboard stats, periodic update via fastapi-utils
* count label tasks for assistant and prompter replies
* Daily stats update every 15 mins, simplify leaderboard endpoint
* add indices for some created_date columns
* make user stats update intervals configurable
* make sure intervals are positive
2023-01-15 12:04:19 +01:00
35e92dbd15
Supporting user pagination on the web admin view
2023-01-15 15:26:12 +09:00
69be679fe1
Merge pull request #701 from LAION-AI/673-enhanced-admin-routing
...
673 enhanced admin management
2023-01-15 08:34:09 +09:00
0f607c7a28
export and auto-commit docs/docs/api/openapi.json as part of test-api-contract workflow ( #719 )
...
- copies that openapi.json file to docs/docs/api/openapi.json as part of start-mock-server.sh.
- adds a step to the end of the test-api-contract workflow that just auto-commits changes to the docs/docs/api/openapi.json file.
- excludes docs/docs/api/openapi.json from pre-commit since its already been formatted by jq.
2023-01-14 22:40:33 +00:00
5cd9c630fc
Small typo fix + Consistent Docstrings ( #698 )
...
* Small typo fix + Consistent Docstrings
Also removed a redundant comment.
* undo comment removal
* Fixed indentation from previous commit
2023-01-14 17:24:02 +01:00
6547058778
add rank to message table and fix message_toxicity migration
2023-01-14 15:12:10 +01:00
a902c600fa
Store Message Toxicity in database ( #553 )
...
* [NEW] MessageToxicity table
* [NEW] Alembic message Toxicity
* [NEW] Model name enum
* [NEW] Refactor Enum HF
* [NEW] Settings: DEBUT_SKIP_TOXICITY_CALCULATION
* [NEW] Store toxicity values
* [FIX] Merge conflict
* [FIX] Documentation
* [NEW] save_toxicity: function
* [FIX] Formatted string
* [NEW] DEBUG_SKIP_TOXICITY_CALCULATION=True
* [FIX] HfClassificationModel
* [FIX] Alembic merge heads
* [NEW] Refactor save_toxicity
* [NEW] Separating score/label
* [NEW] Store score and label
* [FIX] Cleaning Alembic
* [NEW] Clean HF names
* [NEW] Not type hinting
* [NEW] Update alembic versions
* [NEW] Revert the changes
* [NEW] Type hinting label & score
* Updated down_revision in migration script
Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com >
2023-01-14 12:22:55 +00:00
8b8326e22f
add /by_display_name user query & created_date to FrontEndUser model
2023-01-14 11:03:25 +01:00
Andreas Köpf