Commit Graph

30 Commits

Author SHA1 Message Date
Yannic Kilcher 90c3d5640e Added database to inference server (#1446)
* added db for inference

* fixed dockerfiles for inference
2023-02-10 22:51:35 +01:00
Andreas Köpf dfeb969884 move shm_size setting to playbook 2023-02-10 08:27:32 +01:00
Andreas Köpf 164617c445 increase shared-mem setting of backend postgres container to 512mb 2023-02-10 07:49:17 +01:00
Yannic Kilcher 4f821f0f8b switched to HF text-generation-inference 2023-02-09 00:11:05 +01:00
notmd 2d2ea2039c Merge branch 'main' into 911_control_email_signin 2023-02-03 13:31:18 +07:00
Andreas Köpf d5c0adba5f fix typo SERCERT -> SECRET 2023-02-02 22:10:50 +00:00
notmd 0b9b8ed927 allow to enable/disable email sigin via env variable 2023-02-02 00:29:46 +07:00
Keith Stevens 04e81edf5c Merge pull request #932 from notmd/911_sigin_captcha
911 signin captcha
2023-02-01 15:56:14 +09:00
notmd fb26a52c2c allow to enable/disable email signin captcha via env variable 2023-01-29 23:28:26 +07:00
Alan Jean e59b73d942 chore(inference): target dev stage at build time 🐳 2023-01-29 05:30:00 +04:00
Alan Jean 0ab56041b6 feat(inference): integrate inference stack with the unified compose descriptor 🐳 2023-01-29 05:30:00 +04:00
notmd 3c9e5388ba add captcha to signin page 2023-01-26 01:00:31 +07:00
Yannic Kilcher 137f733fef added oasst postgres 2023-01-17 11:30:52 +01:00
Yannic Kilcher 5d441b1570 prepared playbook for prod 2023-01-15 23:30:16 +01:00
Nil Andreu a902c600fa Store Message Toxicity in database (#553)
* [NEW] MessageToxicity table

* [NEW] Alembic message Toxicity

* [NEW] Model name enum

* [NEW] Refactor Enum HF

* [NEW] Settings: DEBUT_SKIP_TOXICITY_CALCULATION

* [NEW] Store toxicity values

* [FIX] Merge conflict

* [FIX] Documentation

* [NEW] save_toxicity: function

* [FIX] Formatted string

* [NEW] DEBUG_SKIP_TOXICITY_CALCULATION=True

* [FIX] HfClassificationModel

* [FIX] Alembic merge heads

* [NEW] Refactor save_toxicity

* [NEW] Separating score/label

* [NEW] Store score and label

* [FIX] Cleaning Alembic

* [NEW] Clean HF names

* [NEW] Not type hinting

* [NEW] Update alembic versions

* [NEW] Revert the changes

* [NEW] Type hinting label & score

* Updated down_revision in migration script

Co-authored-by: Andreas Köpf <andreas.koepf@xamla.com>
2023-01-14 12:22:55 +00:00
Andreas Köpf 14fa08e2e7 Message tree state machine (#555)
* add query_incomplete_rankings()

* Add SQL queries for TreeManager task selection

* first working version of TreeManager.next_task()

* remove old generate_task(), add mandatory_labels to text_labels task

* Add ConversationMessage list to Ranking tasks

* add more sophisticated sql queries to find extendible trees

* add TreeManager.query_extendible_parents()

* fix task validation, seed data insertion (reviewed)

* provide user for task selection in text-frontend

* enter 'growing' state

* enter 'aborted_low_grade' state

* enter 'ranking' state

* check tree 'growing' state upon relpy insertion

* exclude user from labeling their own messages (added DEBUG_ALLOW_SELF_LABELING setting)

* add DEBUG_ALLOW_SELF_LABELING to docker-compose.yaml

* fix ranking submission

* add query_tree_ranking_results()

* add ranked_message_ids to RankingReactionPayload

* fix reply_messages instead of prompt_messages

* incorment 'ranking_count' of ranked replies

* added logic to check_condition_for_scoring_state

* changes to msg_tree_state_machine

* pre-commit changes

* enter 'ready_for_scoring' state

* re-add HF embedding call (lost during merge)

* use prepare_conversation() helper for seed-data creation

* Partially add user specified task selection

Co-authored-by: Daniel Hug <danielpatrickhug@gmail.com>
2023-01-11 10:54:03 +01:00
Yannic Kilcher 942dc42d03 Merge pull request #439 from othrayte/ci-e2e-basic
Automate e2e test of the website (basic)
2023-01-10 00:23:20 +01:00
jojopirker 11d55d572a message embeddings in Messages table 2023-01-08 12:28:38 +01:00
Adrian Cowan c369675991 website: Automate e2e testing with a simpler method than #376 2023-01-07 22:14:37 +11:00
Keith Stevens dbfa77e8b7 Fixing some small merge issues and ensure docker runs the backend properly 2023-01-03 15:01:24 +09:00
kiritowu 0c1eaeb066 Update docker compose to start redis on backend-dev and update REDIS_HOST in compose environment 2023-01-03 00:28:05 +08:00
Gareth Davidson 0b1059f53e fix compose file so it runs in both "docker-compose" and "docker compose" 2023-01-01 23:03:07 +00:00
Graeme Harris 3bb0b04f92 Removed local volume from redis image 2023-01-01 13:35:04 +02:00
Graeme Harris 743de0bb18 Linting + prettier 2023-01-01 13:35:04 +02:00
Graeme Harris 5a18fba9e6 Added basic redis + redis insights config for docker and ansible 2023-01-01 13:35:04 +02:00
Andreas Köpf 30db7254c7 rename flag to DEBUG_USE_SEED_DATA default False, update compose config 2022-12-29 15:58:35 +01:00
Yannic Kilcher 0acacf33fe Merge branch 'main' into add-debug-skip-password-env 2022-12-29 13:34:28 +01:00
Yannic Kilcher 0ee3196064 added sleep instead of true containers in compose 2022-12-29 00:38:12 +01:00
Yannic Kilcher cbcbe8d708 added environment variable DEBUG_LOGIN to display debug credentials provider even in production mode 2022-12-28 17:33:33 +01:00
croumegous 64206895ff Unified multiple Docker Compose files into one for easier usage 2022-12-28 13:26:30 +01:00