Found via `codespell -S .mypy_cache,yarn.lock,*.json,*.ipynb -L
rouge,nam,vie`
This commit is contained in:
Kian-Meng Ang
2023-02-05 20:18:03 +01:00
committed by GitHub
parent 364a4f5aa3
commit 1e321a6fca
32 changed files with 50 additions and 50 deletions
+2 -2
View File
@@ -57,7 +57,7 @@ conversation, or at least as a prompt with replies.
guarantee of the quality of the tweets.
- The tweet quality is the other major issue. We can get conversations through
the currently made scripts, but they most likely don't match a useful
instruction -> fulfilment. We are trying to filter the tweets through various
instruction -> fulfillment. We are trying to filter the tweets through various
means such as matching useful hashtags, or by using cosine similarity against
known instructions.
- The modern Twitter API has conversation_id as a field which can be a way to
@@ -68,7 +68,7 @@ conversation, or at least as a prompt with replies.
## TODO
- Write scripts to filter existing conversations into useful instructions ->
fulfilment with hashtags or cosine similarity.
fulfillment with hashtags or cosine similarity.
- Train model to detect if text is a suitable instruction. This could then be
run through the conversations (or full tweet dump) to simplify the process.
Related to issue #143.
@@ -9,7 +9,7 @@
# This assumes data downloaded from https://archive.org/details/twitterstream
# and that the internal .tar files are extracted locally.
# They are large files so using something like 7Zip or WinRar migth be easier
# They are large files so using something like 7Zip or WinRar might be easier
# than putting all of it in scripts, but it is a possibility.
# I often work in notebooks. If you encounter any issue, please reach out to let me know.