mirror of
https://github.com/wassname/scrape_r_rational.git
synced 2026-08-20 12:50:33 +08:00
readme
This commit is contained in:
@@ -18,7 +18,5 @@ CLIENT_ID=XXXXXXXXXXXXXXX
|
||||
CLIENT_SECRET=XXxxx111xxxXXXXXXXXXX
|
||||
PASSWORD=XXXXxxXX1XXX
|
||||
|
||||
# LLM API keys
|
||||
OPENAI_API_KEY=sk-proj-XXX
|
||||
# from https://openrouter.ai/settings/keys
|
||||
OPENROUTER_API_KEY=sk-or-v1-XXXX
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# rrational
|
||||
|
||||

|
||||
|
||||
scrapping reddit.com/r/rational and analytics
|
||||
|
||||
see https://raw.githubusercontent.com/NightMachinery/.shells/master/scripts/python/reddit/subreddit2org.py
|
||||
@@ -8,18 +10,21 @@ This project has data from r/rational in plan text that you can browse, see <./d
|
||||
|
||||
This project also has a table you can at https://wassname.github.io/scrape_r_rational/
|
||||
|
||||
https://wassname.github.io/scrape_r_rational/
|
||||
|
||||
## Project plan:
|
||||
|
||||
- [x] Init
|
||||
- [x] Fill out README
|
||||
- [x] Scrape r/rational
|
||||
- [x] use [statistics](https://github.com/wassname/scrape_r_rational/blob/main/nbs/links.csv)
|
||||
- [ ] Use llm to get reccomendations, sentiment, karma etc
|
||||
- [x] Use llm to get reccomendations, sentiment, karma etc
|
||||
- [x] share
|
||||
- [x] comment md to html
|
||||
- [x] comment expand
|
||||
- [x] threads where it's mentioned
|
||||
- [x] better tittles and data cleaning
|
||||
- [x] github pages
|
||||
|
||||
|
||||
## Install requirements
|
||||
@@ -42,3 +47,6 @@ First run <nbs/mjc_001_download.ipynb> to update the data in <data/cache2/>
|
||||
|
||||
Then run <nbs/mjc_004_process.ipynb> to analyse the data and output <index.html>
|
||||
|
||||
05 to run an llm (costs around $50)
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 202 KiB |
Reference in New Issue
Block a user