mirror of
https://github.com/wassname/scrape_r_rational.git
synced 2026-09-09 11:33:49 +08:00
Update README.md
This commit is contained in:
@@ -11,43 +11,6 @@ scrapping reddit.com/r/rational and analytics
|
||||

|
||||
|
||||
|
||||
## Project plan:
|
||||
|
||||
- [x] Init
|
||||
- [x] Fill out README
|
||||
- [x] Scrape r/rational
|
||||
- [x] use [statistics](https://github.com/wassname/scrape_r_rational/blob/main/nbs/links.csv)
|
||||
- [x] Use llm to get reccomendations, sentiment, karma etc
|
||||
- [x] share
|
||||
- [x] comment md to html
|
||||
- [x] comment expand
|
||||
- [x] threads where it's mentioned
|
||||
- [x] better tittles and data cleaning
|
||||
- [x] github pages
|
||||
|
||||
|
||||
## Install requirements
|
||||
|
||||
This project uses [poetry](https://python-poetry.org/) for requirement and is set up for torch using cuda.
|
||||
~~~
|
||||
poetry install
|
||||
~~~
|
||||
|
||||
Then
|
||||
~~~
|
||||
cp .env.example .env
|
||||
~~~
|
||||
Then fill out the api keys
|
||||
|
||||
|
||||
## How to run
|
||||
|
||||
First run <nbs/mjc_001_download.ipynb> to update the data in <data/cache2/>
|
||||
|
||||
Then run <nbs/mjc_004_process.ipynb> to analyse the data and output <index.html>
|
||||
|
||||
use <nbs/mjc_005_allm.ipynb> to run an llm from openrouter (costs around $50) and the results are... OK
|
||||
|
||||
## More info:
|
||||
|
||||
Reddit Discussion: https://old.reddit.com/r/rational/comments/1hoonrc/v2_table_which_stories_have_been_linked_most/
|
||||
@@ -88,3 +51,41 @@ For the Table UI I've included
|
||||
- save states
|
||||
- export to excel
|
||||
- column visibility
|
||||
|
||||
|
||||
## Project plan:
|
||||
|
||||
- [x] Init
|
||||
- [x] Fill out README
|
||||
- [x] Scrape r/rational
|
||||
- [x] use [statistics](https://github.com/wassname/scrape_r_rational/blob/main/nbs/links.csv)
|
||||
- [x] Use llm to get reccomendations, sentiment, karma etc
|
||||
- [x] share
|
||||
- [x] comment md to html
|
||||
- [x] comment expand
|
||||
- [x] threads where it's mentioned
|
||||
- [x] better tittles and data cleaning
|
||||
- [x] github pages
|
||||
|
||||
|
||||
## Install requirements
|
||||
|
||||
This project uses [poetry](https://python-poetry.org/) for requirement and is set up for torch using cuda.
|
||||
~~~
|
||||
poetry install
|
||||
~~~
|
||||
|
||||
Then
|
||||
~~~
|
||||
cp .env.example .env
|
||||
~~~
|
||||
Then fill out the api keys
|
||||
|
||||
|
||||
## How to run
|
||||
|
||||
First run <nbs/mjc_001_download.ipynb> to update the data in <data/cache2/>
|
||||
|
||||
Then run <nbs/mjc_004_process.ipynb> to analyse the data and output <index.html>
|
||||
|
||||
use <nbs/mjc_005_allm.ipynb> to run an llm from openrouter (costs around $50) and the results are... OK
|
||||
|
||||
Reference in New Issue
Block a user