From a5d311b2183c36e150cf003d82d264628edbd1be Mon Sep 17 00:00:00 2001
From: AccidentallyOnPurpose
 <4935931+AccidentallyOnPurpose@users.noreply.github.com>
Date: Tue, 24 Dec 2019 04:20:11 +0000
Subject: [PATCH] writingprompts dataset

---
 DATASETS.md | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/DATASETS.md b/DATASETS.md
index 87a9db0..a8eec62 100644
--- a/DATASETS.md
+++ b/DATASETS.md
@@ -29,7 +29,10 @@ Once you have a bunch of scraped html files, you can convert them to txt files u
 
 # Datasets
 ------------------------
-The mormon used a dataset of CYOA texts that can still be found in the data folder of this repo. There are references to other datasets as well. A script used to scrape the /r/writingprompts subreddit, a subreddit where people post short stories. That would be good data if anyone can scrape it properly. Unfortunately reddit's API only shows the last 2,000 posts in any subreddit leaving the vast majority of posts hidden.
+The mormon used a dataset of CYOA texts that can still be found in the data folder of this repo. There are references to other datasets as well. 
+
+For example a script used to scrape the /r/writingprompts subreddit, a subreddit where people post short stories. We can get these files pre-scraped by facebook: 
+https://github.com/pytorch/fairseq/tree/master/examples/stories
 
 There are also references in the code to mechanical turk, a website used to pay thirdworlders to enter data for you for pennies.