added list of story links

2026-10-29 12:40:16 +08:00 · 2019-11-15 16:19:41 -07:00
1 parent eeb09f861d
commit 8ad2f570d0
@@ -0,0 +1 @@
+{"tree_id": "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=10638", "context": "", "first_story_block": "Chapter 1: Born to Kill\n\n\nYear 0\n\n\u201cIs this the one?\u201d\n\u201cYeah I guess.\u201d\n\u201cWhat do you mean you guess? It either is or it isn\u2019t.\u201d\n\u201cI dunno I lost the list.\u201d\n\u201cYou what? That\u2019s just great. You realize how much we\u2019re going to get our asses chewed out because of your fuck up? I\u2019m sick of paying for your mistakes.\u201d\n\u201cWill you calm down? How the hell are they gonna know anyway? You seen one ugly little baldy, you\u2019ve seen \u2018em all. All babies look the same.\u201d\n\u201cNot the elven ones.\u201d\n\u201cEnough with your elf fetish. These babies are all under the care of the Empire anyway, they\u2019re here for the taking.\n\u201cYeah, the key word being the Empire. You know we need the proper paper work, which you just fucking lost!\u201d\n\u201cSo what, I can forge it. You really worry too much, there\u2019s so much red tape they ain\u2019t gonna know the difference and what does it matter? The Empire\u2019s getting a body for their purposes anyway.\u201d\n\u201cYeah, but you know how they like the best.\u201d\n\u201dHow the fuck can they really tell at this age? They do all the real testing later. Look he\u2019s got all his limbs, and he doesn\u2019t display any signs of impurity. I\u2019d say he\u2019s a prime candidate for whatever they have in store for this poor little bastard.\u201d\n\u201cYeah I guess\u2026damn. I can\u2019t imagine what it\u2019s like being brought up like that.\u201d\n\u201cMe either, this program has only been going on a few years, but I\u2019ve already heard some fucked up stories about it\u2026y\u2019know its things like this that sometimes make me question the logic of our glorious leaders.\u201d\n\u201cShhhh! Are you mad? Don\u2019t talk like that! It\u2019s not our place to question anything, we\u2019re just supposed to do our jobs! Besides, the logic is quite obvious. If they train people from birth they\u2019ll have superior combat skills and utter loyalty to the Empire.\u201d\n\u201cThat\u2019s my point though. Is it really necessary? The Empire already produces so much propaganda that it does the job pretty well in producing loyal citizens and if that doesn\u2019t work it also instills so much fear in most that even if someone wanted to start a rebellion; it\u2019s highly unlikely they would.\u201d\n\u201cDon\u2019t be so sure, it has happened in the past.\u201d\n\u201cThe past? When? Like four hundred years ago when the Empire was still in its own infancy? Pfft. Wouldn\u2019t happen anymore and as for having elite combat troops, again what\u2019s the point? We\u2019ve already got camps to train elite groups like the Shadow Guard who are not to be fucked with and we have more than enough resources to create an army of golems that would be just as loyal and kick the shit out of our enemies.\u201d\n\u201cLook, I don\u2019t want to talk about this anymore! Our superiors have their reasons, as does the Emperor! Let us just take this baby and be done with this task!\u201d\n\nThe two clerks left the infirmary with you that day where upon you were accepted as a suitable candidate and put into The Eternal Program.\n\nYou don\u2019t remember too much during the time of your very young years, except that you are mostly confined to a room where all your basic needs are met. On a daily basis, a couple bigger people come in to check on you. They have you perform various physical tasks as well as spending time to teach you things like basic reading and writing. Sometimes they ask you various questions and write down your responses while other times they come in and stab you with painful needles with fluid in them. (You don\u2019t like those times)\n\nYou don\u2019t really understand it all, except that you know that the Emperor is always watching you. He is everywhere even if you don\u2019t see him. He is keeping an eye on your actions and you must always obey his will, lest you be punished. You know this to be true because you have this repeatedly told to you.\n\nEventually your relative isolation would come to an end.", "action_results": [{"action": "Years later...", "result": "Year 5\n\nYou are sitting in a gray room at a desk with several other children your age. All of you glance at each other a lot and it is obvious that there is much curiosity, but nobody speaks. This is very unusual for you, up until now, you have only had minimal contact with others. Most of them are male like you, but a few of them are female.\n\nSoon the silence comes to an end when a dark skinned woman walks into the room. She has long hair and sharp features. One thing you notice is that despite her long hair covering her ears you can still see that the tips of them are pointed. She must be elven, but in the pictures you\u2019ve seen in books, you\u2019ve never seen one with dark skin before.\n\n\u201cHello cLine truncated
@@ -29,10 +29,12 @@ class Scraper:
        exec_path = "/usr/bin/chromedriver"
        self.driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=exec_path)
        self.max_depth = 10
-        self.end_action = "End Game and Leave Comments"
+        self.end_actions = ["End Game and Leave Comments",
+                            "See How Well You Did (you can still back-page afterwards if you like)"]
        self.texts = set()

    def GoToURL(self, url):
+        self.texts = set()
        self.driver.get(url)
        time.sleep(0.5)

@@ -85,7 +87,7 @@ class Scraper:
        action_result["action_results"] = []

        for i, action in enumerate(actions):
-            if actions[i] != self.end_action and actions[i] != "Epilogue":
+            if actions[i] not in self.end_actions and actions[i] != "Epilogue":
                sub_action_result = self.BuildTreeHelper(result, i, depth, actions)
                if action_result is not None:
                    action_result["action_results"].append(sub_action_result)
@@ -105,7 +107,7 @@ class Scraper:
        story_dict["action_results"] = []

        for i, action in enumerate(actions):
-            if action != self.end_action:
+            if action not in self.end_actions:
                action_result = self.BuildTreeHelper(text, i, 0, actions)
                if action_result is not None:
                    story_dict["action_results"].append(action_result)
@@ -119,8 +121,30 @@ def save_tree(tree, filename):
 scraper = Scraper()

 urls = ["http://chooseyourstory.com/story/viewer/default.aspx?StoryId=10638",
-        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=11246"]
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=11246",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=54639",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=7397",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=8041",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=11545",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=7393",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=13875",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=37696",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=31353",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=31013",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=45375",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=41698",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=10634",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=42204",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=6823",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=18988",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=10359",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=5466",
+        "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=28030"
+        ]
+
+for i, url in enumerate(urls):
+    print("****** Extracting Adventure ", url, " ***********")
+    tree = scraper.BuildStoryTree(url)
+    save_tree(tree, "stories/story" + str(i) + ".json")

-tree = scraper.BuildStoryTree(urls[0])
-save_tree(tree, "scraped_story.json")
 print("done")
				`@@ -0,0 +1 @@`
				{"tree_id": "http://chooseyourstory.com/story/viewer/default.aspx?StoryId=10638", "context": "", "first_story_block": "Chapter 1: Born to Kill\n\n\nYear 0\n\n\u201cIs this the one?\u201d\n\u201cYeah I guess.\u201d\n\u201cWhat do you mean you guess? It either is or it isn\u2019t.\u201d\n\u201cI dunno I lost the list.\u201d\n\u201cYou what? That\u2019s just great. You realize how much we\u2019re going to get our asses chewed out because of your fuck up? I\u2019m sick of paying for your mistakes.\u201d\n\u201cWill you calm down? How the hell are they gonna know anyway? You seen one ugly little baldy, you\u2019ve seen \u2018em all. All babies look the same.\u201d\n\u201cNot the elven ones.\u201d\n\u201cEnough with your elf fetish. These babies are all under the care of the Empire anyway, they\u2019re here for the taking.\n\u201cYeah, the key word being the Empire. You know we need the proper paper work, which you just fucking lost!\u201d\n\u201cSo what, I can forge it. You really worry too much, there\u2019s so much red tape they ain\u2019t gonna know the difference and what does it matter? The Empire\u2019s getting a body for their purposes anyway.\u201d\n\u201cYeah, but you know how they like the best.\u201d\n\u201dHow the fuck can they really tell at this age? They do all the real testing later. Look he\u2019s got all his limbs, and he doesn\u2019t display any signs of impurity. I\u2019d say he\u2019s a prime candidate for whatever they have in store for this poor little bastard.\u201d\n\u201cYeah I guess\u2026damn. I can\u2019t imagine what it\u2019s like being brought up like that.\u201d\n\u201cMe either, this program has only been going on a few years, but I\u2019ve already heard some fucked up stories about it\u2026y\u2019know its things like this that sometimes make me question the logic of our glorious leaders.\u201d\n\u201cShhhh! Are you mad? Don\u2019t talk like that! It\u2019s not our place to question anything, we\u2019re just supposed to do our jobs! Besides, the logic is quite obvious. If they train people from birth they\u2019ll have superior combat skills and utter loyalty to the Empire.\u201d\n\u201cThat\u2019s my point though. Is it really necessary? The Empire already produces so much propaganda that it does the job pretty well in producing loyal citizens and if that doesn\u2019t work it also instills so much fear in most that even if someone wanted to start a rebellion; it\u2019s highly unlikely they would.\u201d\n\u201cDon\u2019t be so sure, it has happened in the past.\u201d\n\u201cThe past? When? Like four hundred years ago when the Empire was still in its own infancy? Pfft. Wouldn\u2019t happen anymore and as for having elite combat troops, again what\u2019s the point? We\u2019ve already got camps to train elite groups like the Shadow Guard who are not to be fucked with and we have more than enough resources to create an army of golems that would be just as loyal and kick the shit out of our enemies.\u201d\n\u201cLook, I don\u2019t want to talk about this anymore! Our superiors have their reasons, as does the Emperor! Let us just take this baby and be done with this task!\u201d\n\nThe two clerks left the infirmary with you that day where upon you were accepted as a suitable candidate and put into The Eternal Program.\n\nYou don\u2019t remember too much during the time of your very young years, except that you are mostly confined to a room where all your basic needs are met. On a daily basis, a couple bigger people come in to check on you. They have you perform various physical tasks as well as spending time to teach you things like basic reading and writing. Sometimes they ask you various questions and write down your responses while other times they come in and stab you with painful needles with fluid in them. (You don\u2019t like those times)\n\nYou don\u2019t really understand it all, except that you know that the Emperor is always watching you. He is everywhere even if you don\u2019t see him. He is keeping an eye on your actions and you must always obey his will, lest you be punished. You know this to be true because you have this repeatedly told to you.\n\nEventually your relative isolation would come to an end.", "action_results": [{"action": "Years later...", "result": "Year 5\n\nYou are sitting in a gray room at a desk with several other children your age. All of you glance at each other a lot and it is obvious that there is much curiosity, but nobody speaks. This is very unusual for you, up until now, you have only had minimal contact with others. Most of them are male like you, but a few of them are female.\n\nSoon the silence comes to an end when a dark skinned woman walks into the room. She has long hair and sharp features. One thing you notice is that despite her long hair covering her ears you can still see that the tips of them are pointed. She must be elven, but in the pictures you\u2019ve seen in books, you\u2019ve never seen one with dark skin before.\n\n\u201cHello cLine truncated