wassname/FanFicFare
1
0
Fork 0
mirror of https://github.com/wassname/FanFicFare.git synced 2026-09-09 11:14:08 +08:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • f114a2438e This one might be controversial.. wsuetholz 2010-11-30 14:06:12 -06:00
  • 2f0bb31e24 Take out adding the id: field as a subject. wsuetholz 2010-11-30 13:56:18 -06:00
  • 30455bfda2 Updated the processing for fictionpress.com of the information line to correctly extract the Updated and Published dates, and the Completion status wsuetholz 2010-11-30 12:13:00 -06:00
  • cfb218f21e Moved the check for a completed story higher in the code, this makes the continue's that are in that portion of the if statement not skip that check. wsuetholz 2010-11-29 18:45:47 -06:00
  • 617dc21eb9 Kludge fix for uniode in story descs in appengine. retiefjimm 2010-11-27 17:32:31 -06:00
  • 2d7f6c2e5c Correct story status for ficwad, change story status for fictionalley to 'Unknown' (site doesn't tell us status), change mobi file output mode to 'wb'(binary) so it works on windows. retiefjimm 2010-11-26 19:36:53 -06:00
  • 719aed32a9 Tiny static changes - preparing for doing some SEO sigizmund 2010-11-26 15:55:04 +00:00
  • ccef466090 Change no-class div tags to p tags in mediaminer stories to get paragraph breaks. retiefjimm 2010-11-24 12:36:38 -06:00
  • 708f750bab Added Mobipocket support to the Web UI sigizmund 2010-11-24 09:37:49 +00:00
  • 3b0a2639e7 Merge because Hg is insistent. retiefjimm 2010-11-23 12:53:27 -06:00
  • edb4f5a3bd Change a couple of the example story URLs. retiefjimm 2010-11-23 12:49:57 -06:00
  • f3571959df Change a couple of the example story URLs. retiefjimm 2010-11-23 12:42:33 -06:00
  • ebcce7e42d Adding Mobi format and making final configuration changes before uploading a beta version. sigizmund 2010-11-23 07:15:18 +00:00
  • cf0d1bf09b Change output names for web version, make html & text output work for CLI version. retiefjimm 2010-11-22 20:05:58 -06:00
  • ecdeef4b3b Add new adapters to main.py, update verbiage on index, replace a couple prints with logging.debug. retiefjimm 2010-11-22 15:00:00 -06:00
  • dd483db783 Move books directory down into sub dir next to CLI. retiefjimm 2010-11-22 13:24:50 -06:00
  • 348786f073 cvreating a special tag so we can quickly restore to previous version sigizmund 2010-11-22 07:06:29 +00:00
  • 5549786946 Adding AppEngine files under the source control, moving downloaded code one directory deeper. sigizmund 2010-11-22 07:04:17 +00:00
  • f584b3f1e8 Change calls for str() to unicode(). Default CL type to epub if not given. before_adding_appengine retiefjimm 2010-11-21 11:52:11 -06:00
  • df6b85b723 Fix desc escaping. <sigh> retiefjimm 2010-11-18 23:34:25 -06:00
  • 26f6ddabd9 The with call is not an automatic in python 2.5, you need to import with_statement for it to work. the from __future__ import with_statement needs to be the first import statment as well. wsuetholz 2010-11-18 20:08:55 -06:00
  • 22d0989d0a Raise exceptions instead of an exit when errors are detected. Added static functions to the writer classes that contain the type name and type extension to use when writing out the destination file. Added a function to the zipdir that will check the date the fic was last updated in the current archive of a fic against the current last updated date and return whether the fic is newer then the archive file. This is not useful at this time for when this is running under appEngine. Added a flag to indicate if the program is running from appEngine or not. wsuetholz 2010-11-18 16:07:58 -06:00
  • 9b6fdb4540 Fix numeric entities--correctly this time. Reduce margins slightly, tweak description escaping slightly. retiefjimm 2010-11-18 14:26:16 -06:00
  • e3b4769bfb Misspelled the return variable from getOutputFileName. wsuetholz 2010-11-16 09:09:26 -06:00
  • 5618d8036b MediaMiner was not sending the date with the correct timezone info, so strip it off. We don't need the time anyways. wsuetholz 2010-11-13 23:30:47 -06:00
  • dbec6c95b2 Changed to deal with somebody putting in the mobile url for fanfiction.net... ie <http://m.fanfiction.net/s/11> will get changed to <http://www.fanfiction.net/s/11> wsuetholz 2010-11-12 21:51:46 -06:00
  • c77498e6ea Some changes in order to allow the refactoring of functions to work from within the appEngine. wsuetholz 2010-11-12 21:28:25 -06:00
  • 99ba1b48aa The getSubjects function was returning the author. wsuetholz 2010-11-12 20:27:17 -06:00
  • bdf429b41c Added in mediaminer.org adapter. Still has problems becaue mm seems to use the &#8820; type character encoding, and I couldn't manage to replace them. wsuetholz 2010-11-12 20:25:55 -06:00
  • 981a922d00 Deal with being passed in the first chapter as the url. wsuetholz 2010-11-11 13:53:48 -06:00
  • b51dd549f5 self.numChapters was one too many. wsuetholz 2010-11-11 13:22:30 -06:00
  • 35065593af Refactored source code to remove redundant code.. Moved most of the adapter get and set functions for variables into the adapter.py. Define all the adapter variables in adapter.py. Remove the unneeded functions from the various adapter sources. wsuetholz 2010-11-11 13:14:15 -06:00
  • 696223a456 Added in an adapter for FictionPress.com. This started out as a copy of ffnet.py, but the extractIndividualUrls rapidly became completly different. I have not found a field that says total number of words for the story. wsuetholz 2010-11-11 09:55:37 -06:00
  • 30b914788c Add a new adapter for the potionsandsnitches web site. Once we started looking for the additional information for the title page, things were sufficiently different from the twilighted adapter that it couldn't be used anymore. wsuetholz 2010-11-10 23:11:08 -06:00
  • ed4ff4b6ab Trying to fix outputting some utf-8 text. Added the utf-8 header to all the source code. Ended up modifying the removeEntities function to do a weird decode/encode step on the text passed in. This seems to at least stop things from crashing.. wsuetholz 2010-11-10 13:47:13 -06:00
  • 3f40df82dd Change to allow a chapter url to be passed in, yet still get the information it needs from the story index page. This will get the chapter links from the passed in url. wsuetholz 2010-11-10 12:36:21 -06:00
  • 7c252e83a1 It seems that self.numWords and self.numChapters can on occasion be numeric instead of string, so use the str() function when building the string for the title page. wsuetholz 2010-11-10 12:34:25 -06:00
  • d4c18fe1a8 Changed all references to downaloder.py to downloader.py. wsuetholz 2010-11-10 11:29:17 -06:00
  • b2ea9c3e73 Changed the title page generation to allow for easily adding and removing entries from the table of information shown. wsuetholz 2010-11-10 11:18:48 -06:00
  • 33d1fabd11 Changeset Tag Branch User Date Summary 53:c93e07566456 wsuetholz 2010-11-10 10:11 Ficwad wants you to login in order to view some stories.. They old ficwad.py got around that by starting with the first chapter instead of the story index page. Since I needed the story index page I had changed it to switch to that page, and then scrape the chapter information from there, which doesn't work if the chapters are blocked. While it still won't work if you pass in the URL for the story index page, I now switch back to the page that you passed in originally when looking for the chapters to download.. The one problem I have with this, is I change the self.url to the story index page, which should probably remain so that we have a consistent self.url even if the user starts with chapter 9 this time instead of chapter 1. wsuetholz 2010-11-10 10:18:46 -06:00
  • c326aa47b0 Need to reverse sort entities list to get entities with ';' ahead of versions without. Like '&quot;' and '&quot'. Otherwise '&quot;' becomes '";'. retiefjimm 2010-11-09 23:02:33 -06:00
  • d67668de23 Changes to the format of the title page. Tried to fix the description field on the title page and the contents.opf page. wsuetholz 2010-11-09 18:20:37 -06:00
  • 363c211401 Had left in a debugging statements that should have been removed.. wsuetholz 2010-11-09 18:18:42 -06:00
  • 379efc34f1 Add a title page to the resulting EPUB file. This required scraping more information from the web pages in order to populate the new fields. This change includes a change to the way that the output.py uses the adapters. It now gets passed in the adapter and then calls functions from the adapter in order to retrieve the scraped information. This will make it easier down the road when adding more information, or even pictures. wsuetholz 2010-11-09 16:35:46 -06:00
  • 778deaea00 Replace the haphazardly collected set of html entities with a list of known (browser) supported entities. Will now preserve right/left quotes. retiefjimm 2010-10-28 17:29:00 -05:00
  • 33c3f0b4fd Added tag fanficdownloader-0.2 for changeset 3eff15bceebe retiefjimm 2010-10-20 15:02:11 -05:00
  • 5bca3106df Adding a twipassword.py and a quick readme.txt. fanficdownloader-0.2 retiefjimm 2010-10-20 14:58:48 -05:00
  • c056ed0740 Fix zip routine so it checks for file mimetype doesn't just assume it exists. retiefjimm 2010-10-16 11:30:09 -05:00
  • 25cbab225c potionsandsnitches.net doesn't require login. Indeed, it doesn't even have the same login URL. retiefjimm 2010-10-14 12:42:29 -05:00
  • 4e53481d7d Somewhat kludgey fix for the kludgey way we extract the story text from fictionalley for stories with poor html. retiefjimm 2010-10-13 00:01:24 -05:00
  • b9a72d372d Add &#x0000; entity support, fix "<authorname> aa" from testing, add potionsandsnitches.net. retiefjimm 2010-10-11 20:44:19 -05:00
  • 7f8d7a7dbb Add '\n' after each </p> and <br /> tag. Extremely long(200k+) single line chapters (like ffnet gives) causes problems for nook. Also removed a little dead code from ffnet.py. retiefjimm 2010-10-11 14:05:57 -05:00
  • dafa33c64e Generalize handling of &#<number>; entities. Generalize method to create a valid filename. Remove print from fictionalley.py that was causing the output-before-302-header issue. Some changes to chapter URL formation due to apparent differences in handling '//' in URLs in appengine vs downalod.py in 2.5.2. retiefjimm 2010-10-10 12:54:41 -05:00
  • 275a1dd6ad Add &#039;->'\'' to entity translation list. retiefjimm 2010-10-08 21:54:35 -05:00
  • f8701a9e8d Fix to force mimetype file to be uncompressed and first in epub. Tested on Python 2.5.2. retiefjimm 2010-10-08 18:46:03 -05:00
  • 466d5f4ee3 Remove Entities, fix bar ampersands in story & author names. retiefjimm 2010-10-06 14:31:21 -05:00
  • ca4b09d69d Fixes for some obscure encoding errors. UTF8 versions of apostrophe and dash in story titles & chapter titles could cause problems in a few places. retiefjimm 2010-10-03 19:23:39 -05:00
  • f75910ce7d Stop doing prettify on the HTML, it sometimes introduces extraneous white space. Change the chapter file names and ids inside the metadata to sequential chapter0001 rather than base64 of the chapter title. Replace invalid center & u tags with div styled centered and span styled underlined. Remove dead ffa.py file. Change downloda.py to not call ffa.py, but do call fictionalley. retiefjimm 2010-09-30 21:33:58 -05:00
  • 773ff3c03c Improved metadata for epubcheck, fixed a problem I introduced with &gt;/&lt;. retiefjimm 2010-09-29 21:32:04 -05:00
  • 9434431c7b Add twilighted.net(not com) to downaloder.py so it actually *works*. Add one-shot support for twilighted.net. retiefjimm 2010-09-28 19:52:30 -05:00
  • 3be9b3331e Replace &#XXX; entities in file names, allow BR tags--some stories use multiple BRs are section breaks. retiefjimm 2010-09-28 19:26:19 -05:00
  • 84fea5b896 Fictionalley.com working, also add twilighted.com to downaloder.py. retiefjimm 2010-09-28 19:26:06 -05:00
  • fe67201f95 Fix up code for harrypotterfanfiction.com so it works and add one-shot support. Add &nbsp; &lt; and &gt; to entities list. retiefjimm 2010-09-28 13:26:49 -05:00
  • ccb3fb36c5 Allow '&amp;' and '&pound;'. Allow single chapter/one-shot stories from FicWad. Correct story names from FicWad. Stop creating empty 'Story Index' chapter in stories from FicWad. Exit on chapter d/l error from FicWad. retiefjimm 2010-09-26 19:08:49 -05:00
  • 31c68ff796 Exit on ff.net error downloading chapter. Replace ':' with '_' in file name for html output, too. Jimm 2010-09-25 11:55:31 -05:00
  • 950807747c Replace ':' in filename with '_' for Windoze. appspot version appears to already do this. retiefjimm 2010-09-25 00:44:56 -05:00
  • b22d961e73 Allow one-shot fics for ffnet. Write epub files as mode 'b' binary so it works on Windows. Allow hr tags in epub output. Don't compress file mimetype in epub file--required by standard. retiefjimm 2010-09-24 23:36:05 -05:00
  • 2b2e7d52ec Starting to use source control again after a long break :-) sigizmund 2010-07-17 14:44:14 +01:00
  • 48d561de1a Fixed multiline ff.net sigizmund 2009-12-21 13:40:11 +00:00
  • 1740e0da48 Changes while moving to AJAX service sigizmund 2009-12-21 13:12:47 +00:00
  • ed9e2992c0 hpfiction added - not yet working sigizmund 2009-12-18 18:57:53 +00:00
  • c123e77221 somemerges sigizmund 2009-12-18 18:56:44 +00:00
  • 5143a2187a fictionpress.com support added sigizmund 2009-12-18 18:55:57 +00:00
  • 1f897843e0 Whole lot of fixes related to appengine sigizmund 2009-12-18 14:51:53 +00:00
  • ef97755ed3 AppEngine version. Not quite clear is it working or n ot sigizmund 2009-12-18 12:26:23 +00:00
  • 33855b7877 zipdir with in-memory stuff added sigizmund 2009-12-18 11:40:29 +00:00
  • 615228a2ac changed slightly to make it work with AppEngine sigizmund 2009-12-18 11:23:37 +00:00
  • 028c2df521 Fixed a problem with &#[digits]; for fanfiction.net. Just why BeautifulSoup cannot handle them? sigizmund 2009-12-17 15:32:05 +00:00
  • 2352ba6dfc forgot about html_constants sigizmund 2009-12-17 15:18:31 +00:00
  • 59697f1a75 Added adapter.py, now all working adapters must implement FanfictionSiteAdapter interface sigizmund 2009-12-17 15:16:51 +00:00
  • a01e0e2ed6 Fixed few problems, including the slash in filename sigizmund 2009-12-17 14:37:31 +00:00
  • 8c621d6a59 Small bug in ff.net parsing was fixed sigizmund 2009-12-17 13:44:07 +00:00
  • f55e6d3c05 epub zip is now produced correctly, also TOC works with Stanza now sigizmund 2009-12-17 13:37:13 +00:00
  • d2d183430d small changes to make work on other machines. why there's no Python 2.6 on Ubuntu? sigizmund 2009-12-17 12:53:59 +00:00
  • 6e73e5dc11 login_password again sigizmund 2009-12-17 12:46:06 +00:00
  • fd2ccb30ff login_password sigizmund 2009-12-17 12:45:12 +00:00
  • 41caae6226 fanfiction.net support added sigizmund 2009-12-17 12:38:34 +00:00
  • 5adcf131b3 fanfiction.net support added kirillov 2009-12-17 12:38:15 +00:00
  • bcf82e2632 zipdir added sigizmund 2009-12-17 11:34:15 +00:00
  • 259c0f3650 sample change sigizmund 2009-12-17 11:30:23 +00:00
  • 861d1561d9 test change sigizmund 2009-12-17 11:29:10 +00:00
  • 6ef95c634d FFA changed, architecture changed, not quite ready yet sigizmund 2009-12-17 11:26:55 +00:00
  • a9748061f0 minor changes sigizmund 2009-12-15 15:26:45 +00:00
  • c0459faa43 added sigizmund 2009-12-15 15:23:48 +00:00