wassname/FanFicFare
1
0
Fork 0
mirror of https://github.com/wassname/FanFicFare.git synced 2026-09-11 11:52:07 +08:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • e202105248 Adding Google +1 button to see how it goes. Minor static fixes. sigizmund 2011-06-08 22:18:25 +01:00
  • 4f0ce05006 Added tag fanficdownloader-4.0.2 for changeset fbf33a024a38 Jim Miller 2011-06-07 20:12:32 -05:00
  • 57534c26c5 My fix for ffnet latest update chapters broke m. mobile support. Fixed. fanficdownloader-4.0.2 Jim Miller 2011-06-07 20:12:11 -05:00
  • df3b95aa6f Added tag fanficdownloader-4.0.2 for changeset 2d6980d864a9 Jim Miller 2011-06-07 19:50:01 -05:00
  • fd2d4a935d Bump to 4-0-2 primarily for new download version for ffnet/fpcom fix. Jim Miller 2011-06-07 19:49:49 -05:00
  • 71ae63e1ea Update index, remove new version notice, put ffnet/fpcom notice. Jim Miller 2011-06-07 14:26:31 -05:00
  • 2a353a74f9 Make ffnet/fpcom use BeautifulSoup (not Stone) for nested divs. Jim Miller 2011-06-07 14:19:25 -05:00
  • aa629c2c39 Merge ffnet/fpcom, make both use URL given to pull meta/chap list to avoid missing latest chapter due to out of date servers. Jim Miller 2011-06-07 13:00:39 -05:00
  • 520a295f88 Try both utf8 and windows-1252 (order depends on adapter), then strip non-ascii characters entirely if both fail. Jim Miller 2011-06-05 16:49:35 -05:00
  • 951bc3b030 Put orphan search in cron--it works now. Not that there were any. Jim Miller 2011-06-03 22:38:30 -05:00
  • 09d37c4426 Use cursor saved in memcache to search for orphan chunks. Smaller, ordered search doesn't work because ids are not increasing. Jim Miller 2011-06-03 22:09:59 -05:00
  • b76e812eaa Orphan search exceeds queued task DB timeout as well. Use smaller search like the regular cleanup and leave manual instead of cron. Jim Miller 2011-06-03 21:07:02 -05:00
  • ecabbd3af2 Use task queue for orphan chunk clean up. Jim Miller 2011-06-03 20:40:40 -05:00
  • 152ee61ed4 Now I remember why cleanup was limited--task timeout. Basic reverted to 100 each time, but still by time not day, orphan search will have to become a queue task--unscheduled for now. Jim Miller 2011-06-03 20:12:04 -05:00
  • f2283a2052 Improve download cleanup, add orphan chunk cleanup, add recent list clear, improve mediaminer no such story detect. Jim Miller 2011-06-03 20:03:14 -05:00
  • fb5740c549 Install hook to make urlfetch deadline 10s, add limited retry on fetch error. Jim Miller 2011-06-02 16:10:19 -05:00
  • 7133189601 Added tag fanficdownloader-4.0.1 for changeset 265d84461442 Jim Miller 2011-06-02 12:35:37 -05:00
  • 6d24caee1e Tweak status.html and test1.com. fanficdownloader-4.0.1 Jim Miller 2011-06-02 12:35:13 -05:00
  • 583275f037 Added tag fanficdownloader-4.0.1 for changeset 01d49b13cc3c Jim Miller 2011-05-30 23:02:16 -05:00
  • 50b80b6d68 4-0-1 Add mobi back in, changes to recent&status appearance, more reliability. Jim Miller 2011-05-30 23:02:00 -05:00
  • f7857ad6dc Suppress warnings about Django version. Jim Miller 2011-05-30 15:48:35 -05:00
  • a3f75cb3eb Improve error handling/reporting, make redownloads appear at top of recent list. Jim Miller 2011-05-30 15:05:00 -05:00
  • 94063d478c Pass normalized URL to queue task, not original url. Jim Miller 2011-05-30 09:59:37 -05:00
  • f25eae83d3 Save corrected URL in DownloadMeta record so status & recent have good URL. Jim Miller 2011-05-29 21:21:02 -05:00
  • 433e371b74 Enable (www|m). for fictionpress.com, like ff.net. Jim Miller 2011-05-29 17:03:02 -05:00
  • def2937327 Save appengine version in DownloadMeta to aid troubleshooting. Jim Miller 2011-05-29 13:07:12 -05:00
  • e99adf3fa9 Allow trailing / after /Story_Title on ff.net, fp.com urls. Jim Miller 2011-05-29 10:44:46 -05:00
  • a05bf24ed3 Fix typo that prevented some proper fictionpress.com urls from working. Jim Miller 2011-05-28 21:28:54 -05:00
  • aa58f1c1ee Make much more forgiving of inexact URLs--that's most of our errors now. Jim Miller 2011-05-28 16:09:34 -05:00
  • 35aeae59f2 Tweaking logging slightly. Jim Miller 2011-05-27 20:43:36 -05:00
  • 6d5785a5ac Added tag fanficdownloader-4.0.0 for changeset 3dbb4938aead Jim Miller 2011-05-23 18:33:56 -05:00
  • 07fbb8c5d5 Bring 4.0.0 branch into trunk. fanficdownloader-4.0.0 Jim Miller 2011-05-23 18:26:24 -05:00
  • ac74918005 Make this default appengine version, downloader.py honors -f, remove extratags from default title page. Jim Miller 2011-05-23 17:36:56 -05:00
  • b85a5ab95b Include announcement of new version. Jim Miller 2011-05-23 17:35:24 -05:00
  • 215552af00 Strip html tags and entities from epub chapter titles. Jim Miller 2011-05-20 09:56:53 -05:00
  • 7f0c989319 Reduce logging level of normal system exceptions. Jim Miller 2011-05-19 20:36:19 -05:00
  • d38360be6b Fix zip output file name. Jim Miller 2011-05-19 17:30:41 -05:00
  • bf52020876 Bump html output title size back up to h1. Jim Miller 2011-05-19 10:35:41 -05:00
  • 58b95af0da Fixes for various problems found during testing, remove old files. Beta ready. Jim Miller 2011-05-16 12:38:42 -05:00
  • df9e877865 Fixes to various from first round testing and some code clean up. Jim Miller 2011-05-13 21:34:58 -05:00
  • a88d4cac50 Support for fictionalley.org, add feature to not overwrite files newer than last update, fix to config section precedence, consolidate date parsing a bit. Jim Miller 2011-05-11 22:44:18 -05:00
  • 756b6944fc Support for mediaminer.org, update copyrights. Jim Miller 2011-05-11 12:29:59 -05:00
  • 6d2f657729 Support for thewriterscoffeeshop.com, allow both login/pass and Are Adult in one site. Jim Miller 2011-05-10 13:26:34 -05:00
  • 9b2450a609 Support for twiwrite.net, potionsandsnitches.com & harrypotterfanfiction.com. Jim Miller 2011-05-09 21:20:06 -05:00
  • 36d4b9afcf Add adastrafanfic.com support and support to confirm adult w/o login/pass. Jim Miller 2011-05-09 14:46:05 -05:00
  • d90c9ebced Support for ficwad.com. Further tweaks/improvments, especially to user config. (I'm getting a bit carried away with making things configurable, honestly.) Jim Miller 2011-05-08 21:53:06 -05:00
  • ecd1690b70 Add fictionpress.com adapter, some more tweak/fixes. Jim Miller 2011-05-07 15:14:46 -05:00
  • 87a57cc286 Add (partial) CLI options, fixes to site editconfig. Jim Miller 2011-05-07 12:51:03 -05:00
  • d91f2e74fd Only ask for Login/Password when required. Save and use user configuration. Jim Miller 2011-05-06 20:33:54 -05:00
  • d3b1ddcda9 Added tag fanficdownloader-3.0.2 for changeset 564ada569d46 Jim Miller 2011-05-06 19:59:04 -05:00
  • a7b80388dd Merge changes from trunk to 4-0 branch. Jim Miller 2011-05-06 13:00:32 -05:00
  • 9515ff00be Tweak output a bit, allow any 'entry' as epub subject. Jim Miller 2011-05-06 12:42:20 -05:00
  • 605c4c40fc Mis-aligned ad block fanficdownloader-3.0.2 sigizmund 2011-05-06 12:00:50 +01:00
  • 3e1f0dd53b Fixing screwed Google Analytics tracking code. I don't know how, but it nearly disappeared and our impressions went down from 250+ a day to around 30. sigizmund 2011-05-06 11:24:27 +01:00
  • 0e28d86a7b Fix for zip files, status in epub. Jim Miller 2011-05-05 20:47:39 -05:00
  • 94669a2179 Fix up exception handling, first working appengine(SDK) version. Jim Miller 2011-05-04 17:54:36 -05:00
  • 547411666d Move CLI up a level to match appengine, change module packaging to suit. Change mechanism for loading adapters to avoid problems with module init. Move adapter exceptions to own file. Jim Miller 2011-05-04 14:29:24 -05:00
  • 150316f460 Commit first version of reorg/rewrite. Currently CLI only. Jim Miller 2011-05-03 11:27:58 -05:00
  • f75abaad72 3-0-2 Add convert link feature. Remove title page table(for convert). Make favicon.ico work. Jim Miller 2011-05-02 13:54:48 -05:00
  • bdaea79875 Previous change to elim newlines in debug sometimes output breaks download. Cheap fix, comment out debugs. Jim Miller 2011-05-02 13:12:16 -05:00
  • 7420d27652 More log clean up, better handling of bad URL in ficwad, missing story in ffnet. Jim Miller 2011-04-26 17:22:51 -05:00
  • 1426626d0a Strip leading newline so it doesn't (falsely) show up in appengine logs as error. Jim Miller 2011-04-25 19:57:40 -05:00
  • 666e9666cd Attempting to help with ffnet failures. Increase fetch deadline, add retries with backoff, 1/2 sleep. Also remove <> from error message--HTML eats it. Jim Miller 2011-04-22 17:03:43 -05:00
  • 1c2c1a5d62 Add support for twiwrite.net. Requires User/Pass like twilighted.net. Jim Miller 2011-04-20 22:34:59 -05:00
  • 7c5aaa36d5 Save recent downloads longer(5 days, was 2). Fix some redirects. Reduces log errors and makes auto-re-login more transparent. Jim Miller 2011-04-20 17:32:30 -05:00
  • 13b96946d1 Improving logging, can dump the data which was downloaded if download fails. sigizmund 2011-04-19 12:49:37 +01:00
  • 1119ba52a0 Replaced ad block sigizmund 2011-04-18 11:07:56 +01:00
  • b058477b97 Fix so recent, but pre-compress mobis will still download. Jim Miller 2011-04-03 16:55:51 -05:00
  • 1f303013bd Fix to allow retrying, compress mobi chunks, Working/Finished/Failed in status.html title, keep 3 days worth. Jim Miller 2011-04-03 16:47:08 -05:00
  • 302a8f12a3 Removing storage for old version. Fixes for whofic meta data. Jim Miller 2011-04-01 22:14:05 -05:00
  • 3e381dd33f Still tweaking the clean up cron. retiefjimm 2011-03-24 22:09:11 -05:00
  • bd82311d51 Mobi improvements: Mark TOC so reader can find it, don't let reader eat every 4096th char, don't prettify. Remove debug outputs from whofic. Increase fetch size of remover. retiefjimm 2011-03-24 12:57:38 -05:00
  • efb521c829 Added tag fanficdownloader-0.5 for changeset 0f08ff79de17 retiefjimm 2011-03-22 19:45:47 -05:00
  • 925324bd13 Improved (but not perfected) mobi file output. Reduce cron clean up time to 2 days. fanficdownloader-0.5 retiefjimm 2011-03-22 19:17:41 -05:00
  • 1d1ed1ef71 Add support for www.whofic.com. retiefjimm 2011-03-22 19:11:17 -05:00
  • 389fc010f2 After merging QueueProc changes, tweaks clean up cron times and change appspot instance name. retiefjimm 2011-03-22 12:59:12 -05:00
  • 5ab5991d6e Merging QueueProc changes to trunk. Jim Miller 2011-03-22 12:48:17 -05:00
  • 8c606e45fd Remove 2 sec sleep before making live version. retiefjimm 2011-03-17 17:03:25 -05:00
  • e645c054d8 adastrafanfic can have chapters with tags (<i>) in them. Allow get as well as post. retiefjimm 2011-03-15 14:49:16 -05:00
  • e6ebd956f8 Datastore max size is 1,000,000, not 1M(=2^20). retiefjimm 2011-03-10 14:30:42 -06:00
  • ef5e7700b9 Back out some changes I didn't actually mean to commit. Just some custom stuff. retiefjimm 2011-03-10 13:14:51 -06:00
  • 53e2a0f204 First version with background queue processing and support for stories >1M. retiefjimm 2011-03-10 13:00:11 -06:00
  • 0e9d992e13 Make Branch for conversion to Queue Processing. retiefjimm 2011-03-04 11:58:53 -06:00
  • 790aca39da Add epubmerge.py, a standalone CLI program to merge multiple epubs together into one. retiefjimm 2011-02-24 18:02:49 -06:00
  • 7064873b4e Put title, author and publisher into mobi files properly as meta data. Also strips non-ascii chars from t,a,p--caused problems with mobi gen. Calibre's book reader cannot read the mobi files produced from fanficdownloader, either before or after these changes. retiefjimm 2011-01-23 13:37:29 -06:00
  • 95e7ba636d Add commas to word counts. retiefjimm 2011-01-15 10:41:28 -06:00
  • fe557bbce5 Fixed downloading mobi from fdown sigizmund 2011-01-03 20:35:15 +00:00
  • 783a614345 Enable collection of user's own user/password again. Only used by twilighted.net. retiefjimm 2010-12-27 11:29:12 -06:00
  • a32ccf7dac Work around an SGMLParser/BeautifulStoneSoup entity bug--incorrectly inserts (;) when it shouldn't, "AT&T" becomes "AT&T;". Also update to latest BeautifulSoup: has one minor, unrelated change. retiefjimm 2010-12-19 12:21:55 -06:00
  • d3abfbf5d6 Add Star Trek fanfic site adastrafanfic.com, plus add filename(linenum) to logger output for CLI. retiefjimm 2010-12-13 23:31:35 -06:00
  • d95c72f447 Split info line in ffnet by ' - ' instead of just '-' so Sci-Fi doesn't become just 'Sci'. fpcom already does. retiefjimm 2010-12-08 17:40:10 -06:00
  • e05749678e Move status->subject into output.py as per Bill's suggestion. retiefjimm 2010-12-06 15:03:07 -06:00
  • f7d94105ee Correct example URL. retiefjimm 2010-12-05 21:40:04 -06:00
  • 50b0573033 Add status (In-Progress or Completed) to metadata as a Subject tag. retiefjimm 2010-12-05 21:38:16 -06:00
  • 48b882c745 2 second sleep was in Mediaminer as well. wsuetholz 2010-12-05 21:21:44 -06:00
  • 3fdd079b97 Merge with 8d1772310e5c4ad1772a92b1f0408952b0b370e2 wsuetholz 2010-12-05 20:58:38 -06:00
  • e674f7c7df There was still a 2 second sleep between chapters for the fictonpress adapter.. Keep in mind that if you do a lot of downloading from fanfiction.net or fictionpress.com it is possible that your IP address could be locked without the sleep in place. wsuetholz 2010-12-05 20:52:13 -06:00
  • 8f58164dde Added tag fanficdownloader-0.4 for changeset f23fd0e4cbc3 retiefjimm 2010-12-02 17:36:52 -06:00
  • 4c99fdbcc1 Removed some leftover functions that were left from using ffnet.py as a base for fpcom.py and mediaminer.py. wsuetholz 2010-11-30 15:17:00 -06:00
  • c6b79cb57b Changed the "Title Page" reference in the TOC and CONTENT epub files to be title_page instead. fanficdownloader-0.4 wsuetholz 2010-11-30 15:11:20 -06:00