mirror of
https://github.com/wassname/FanFicFare.git
synced 2026-09-16 11:47:12 +08:00
Compare commits
32
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
38ab66907b | ||
|
|
1dedde2f9d | ||
|
|
fc5dd20853 | ||
|
|
4e9fe9f86f | ||
|
|
7325be98f5 | ||
|
|
e2545a0da8 | ||
|
|
5cd3dba4a6 | ||
|
|
bc2420d143 | ||
|
|
099e8a62ca | ||
|
|
7f9e22e1b9 | ||
|
|
a46532bc72 | ||
|
|
f7429140f8 | ||
|
|
7872b25034 | ||
|
|
3d468a4dc5 | ||
|
|
413525a881 | ||
|
|
c066a809c7 | ||
|
|
443b2b5ef8 | ||
|
|
3e7f05ba44 | ||
|
|
25e65da239 | ||
|
|
c5f211f49f | ||
|
|
20fff58813 | ||
|
|
8fe250747c | ||
|
|
4d41c28f3c | ||
|
|
f8d1b21007 | ||
|
|
72861e6cd5 | ||
|
|
bd9dba5813 | ||
|
|
1478df572e | ||
|
|
68466af944 | ||
|
|
7c9c84256c | ||
|
|
fcbfe96c64 | ||
|
|
e47748dbbc | ||
|
|
91a6c668d1 |
@@ -1,6 +1,6 @@
|
||||
# ffd-retief-hrd fanfictiondownloader
|
||||
application: fanfictiondownloader
|
||||
version: 4-4-86
|
||||
version: 4-4-90
|
||||
runtime: python27
|
||||
api_version: 1
|
||||
threadsafe: true
|
||||
|
||||
@@ -42,8 +42,8 @@ class FanFictionDownLoaderBase(InterfaceActionBase):
|
||||
description = _('UI plugin to download FanFiction stories from various sites.')
|
||||
supported_platforms = ['windows', 'osx', 'linux']
|
||||
author = 'Jim Miller'
|
||||
version = (1, 8, 02)
|
||||
minimum_calibre_version = (0, 8, 57)
|
||||
version = (1, 8, 07)
|
||||
minimum_calibre_version = (1, 13, 0)
|
||||
|
||||
#: This field defines the GUI plugin class that contains all the code
|
||||
#: that actually does something. Its format is module_path:class_name
|
||||
|
||||
@@ -1393,7 +1393,10 @@ class FanFictionDownLoaderPlugin(InterfaceAction):
|
||||
if prefs['mark'] or (prefs['errorcol'] != '' and prefs['errorcol'] in custom_columns):
|
||||
self.previous = self.gui.library_view.currentIndex() # used by update_books_finish.
|
||||
self.gui.status_bar.show_message(_('Adding/Updating %s BAD books.')%len(book_list))
|
||||
label = custom_columns[prefs['errorcol']]['label']
|
||||
if (prefs['errorcol'] != '' and prefs['errorcol'] in custom_columns):
|
||||
label = custom_columns[prefs['errorcol']]['label']
|
||||
else:
|
||||
label = None
|
||||
LoopProgressDialog(self.gui,
|
||||
book_list,
|
||||
partial(self.update_error_column_loop, db=self.gui.current_db, label=label),
|
||||
@@ -1402,12 +1405,10 @@ class FanFictionDownLoaderPlugin(InterfaceAction):
|
||||
win_title=_("Update calibre for BAD FanFiction stories"),
|
||||
status_prefix=_("Updated"))
|
||||
|
||||
def update_error_column_loop(self,book,db=None,label='errorcol'):
|
||||
if book['calibre_id']:
|
||||
custom_columns = self.gui.library_view.model().custom_columns
|
||||
if (prefs['errorcol'] != '' and prefs['errorcol'] in custom_columns):
|
||||
logger.debug("add/update bad %s %s %s"%(book['title'],book['url'],book['comment']))
|
||||
db.set_custom(book['calibre_id'], book['comment'], label=label, commit=True)
|
||||
def update_error_column_loop(self,book,db=None,label=None):
|
||||
if book['calibre_id'] and label:
|
||||
logger.debug("add/update bad %s %s %s"%(book['title'],book['url'],book['comment']))
|
||||
db.set_custom(book['calibre_id'], book['comment'], label=label, commit=True)
|
||||
|
||||
def add_book_or_update_format(self,book,options,prefs,mi=None):
|
||||
db = self.gui.current_db
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+197
-175
@@ -538,6 +538,13 @@ extratags: FanFiction,Testing,HTML
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## archiveofourown.org stories allow chapters to be added out of
|
||||
## order. So the newest chapter may not be the last one. FFDL update
|
||||
## doesn't like that. If do_update_hook is uncommented and set true,
|
||||
## the adapter will discard all existing chapters from the newest one
|
||||
## on when updating to enforce accurate chapters.
|
||||
#do_update_hook:false
|
||||
|
||||
## AO3 adapter defines a few extra metadata entries.
|
||||
## If there's ever more than 4 series, add series04,series04Url etc.
|
||||
extra_valid_entries:fandoms,freeformtags,freefromtags,ao3categories,comments,kudos,hits,bookmarks,collections,series00,series01,series02,series03,series00Url,series01Url,series02Url,series03Url
|
||||
@@ -562,11 +569,6 @@ include_in_freefromtags:freeformtags
|
||||
#extra_subject_tags:fandoms,freeformtags,ao3categories
|
||||
|
||||
[ashwinder.sycophanthex.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Hermione Granger
|
||||
extraships:Severus Snape/Hermione Granger
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -574,6 +576,11 @@ extraships:Severus Snape/Hermione Granger
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Hermione Granger
|
||||
extraships:Severus Snape/Hermione Granger
|
||||
|
||||
[asr3.slashzone.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:The Sentinel
|
||||
@@ -584,9 +591,6 @@ extracategories:The Sentinel
|
||||
#is_adult:true
|
||||
|
||||
[bloodties-fans.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Blood Ties
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -599,6 +603,9 @@ extracategories:Blood Ties
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Blood Ties
|
||||
|
||||
[buffynfaith.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Buffy: The Vampire Slayer
|
||||
@@ -609,9 +616,6 @@ extracategories:Buffy: The Vampire Slayer
|
||||
#is_adult:true
|
||||
|
||||
[castlefans.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Castle
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -624,6 +628,9 @@ extracategories:Castle
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Castle
|
||||
|
||||
[chaos.sycophanthex.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
@@ -636,10 +643,23 @@ cover_exclusion_regexp:/images/.*?ribbon.gif
|
||||
|
||||
[dark-solace.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
extracategories:Buffy: The Vampire Slayer
|
||||
extracharacters:Buffy, Spike
|
||||
extraships:Spike/Buffy
|
||||
|
||||
[dramione.org]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -652,24 +672,11 @@ extraships:Spike/Buffy
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[dramione.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Draco Malfoy,Hermione Granger
|
||||
extraships:Draco Malfoy/Hermione Granger
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## some sites include images that we don't ever want becoming the
|
||||
## cover image. This lets you exclude them.
|
||||
cover_exclusion_regexp:/images/.*?ribbon.gif
|
||||
@@ -712,22 +719,19 @@ cliches_label:Character Cliches
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
[erosnsappho.sycophanthex.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## some sites include images that we don't ever want becoming the
|
||||
## cover image. This lets you exclude them.
|
||||
cover_exclusion_regexp:/images/.*?ribbon.gif
|
||||
|
||||
[fanfiction.mugglenet.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
@@ -740,10 +744,10 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[fanfic.potterheadsanonymous.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
@@ -756,16 +760,19 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[fanfiction.portkey.org]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extraships:Harry Potter/Hermione Granger
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[fanfiction.tenhawkpresents.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -810,18 +817,26 @@ dislikes_label:Dislikes
|
||||
## Clear FanFiction from defaults, site is original fiction.
|
||||
extratags:
|
||||
|
||||
extra_valid_entries:size,universe,codes
|
||||
#extra_titlepage_entries:size,universe,codes
|
||||
extra_valid_entries:size,universe,universeUrl,universeHTML,codes,notice
|
||||
#extra_titlepage_entries:size,universeHTML,codes,notice
|
||||
|
||||
size_label:Size
|
||||
universe_label:Universe
|
||||
universeUrl_label:Universe URL
|
||||
universeHTML_label:Universe
|
||||
codes_label:Codes
|
||||
notice_label:Notice
|
||||
|
||||
## Assume entryUrl, apply to "<a class='%slink' href='%s'>%s</a>" to
|
||||
## make entryHTML.
|
||||
make_linkhtml_entries:universe
|
||||
|
||||
## storiesonline.net stories can be in a series or a universe, but not
|
||||
## both. By default, universe will be populated in 'series' with
|
||||
## index=0
|
||||
universe_as_series: true
|
||||
|
||||
[grangerenchanted.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Hermione Granger
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -834,6 +849,10 @@ extracharacters:Hermione Granger
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Hermione Granger
|
||||
|
||||
## Extra metadata that this adapter knows about. See [dramione.org]
|
||||
## for examples of how to use them.
|
||||
extra_valid_entries:read,reviews
|
||||
@@ -870,18 +889,15 @@ extracharacters:Kirk,Spock
|
||||
extraships:Kirk/Spock
|
||||
|
||||
[lumos.sycophanthex.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[merlinfic.dtwins.co.uk]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Merlin
|
||||
extracategories:Harry Potter
|
||||
|
||||
[merlinfic.dtwins.co.uk]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -894,6 +910,9 @@ extracategories:Merlin
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Merlin
|
||||
|
||||
[national-library.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:West Wing
|
||||
@@ -905,24 +924,32 @@ extracategories:NCIS
|
||||
[netraptor.org]
|
||||
|
||||
[nfacommunity.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:NCIS
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:NCIS
|
||||
|
||||
[nha.magical-worlds.us]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Buffy: The Vampire Slayer
|
||||
extracharacters:Willow
|
||||
|
||||
[occlumency.sycophanthex.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape
|
||||
|
||||
[onedirectionfanfiction.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -930,10 +957,15 @@ extracharacters:Severus Snape
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
[onedirectionfanfiction.com]
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:One Direction
|
||||
|
||||
[pommedesang.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -946,40 +978,27 @@ extracategories:One Direction
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[pommedesang.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Anita Blake Vampire Hunter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
[ponyfictionarchive.net]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[ponyfictionarchive.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:My Little Pony: Friendship is Magic
|
||||
|
||||
[pretendercentre.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[pretendercentre.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:The Pretender
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[samdean.archive.nu]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Supernatural
|
||||
@@ -987,9 +1006,6 @@ extracharacters:Sam,Dean
|
||||
extraships:Sam/Dean
|
||||
|
||||
[scarhead.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1002,6 +1018,9 @@ extracategories:Harry Potter
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[sg1-heliopolis.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
@@ -1032,9 +1051,6 @@ extracharacters:Draco Malfoy,Harry Potter
|
||||
extraships:Harry Potter/Draco Malfoy
|
||||
|
||||
[thehookupzone.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Criminal Minds
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1047,6 +1063,9 @@ extracategories:Criminal Minds
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Criminal Minds
|
||||
|
||||
[themasque.net]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -1061,9 +1080,6 @@ extracategories:Criminal Minds
|
||||
#is_adult:true
|
||||
|
||||
[thequidditchpitch.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1076,19 +1092,19 @@ extracategories:Harry Potter
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[tokra.fandomnet.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: SG-1
|
||||
extracategories:Harry Potter
|
||||
|
||||
[tokra.fandomnet.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[trekiverse.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Star Trek
|
||||
extracategories:Stargate: SG-1
|
||||
|
||||
[trekiverse.org]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1101,26 +1117,24 @@ extracategories:Star Trek
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Star Trek
|
||||
|
||||
extra_valid_entries:awards
|
||||
awards_label:Awards
|
||||
|
||||
cover_exclusion_regexp:art/.*Awards.jpg
|
||||
|
||||
[www.adastrafanfic.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Star Trek
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[www.dracoandginny.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Draco Malfoy,Ginny Weasley
|
||||
extraships:Draco Malfoy/Ginny Weasley
|
||||
extracategories:Star Trek
|
||||
|
||||
[www.dracoandginny.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1133,14 +1147,16 @@ extraships:Draco Malfoy/Ginny Weasley
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Draco Malfoy,Ginny Weasley
|
||||
extraships:Draco Malfoy/Ginny Weasley
|
||||
|
||||
[www.thealphagate.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: SG-1
|
||||
|
||||
[www.checkmated.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1148,6 +1164,9 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.destinysgateway.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
@@ -1155,11 +1174,6 @@ extracategories:Harry Potter
|
||||
#is_adult:true
|
||||
|
||||
[www.dokuga.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:Sesshoumaru,Kagome
|
||||
extraships:Sesshoumaru/Kagome
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1167,6 +1181,11 @@ extraships:Sesshoumaru/Kagome
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:Sesshoumaru,Kagome
|
||||
extraships:Sesshoumaru/Kagome
|
||||
|
||||
[www.dotmoon.net]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -1222,14 +1241,14 @@ dateUpdated_format:%%Y-%%m-%%d %%H:%%M:%%S
|
||||
[www.ficbook.net]
|
||||
|
||||
[www.fictionalley.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## fictionally.org storyIds are not unique. Combine with authorId.
|
||||
output_filename: ${title}-${siteabbrev}_${authorId}_${storyId}${formatext}
|
||||
|
||||
@@ -1255,6 +1274,23 @@ extra_valid_entries:reviews,favs,follows
|
||||
#password:yourpassword
|
||||
|
||||
[www.fimfiction.net]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## fimfiction.net stories can be locked requiring individual
|
||||
## passwords. If fail_on_password is set, the downloader will fail
|
||||
## when a password is required rather than prompting every time.
|
||||
#fail_on_password: false
|
||||
|
||||
## fimfiction.net stories allow chapters to be added out of order. So
|
||||
## the newest chapter may not be the last one. FFDL update doesn't
|
||||
## like that. If do_update_hook is uncommented and set true, the
|
||||
## adapter will discard all existing chapters from the newest one on
|
||||
## when updating to enforce accurate chapters.
|
||||
#do_update_hook:false
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:My Little Pony: Friendship is Magic
|
||||
|
||||
@@ -1285,55 +1321,33 @@ keep_in_order_groupsUrl:true
|
||||
## make entryHTML.
|
||||
make_linkhtml_entries:prequel,sequels,groups
|
||||
|
||||
[www.harrypotterfanfiction.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## fimfiction.net stories can be locked requiring individual
|
||||
## passwords. If fail_on_password is set, the downloader will fail
|
||||
## when a password is required rather than prompting every time.
|
||||
#fail_on_password: false
|
||||
|
||||
## fimfiction.net stories allow chapters to be added out of order. So
|
||||
## the newest chapter may not be the last one. FFDL update doesn't
|
||||
## like that. If do_update_hook is uncommented and set true, the
|
||||
## adapter will discard all existing chapters from the newest one on
|
||||
## when updating to enforce accurate chapters.
|
||||
#do_update_hook:false
|
||||
|
||||
[www.harrypotterfanfiction.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[www.henneth-annun.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:The Hobbit
|
||||
|
||||
[www.hpfandom.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.hpfanficarchive.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.ik-eternal.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:InuYasha,Kagome
|
||||
extraships:InuYasha/Kagome
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1346,30 +1360,35 @@ extraships:InuYasha/Kagome
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:InuYasha,Kagome
|
||||
extraships:InuYasha/Kagome
|
||||
|
||||
[www.jlaunlimited.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:JLA
|
||||
|
||||
[www.libraryofmoria.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
[www.mediaminer.org]
|
||||
|
||||
[www.midnightwhispers.ca]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
## some sites include images that we don't ever want becoming the
|
||||
## cover image. This lets you exclude them.
|
||||
cover_exclusion_regexp:/stories/999/images/.*?_trophy.png
|
||||
@@ -1384,10 +1403,6 @@ extracategories:CSI
|
||||
extraships:Nick Stokes/Greg Sanders
|
||||
|
||||
[www.phoenixsong.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extraships:Harry Potter/Ginny Weasley
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1398,9 +1413,13 @@ extraships:Harry Potter/Ginny Weasley
|
||||
## phoenixsong.net, oddly, can have high rated chapters (login
|
||||
## required) in the middle of a lower rated story. Use this to force
|
||||
## FFDL to always login to phoenixsong.net so those stories download
|
||||
## correctly.
|
||||
## correctly. If you have a login, this is recommended.
|
||||
#force_login:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extraships:Harry Potter/Ginny Weasley
|
||||
|
||||
[www.potionsandsnitches.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
@@ -1423,17 +1442,15 @@ extracategories:Psych
|
||||
#is_adult:true
|
||||
|
||||
[www.qaf-fic.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
[www.restrictedsection.org]
|
||||
extracategories:Harry Potter
|
||||
extragenres:Erotica
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1441,20 +1458,21 @@ extragenres:Erotica
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extragenres:Erotica
|
||||
|
||||
[www.scarvesandcoffee.net]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Glee
|
||||
extracharacters:Kurt Hummel,Blaine Anderson
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[www.simplyundeniable.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1462,15 +1480,18 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
[www.sinful-desire.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Supernatural
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.sinful-desire.org]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Supernatural
|
||||
|
||||
[www.siye.co.uk]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
@@ -1478,25 +1499,26 @@ extracharacters:Harry Potter,Ginny Weasley
|
||||
extraships:Harry Potter/Ginny Weasley
|
||||
|
||||
[www.squidge.org/peja]
|
||||
# www.squidge.org/peja calls it Fandom <shrug>
|
||||
## www.squidge.org/peja calls it Fandom <shrug>
|
||||
category_label:Fandom
|
||||
# Remove numWords -- www.squidge.org/peja word counts are inaccurate
|
||||
|
||||
## Remove numWords -- www.squidge.org/peja word counts are inaccurate
|
||||
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,site,description
|
||||
|
||||
[www.squidge.org/peja:txt]
|
||||
## Add URLs since there aren't links.
|
||||
# Remove numWords -- www.squidge.org/peja word counts are inaccurate
|
||||
## Add URLs since there aren't links and remove numWords --
|
||||
## www.squidge.org/peja word counts are inaccurate
|
||||
titlepage_entries: series,seriesUrl,category,genre,language,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,site,storyUrl, authorUrl, description
|
||||
|
||||
[www.storiesofarda.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
[www.thepetulantpoetess.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -1539,9 +1561,6 @@ collect_series: false
|
||||
extracategories:Twilight
|
||||
|
||||
[www.twilighted.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1549,13 +1568,13 @@ extracategories:Twilight
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## twilighted.net (ab)uses series as personal reading lists.
|
||||
collect_series: false
|
||||
|
||||
[www.twiwrite.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1563,20 +1582,23 @@ extracategories:Twilight
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## twiwrite.net (ab)uses series as personal reading lists.
|
||||
collect_series: false
|
||||
|
||||
[www.walkingtheplank.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Harry Potter
|
||||
extraships:Severus Snape/Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Harry Potter
|
||||
extraships:Severus Snape/Harry Potter
|
||||
|
||||
[www.whofic.com]
|
||||
|
||||
[www.wizardtales.net]
|
||||
@@ -1598,14 +1620,14 @@ extracategories:X-Men Movie
|
||||
extracharacters:Wolverine,Rogue
|
||||
|
||||
[www.wraithbait.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: Atlantis
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: Atlantis
|
||||
|
||||
[overrides]
|
||||
## It may sometimes be useful to override all of the specific format,
|
||||
## site and site:format sections in your private configuration. For
|
||||
|
||||
Binary file not shown.
@@ -122,6 +122,7 @@ import adapter_scarheadnet
|
||||
import adapter_fictionpadcom
|
||||
import adapter_storiesonlinenet
|
||||
import adapter_trekiverseorg
|
||||
import adapter_literotica
|
||||
|
||||
## This bit of complexity allows adapters to be added by just adding
|
||||
## importing. It eliminates the long if/else clauses we used to need
|
||||
|
||||
@@ -77,7 +77,7 @@ class ArchiveOfOurOwnOrgAdapter(BaseSiteAdapter):
|
||||
# The site domain. Does have www here, if it uses it.
|
||||
return 'archiveofourown.org'
|
||||
|
||||
@classmethod
|
||||
@classmethod
|
||||
def getSiteExampleURLs(self):
|
||||
return "http://"+self.getSiteDomain()+"/works/123456 http://"+self.getSiteDomain()+"/collections/Some_Archive/works/123456 http://"+self.getSiteDomain()+"/works/123456/chapters/78901"
|
||||
|
||||
@@ -177,19 +177,28 @@ class ArchiveOfOurOwnOrgAdapter(BaseSiteAdapter):
|
||||
self.story.addToList('authorUrl','http://'+self.host+a['href'])
|
||||
self.story.addToList('author',a.text)
|
||||
|
||||
newestChapter = None
|
||||
self.newestChapterNum = None # save for comparing during update.
|
||||
# Scan all chapters to find the oldest and newest, on AO3 it's
|
||||
# possible for authors to insert new chapters out-of-order or
|
||||
# change the dates of earlier ones by editing them--That WILL
|
||||
# break epub update.
|
||||
# Find the chapters:
|
||||
chapters=soup.findAll('a', href=re.compile(r'/works/'+self.story.getMetadata('storyId')+"/chapters/\d+$"))
|
||||
self.story.setMetadata('numChapters',len(chapters))
|
||||
logger.debug("numChapters: (%s)"%self.story.getMetadata('numChapters'))
|
||||
for x in range(0,len(chapters)):
|
||||
# just in case there's tags, like <i> in chapter titles.
|
||||
chapter=chapters[x]
|
||||
if len(chapters)==1:
|
||||
self.chapterUrls.append((self.story.getMetadata('title'),'http://'+self.host+chapter['href']+addurl))
|
||||
else:
|
||||
if len(chapters)==1:
|
||||
self.chapterUrls.append((self.story.getMetadata('title'),'http://'+self.host+chapters[0]['href']+addurl))
|
||||
else:
|
||||
for index, chapter in enumerate(chapters):
|
||||
# strip just in case there's tags, like <i> in chapter titles.
|
||||
self.chapterUrls.append((stripHTML(chapter),'http://'+self.host+chapter['href']+addurl))
|
||||
|
||||
|
||||
# (2013-09-21)
|
||||
date = stripHTML(chapter.findNext('span'))[1:-1]
|
||||
chapterDate = makeDate(date,self.dateformat)
|
||||
if newestChapter == None or chapterDate > newestChapter:
|
||||
newestChapter = chapterDate
|
||||
self.newestChapterNum = index
|
||||
|
||||
a = metasoup.find('blockquote',{'class':'userstuff'})
|
||||
if a != None:
|
||||
@@ -299,6 +308,12 @@ class ArchiveOfOurOwnOrgAdapter(BaseSiteAdapter):
|
||||
self.setSeries(series_name, series_index)
|
||||
self.story.setMetadata('seriesUrl',series_url)
|
||||
|
||||
def hookForUpdates(self,chaptercount):
|
||||
if self.oldchapters and len(self.oldchapters) > self.newestChapterNum:
|
||||
print("Existing epub has %s chapters\nNewest chapter is %s. Discarding old chapters from there on."%(len(self.oldchapters), self.newestChapterNum+1))
|
||||
self.oldchapters = self.oldchapters[:self.newestChapterNum]
|
||||
return len(self.oldchapters)
|
||||
|
||||
# grab the text for an individual chapter.
|
||||
def getChapterText(self, url):
|
||||
logger.debug('Getting chapter text from: %s' % url)
|
||||
|
||||
@@ -72,6 +72,13 @@ class FanFictionNetSiteAdapter(BaseSiteAdapter):
|
||||
def getSiteURLPattern(self):
|
||||
return r"https?://(www|m)?\.fanfiction\.net/s/\d+(/\d+)?(/|/[^/]+)?/?$"
|
||||
|
||||
def _fetchUrl(self,url):
|
||||
time.sleep(1.0) ## ffnet(and, I assume, fpcom) tends to fail
|
||||
## more if hit too fast. This is in
|
||||
## additional to what ever the
|
||||
## slow_down_sleep_time setting is.
|
||||
return BaseSiteAdapter._fetchUrl(self,url)
|
||||
|
||||
def extractChapterUrlsAndMetadata(self):
|
||||
|
||||
# fetch the chapter. From that we will get almost all the
|
||||
@@ -157,7 +164,7 @@ class FanFictionNetSiteAdapter(BaseSiteAdapter):
|
||||
|
||||
|
||||
|
||||
a = soup.find('a', href='http://www.fictionratings.com/')
|
||||
a = soup.find('a', href=re.compile(r'https?://www\.fictionratings\.com/'))
|
||||
rating = a.string
|
||||
if 'Fiction' in rating: # if rating has 'Fiction ', strip that out for consistency with past.
|
||||
rating = rating[8:]
|
||||
@@ -275,7 +282,7 @@ class FanFictionNetSiteAdapter(BaseSiteAdapter):
|
||||
return
|
||||
|
||||
def getChapterText(self, url):
|
||||
time.sleep(5.0) ## ffnet(and, I assume, fpcom) tends to fail
|
||||
time.sleep(4.0) ## ffnet(and, I assume, fpcom) tends to fail
|
||||
## more if hit too fast. This is in
|
||||
## additional to what ever the
|
||||
## slow_down_sleep_time setting is.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2012 Fanficdownloader team
|
||||
# Copyright 2013 Fanficdownloader team
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
@@ -37,11 +37,6 @@ class FineStoriesComAdapter(BaseSiteAdapter):
|
||||
def __init__(self, config, url):
|
||||
BaseSiteAdapter.__init__(self, config, url)
|
||||
|
||||
self.decode = ["Windows-1252",
|
||||
"utf8"] # 1252 is a superset of iso-8859-1.
|
||||
# Most sites that claim to be
|
||||
# iso-8859-1 (and some that claim to be
|
||||
# utf8) are really windows-1252.
|
||||
self.username = "NoneGiven" # if left empty, site doesn't return any message at all.
|
||||
self.password = ""
|
||||
self.is_adult=False
|
||||
@@ -51,7 +46,6 @@ class FineStoriesComAdapter(BaseSiteAdapter):
|
||||
if 'storyInfo' in self.story.getMetadata('storyId'):
|
||||
self.story.setMetadata('storyId',self.parsedUrl.query.split('=',)[1])
|
||||
|
||||
|
||||
# normalized story URL.
|
||||
self._setURL('http://' + self.getSiteDomain() + '/s/storyInfo.php?id='+self.story.getMetadata('storyId'))
|
||||
|
||||
|
||||
@@ -128,23 +128,22 @@ class HPFanficArchiveComAdapter(BaseSiteAdapter):
|
||||
# <span class="label">Rated:</span> NC-17<br /> etc
|
||||
labels = soup.findAll('span',{'class':'label'})
|
||||
for labelspan in labels:
|
||||
value = labelspan.nextSibling
|
||||
val = labelspan.nextSibling
|
||||
value = unicode('')
|
||||
while val and not defaultGetattr(val,'class') == 'label':
|
||||
value += unicode(val)
|
||||
val = val.nextSibling
|
||||
label = labelspan.string
|
||||
#print("label:%s\nvalue:%s"%(label,value))
|
||||
|
||||
if 'Summary' in label:
|
||||
## Everything until the next span class='label'
|
||||
svalue = ""
|
||||
while not defaultGetattr(value,'class') == 'label':
|
||||
svalue += str(value)
|
||||
value = value.nextSibling
|
||||
self.setDescription(url,svalue)
|
||||
#self.story.setMetadata('description',stripHTML(svalue))
|
||||
self.setDescription(url,value)
|
||||
|
||||
if 'Rated' in label:
|
||||
self.story.setMetadata('rating', value)
|
||||
self.story.setMetadata('rating', stripHTML(value))
|
||||
|
||||
if 'Word count' in label:
|
||||
self.story.setMetadata('numWords', value)
|
||||
self.story.setMetadata('numWords', stripHTML(value))
|
||||
|
||||
if 'Categories' in label:
|
||||
cats = labelspan.parent.findAll('a',href=re.compile(r'browse.php\?type=categories'))
|
||||
@@ -167,7 +166,7 @@ class HPFanficArchiveComAdapter(BaseSiteAdapter):
|
||||
self.story.addToList('warnings',warning.string)
|
||||
|
||||
if 'Completed' in label:
|
||||
if 'Yes' in value:
|
||||
if 'Yes' in stripHTML(value):
|
||||
self.story.setMetadata('status', 'Completed')
|
||||
else:
|
||||
self.story.setMetadata('status', 'In-Progress')
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2013 Fanficdownloader team
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
import time
|
||||
import logging
|
||||
logger = logging.getLogger(__name__)
|
||||
import re
|
||||
import urllib2
|
||||
import urlparse
|
||||
import time
|
||||
|
||||
from .. import BeautifulSoup as bs
|
||||
from ..htmlcleanup import stripHTML
|
||||
from .. import exceptions as exceptions
|
||||
|
||||
from base_adapter import BaseSiteAdapter, makeDate
|
||||
|
||||
class LiteroticaSiteAdapter(BaseSiteAdapter):
|
||||
|
||||
def __init__(self, config, url):
|
||||
BaseSiteAdapter.__init__(self, config, url)
|
||||
|
||||
self.decode = ["utf8",
|
||||
"Windows-1252"] # 1252 is a superset of iso-8859-1.
|
||||
# Most sites that claim to be
|
||||
# iso-8859-1 (and some that claim to be
|
||||
# utf8) are really windows-1252.
|
||||
|
||||
self.story.setMetadata('siteabbrev','litero')
|
||||
|
||||
# get storyId from url--url validation guarantees query is only sid=1234
|
||||
self.story.setMetadata('storyId',self.parsedUrl.path.split('/',)[2])
|
||||
|
||||
self.origurl = url
|
||||
if "http://www.i." in self.origurl:
|
||||
## accept m(mobile)url, but use www.
|
||||
self.origurl = self.origurl.replace("http://www.i.","http://www.")
|
||||
|
||||
# normalized story URL.
|
||||
self._setURL("http://"+self.getSiteDomain()\
|
||||
+"/s/"+self.story.getMetadata('storyId'))
|
||||
|
||||
# The date format will vary from site to site.
|
||||
# http://docs.python.org/library/datetime.html#strftime-strptime-behavior
|
||||
self.dateformat = '%m/%d/%y'
|
||||
|
||||
@staticmethod
|
||||
def getSiteDomain():
|
||||
return 'www.literotica.com'
|
||||
|
||||
@classmethod
|
||||
def getAcceptDomains(cls):
|
||||
return ['www.literotica.com', 'www.i.literotica.com']
|
||||
|
||||
@classmethod
|
||||
def getSiteExampleURLs(self):
|
||||
#return "http://www.literotica.com/s/story-title http://www.literotica.com/stories/showstory.php?id=1234 http://www.i.literotica.com/stories/showstory.php?id=1234"
|
||||
return "http://www.literotica.com/s/story-title"
|
||||
|
||||
def getSiteURLPattern(self):
|
||||
return r"http://www(\.i)?\.literotica\.com/s/([a-zA-Z0-9_-]+)"
|
||||
|
||||
def extractChapterUrlsAndMetadata(self):
|
||||
|
||||
if not (self.is_adult or self.getConfig("is_adult")):
|
||||
raise exceptions.AdultCheckRequired(self.url)
|
||||
|
||||
url1 = self.origurl
|
||||
logger.debug("first page URL: "+url1)
|
||||
|
||||
try:
|
||||
data1 = self._fetchUrl(url1)
|
||||
soup1 = bs.BeautifulSoup(data1)
|
||||
except urllib2.HTTPError, e:
|
||||
if e.code == 404:
|
||||
raise exceptions.StoryDoesNotExist(url1)
|
||||
else:
|
||||
raise e
|
||||
|
||||
#strip comments from soup
|
||||
[comment.extract() for comment in soup1.findAll(text=lambda text:isinstance(text, bs.Comment))]
|
||||
|
||||
# author
|
||||
a = soup1.find("span", "b-story-user-y")
|
||||
self.story.setMetadata('authorId', urlparse.parse_qs(a.a['href'].split('?')[1])['uid'])
|
||||
self.story.setMetadata('authorUrl', a.a['href'])
|
||||
self.story.setMetadata('author', a.text)
|
||||
|
||||
# get the author page
|
||||
try:
|
||||
dataAuth = self._fetchUrl(a.a['href'])
|
||||
soupAuth = bs.BeautifulSoup(dataAuth)
|
||||
except urllib2.HTTPError, e:
|
||||
if e.code == 404:
|
||||
raise exceptions.StoryDoesNotExist(a.a['href'])
|
||||
else:
|
||||
raise e
|
||||
|
||||
storyLink = soupAuth.find('a', href=url1)
|
||||
|
||||
if storyLink is not None:
|
||||
# pull the published date from the author page
|
||||
# default values from single link. Updated below if multiple chapter.
|
||||
date = storyLink.parent.parent.findAll('td')[-1].text
|
||||
self.story.setMetadata('datePublished', makeDate(date, self.dateformat))
|
||||
self.story.setMetadata('dateUpdated',makeDate(date, self.dateformat))
|
||||
|
||||
# find num of pages
|
||||
# find a "3 Pages:" string on the page and parse it
|
||||
pgs = soup1.find("span", "b-pager-caption-t r-d45").string.split(' ')[0]
|
||||
|
||||
# If there are multiple pages, find and request the last page
|
||||
if "1" != pgs:
|
||||
logger.debug("last page number: "+pgs)
|
||||
try:
|
||||
data2 = self._fetchUrl(url1, {'page': pgs})
|
||||
soup2 = bs.BeautifulSoup(data2)
|
||||
[comment.extract() for comment in soup2.findAll(text=lambda text:isinstance(text, bs.Comment))]
|
||||
except urllib2.HTTPError, e:
|
||||
if e.code == 404:
|
||||
# TODO: Probably should reformat this
|
||||
raise exceptions.StoryDoesNotExist(url1, {'page': pgs})
|
||||
else:
|
||||
raise e
|
||||
else:
|
||||
#If we're already on the last page, copy the soup
|
||||
soup2 = soup1
|
||||
|
||||
# parse out the list of chapters
|
||||
chaps = soup2.find('div', id='b-series')
|
||||
if chaps: # may be one post only
|
||||
#self.chapterUrls = [(ch.a.text, ch.a['href']) for ch in chaps.findAll('li')]
|
||||
|
||||
# if there are chapters, lets pull them and title from the
|
||||
# author page because *this* chapter is omitted from the
|
||||
# list on the last page.
|
||||
row = storyLink.parent.parent.previousSibling
|
||||
while row['class'] != 'ser-ttl':
|
||||
row = row.previousSibling
|
||||
|
||||
seriesTitle = stripHTML(row)
|
||||
if seriesTitle:
|
||||
# this regex is deliberately greedy. We want to get the biggest match before a ':'
|
||||
self.story.setMetadata('title', re.match('(.*):[^:]*$', seriesTitle).group(1))
|
||||
else:
|
||||
self.story.setMetadata('title', soup1.h1.string)
|
||||
|
||||
# now chapter list. Assumed oldest to newest.
|
||||
self.chapterUrls = []
|
||||
row = row.nextSibling
|
||||
|
||||
self.story.setMetadata('datePublished',makeDate(stripHTML(row.find('td',{'class':'dt'})), self.dateformat))
|
||||
while row['class'] == 'sl':
|
||||
# pages include full URLs.
|
||||
self.chapterUrls.append((row.a.string,row.a['href']))
|
||||
if not row.nextSibling:
|
||||
break
|
||||
row = row.nextSibling
|
||||
|
||||
row = row.previousSibling
|
||||
self.story.setMetadata('dateUpdated',makeDate(stripHTML(row.find('td',{'class':'dt'})), self.dateformat))
|
||||
|
||||
else: # if one post only
|
||||
self.chapterUrls = [(soup1.h1.string, url1)]
|
||||
self.story.setMetadata('title', soup1.h1.string)
|
||||
|
||||
# normalize on first chapter URL.
|
||||
self._setURL(self.chapterUrls[0][1])
|
||||
|
||||
# reset storyId to first chapter.
|
||||
self.story.setMetadata('storyId',self.parsedUrl.path.split('/',)[2])
|
||||
|
||||
self.story.setMetadata('numChapters', len(self.chapterUrls))
|
||||
|
||||
self.story.setMetadata('category', soup1.find('div', 'b-breadcrumbs').findAll('a')[1].string)
|
||||
self.story.setMetadata('description', soup1.find('meta', {'name': 'description'})['content'])
|
||||
|
||||
return
|
||||
|
||||
def getChapterText(self, url):
|
||||
logger.debug('Getting chapter text from: %s' % url)
|
||||
time.sleep(0.5)
|
||||
data1 = self._fetchUrl(url)
|
||||
soup1 = bs.BeautifulSoup(data1)
|
||||
|
||||
#strip comments from soup
|
||||
[comment.extract() for comment in soup1.findAll(text=lambda text:isinstance(text, bs.Comment))]
|
||||
|
||||
# get story text
|
||||
story1 = soup1.find('div', 'b-story-body-x').p
|
||||
story1.name='div'
|
||||
story1.append('<br>')
|
||||
storytext = self.utf8FromSoup(url,story1)
|
||||
|
||||
# find num pages
|
||||
pgs = int(soup1.find("span", "b-pager-caption-t r-d45").string.split(' ')[0])
|
||||
logger.debug("pages: "+str(pgs))
|
||||
|
||||
# get all the pages
|
||||
for i in xrange(2, pgs+1):
|
||||
try:
|
||||
logger.debug("fetching page "+str(i))
|
||||
time.sleep(0.5)
|
||||
data2 = self._fetchUrl(url, {'page': i})
|
||||
soup2 = bs.BeautifulSoup(data2)
|
||||
[comment.extract() for comment in soup2.findAll(text=lambda text:isinstance(text, bs.Comment))]
|
||||
story2 = soup2.find('div', 'b-story-body-x').p
|
||||
story2.name='div'
|
||||
story2.append('<br>')
|
||||
storytext += self.utf8FromSoup(url,story2)
|
||||
except urllib2.HTTPError, e:
|
||||
if e.code == 404:
|
||||
raise exceptions.StoryDoesNotExist(url)
|
||||
else:
|
||||
raise e
|
||||
return storytext
|
||||
|
||||
|
||||
def getClass():
|
||||
return LiteroticaSiteAdapter
|
||||
|
||||
|
||||
|
||||
@@ -37,11 +37,6 @@ class StoriesOnlineNetAdapter(BaseSiteAdapter):
|
||||
def __init__(self, config, url):
|
||||
BaseSiteAdapter.__init__(self, config, url)
|
||||
|
||||
self.decode = ["Windows-1252",
|
||||
"utf8"] # 1252 is a superset of iso-8859-1.
|
||||
# Most sites that claim to be
|
||||
# iso-8859-1 (and some that claim to be
|
||||
# utf8) are really windows-1252.
|
||||
self.username = "NoneGiven" # if left empty, site doesn't return any message at all.
|
||||
self.password = ""
|
||||
self.is_adult=False
|
||||
@@ -147,6 +142,10 @@ class StoriesOnlineNetAdapter(BaseSiteAdapter):
|
||||
a = soup.find('h1')
|
||||
self.story.setMetadata('title',stripHTML(a))
|
||||
|
||||
notice = soup.find('div', {'class' : 'notice'})
|
||||
if notice:
|
||||
self.story.setMetadata('notice',unicode(notice))
|
||||
|
||||
# Find authorid and URL from... author url.
|
||||
a = soup.find('a', href=re.compile(r"/a/\w+"))
|
||||
self.story.setMetadata('authorId',a['href'].split('/')[2])
|
||||
@@ -185,23 +184,74 @@ class StoriesOnlineNetAdapter(BaseSiteAdapter):
|
||||
self.story.setMetadata('size', lc2.findNext('td', {'class' : 'num'}).text)
|
||||
|
||||
lc4 = lc2.findNext('td', {'class' : 'lc4'})
|
||||
|
||||
desc = lc4.contents[0]
|
||||
|
||||
try:
|
||||
a = lc4.find('a', href=re.compile(r"/library/show_series.php\?id=\d+"))
|
||||
i = a.parent.text.split('(')[1].split(')')[0]
|
||||
self.setSeries(stripHTML(a), i)
|
||||
self.story.setMetadata('seriesUrl','http://'+self.host+a['href'])
|
||||
if a:
|
||||
# if there's a number after the series name, series_contents is a two element list:
|
||||
# [<a href="...">Title</a>, u' (2)']
|
||||
series_contents = a.parent.contents
|
||||
i = 0 if len(series_contents) == 1 else series_contents[1].strip(' ()')
|
||||
seriesUrl = 'http://'+self.host+a['href']
|
||||
self.story.setMetadata('seriesUrl',seriesUrl)
|
||||
series_name = stripHTML(a)
|
||||
logger.debug("Series name= %s" % series_name)
|
||||
series_soup = bs.BeautifulSoup(self._fetchUrl(seriesUrl))
|
||||
if series_soup:
|
||||
logger.debug("Retrieving Series - looking for name")
|
||||
series_name = series_soup.find('span', {'id' : 'ptitle'}).text.partition(' — ')[0]
|
||||
logger.debug("Series name: '{0}'".format(series_name))
|
||||
self.setSeries(series_name, i)
|
||||
desc = lc4.contents[2]
|
||||
# Check if series is in a universe
|
||||
universes_soup = bs.BeautifulSoup(self._fetchUrl(self.story.getMetadata('authorUrl') + "&type=uni"))
|
||||
# logger.debug("Universe page=", universes_soup)
|
||||
if universes_soup:
|
||||
universes = universes_soup.findAll('div', {'class' : 'ser-box'})
|
||||
logger.debug("Number of Universes: %d" % len(universes))
|
||||
for universe in universes:
|
||||
logger.debug("universe.find('a')={0}".format(universe.find('a')))
|
||||
# The universe id is in an "a" tag that has an id but nothing else. It is the first tag.
|
||||
# The id is prefixed with the letter "u".
|
||||
universe_id = universe.find('a')['id'][1:]
|
||||
logger.debug("universe_id='%s'" % universe_id)
|
||||
universe_name = universe.find('div', {'class' : 'ser-name'}).text.partition(' ')[2]
|
||||
logger.debug("universe_name='%s'" % universe_name)
|
||||
# If there is link to the story, we have the right universe
|
||||
story_a = universe.find('a', {'href' : '/s/'+self.story.getMetadata('storyId')})
|
||||
if story_a:
|
||||
logger.debug("Story is in a series that is in a universe! The universe is '%s'" % universe_name)
|
||||
self.story.setMetadata("universe", universe_name)
|
||||
self.story.setMetadata('universeUrl','http://'+self.host+ '/library/universe.php?id=' + universe_id)
|
||||
break
|
||||
except:
|
||||
pass
|
||||
try:
|
||||
a = lc4.find('a', href=re.compile(r"/library/universe.php\?id=\d+"))
|
||||
if a:
|
||||
self.story.setMetadata("universe",stripHTML(a))
|
||||
desc = lc4.contents[2]
|
||||
# Assumed only one universe, but it does have a URL--use universeHTML
|
||||
universe_name = stripHTML(a)
|
||||
universeUrl = 'http://'+self.host+a['href']
|
||||
logger.debug("Retrieving Universe - about to get page")
|
||||
universe_soup = bs.BeautifulSoup(self._fetchUrl(universeUrl))
|
||||
logger.debug("Retrieving Universe - have page")
|
||||
if universe_soup:
|
||||
logger.debug("Retrieving Universe - looking for name")
|
||||
universe_name = universe_soup.find('span', {'id' : 'ptitle'}).text.partition(' —')[0]
|
||||
logger.debug("Universes name: '{0}'".format(universe_name))
|
||||
|
||||
self.story.setMetadata('universeUrl',universeUrl)
|
||||
logger.debug("Setting universe name: '{0}'".format(universe_name))
|
||||
self.story.setMetadata('universe',universe_name)
|
||||
if self.getConfig("universe_as_series"):
|
||||
self.setSeries(universe_name, 0)
|
||||
self.story.setMetadata('seriesUrl',universeUrl)
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
desc = lc4.contents[0]
|
||||
self.setDescription('http://'+self.host+'/s/'+self.story.getMetadata('storyId'),desc)
|
||||
|
||||
for b in lc4.findAll('b'):
|
||||
|
||||
@@ -65,12 +65,16 @@ class TrekiverseOrgAdapter(BaseSiteAdapter):
|
||||
# The site domain.
|
||||
return 'trekiverse.org'
|
||||
|
||||
@classmethod
|
||||
def getAcceptDomains(cls):
|
||||
return ['trekiverse.org','efiction.trekiverse.org']
|
||||
|
||||
@classmethod
|
||||
def getSiteExampleURLs(self):
|
||||
return "http://"+self.getSiteDomain()+"/efiction/viewstory.php?sid=1234"
|
||||
return "http://"+self.getSiteDomain()+"/efiction/viewstory.php?sid=1234 http://efiction."+self.getSiteDomain()+"/viewstory.php?sid=1234"
|
||||
|
||||
def getSiteURLPattern(self):
|
||||
return r'http://trekiverse\.org/efiction/viewstory\.php\?sid=\d+'
|
||||
return r'(http://trekiverse\.org/efiction/viewstory\.php\?sid=\d+|http://efiction\.trekiverse\.org/viewstory\.php\?sid=\d+)'
|
||||
|
||||
## Login seems to be reasonably standard across eFiction sites.
|
||||
def needToLoginCheck(self, data):
|
||||
|
||||
@@ -141,28 +141,28 @@ def replace_br_with_p(body):
|
||||
averageLineLength = contentLinesSum/contentLines
|
||||
|
||||
logger.debug(u'---')
|
||||
logger.debug(u'Lines.............: ' + str(len(lines)))
|
||||
logger.debug(u'contentLines......: ' + str(contentLines))
|
||||
logger.debug(u'contentLinesSum...: ' + str(contentLinesSum))
|
||||
logger.debug(u'longestLineLength.: ' + str(longestLineLength))
|
||||
logger.debug(u'averageLineLength.: ' + str(averageLineLength))
|
||||
logger.debug(u'Lines.............: ' + unicode(len(lines)))
|
||||
logger.debug(u'contentLines......: ' + unicode(contentLines))
|
||||
logger.debug(u'contentLinesSum...: ' + unicode(contentLinesSum))
|
||||
logger.debug(u'longestLineLength.: ' + unicode(longestLineLength))
|
||||
logger.debug(u'averageLineLength.: ' + unicode(averageLineLength))
|
||||
|
||||
if breaksMaxIndex == len(breaksCount)-1 and breaksMax < 2:
|
||||
breaksMaxIndex = 0
|
||||
breaksMax = breaksCount[0]
|
||||
|
||||
logger.debug(u'---')
|
||||
logger.debug(u'breaks 1: ' + str(breaksCount[0]))
|
||||
logger.debug(u'breaks 2: ' + str(breaksCount[1]))
|
||||
logger.debug(u'breaks 3: ' + str(breaksCount[2]))
|
||||
logger.debug(u'breaks 4: ' + str(breaksCount[3]))
|
||||
logger.debug(u'breaks 5: ' + str(breaksCount[4]))
|
||||
logger.debug(u'breaks 6: ' + str(breaksCount[5]))
|
||||
logger.debug(u'breaks 7: ' + str(breaksCount[6]))
|
||||
logger.debug(u'breaks 8: ' + str(breaksCount[7]))
|
||||
logger.debug(u'breaks 1: ' + unicode(breaksCount[0]))
|
||||
logger.debug(u'breaks 2: ' + unicode(breaksCount[1]))
|
||||
logger.debug(u'breaks 3: ' + unicode(breaksCount[2]))
|
||||
logger.debug(u'breaks 4: ' + unicode(breaksCount[3]))
|
||||
logger.debug(u'breaks 5: ' + unicode(breaksCount[4]))
|
||||
logger.debug(u'breaks 6: ' + unicode(breaksCount[5]))
|
||||
logger.debug(u'breaks 7: ' + unicode(breaksCount[6]))
|
||||
logger.debug(u'breaks 8: ' + unicode(breaksCount[7]))
|
||||
logger.debug(u'----')
|
||||
logger.debug(u'max found: ' + str(breaksMax))
|
||||
logger.debug(u'max Index: ' + str(breaksMaxIndex))
|
||||
logger.debug(u'max found: ' + unicode(breaksMax))
|
||||
logger.debug(u'max Index: ' + unicode(breaksMaxIndex))
|
||||
logger.debug(u'----')
|
||||
|
||||
if breaksMaxIndex > 0 and breaksCount[0] > breaksMax and averageLineLength < 90:
|
||||
@@ -173,13 +173,13 @@ def replace_br_with_p(body):
|
||||
for i in range(len(breaksCount)):
|
||||
# if i > 0 or breaksMaxIndex == 0:
|
||||
if i <= breaksMaxIndex:
|
||||
logger.debug(str(i) + u' <= breaksMaxIndex (' + str(breaksMaxIndex) + u')')
|
||||
logger.debug(unicode(i) + u' <= breaksMaxIndex (' + unicode(breaksMaxIndex) + u')')
|
||||
body = breaksRegexp[i].sub(r'\1</p>\n<p>\3', body)
|
||||
elif i == breaksMaxIndex+1:
|
||||
logger.debug(str(i) + u' == breaksMaxIndex+1 (' + str(breaksMaxIndex+1) + u')')
|
||||
logger.debug(unicode(i) + u' == breaksMaxIndex+1 (' + unicode(breaksMaxIndex+1) + u')')
|
||||
body = breaksRegexp[i].sub(r'\1</p>\n<p><br/></p>\n<p>\3', body)
|
||||
else:
|
||||
logger.debug(str(i) + u' > breaksMaxIndex+1 (' + str(breaksMaxIndex+1) + u')')
|
||||
logger.debug(unicode(i) + u' > breaksMaxIndex+1 (' + unicode(breaksMaxIndex+1) + u')')
|
||||
body = breaksRegexp[i].sub(r'\1</p>\n<hr />\n<p>\3', body)
|
||||
|
||||
body = breaksRegexp[8].sub(r'</p>\n<hr />\n<p>', body)
|
||||
@@ -230,7 +230,7 @@ def replace_br_with_p(body):
|
||||
return tag_sanitizer(body)
|
||||
|
||||
def is_valid_block(block):
|
||||
return str(block).find('<') == 0 and str(block).find('<!') != 0
|
||||
return unicode(block).find('<') == 0 and unicode(block).find('<!') != 0
|
||||
|
||||
def soup_up_div(body):
|
||||
blockTags = ['address', 'blockquote', 'del', 'div', 'dl', 'fieldset', 'form', 'ins', 'noscript', 'ol', 'p', 'pre', 'table', 'ul']
|
||||
@@ -247,8 +247,8 @@ def soup_up_div(body):
|
||||
lastElement = 1 # 1 = block, 2 = nested, 3 = invalid
|
||||
|
||||
for i in soup.contents[0]:
|
||||
if str(i).strip().__len__() > 0:
|
||||
s = str(i)
|
||||
if unicode(i).strip().__len__() > 0:
|
||||
s = unicode(i)
|
||||
if type(i) == bs.Tag:
|
||||
if i.name in blockTags:
|
||||
if lastElement > 1:
|
||||
@@ -308,7 +308,7 @@ def tag_sanitizer(html):
|
||||
is_closed = is_closed_tag(rTag[0]) or is_comment_tag(rTag[0])
|
||||
|
||||
# is_comment = is_comment_tag(rTag[0])
|
||||
# logger.debug(u'%s > isEnd: %s > isClosed: %s > isComment: %s'%(name, str(is_end), str(is_closed), str(is_comment)))
|
||||
# logger.debug(u'%s > isEnd: %s > isClosed: %s > isComment: %s'%(name, unicode(is_end), unicode(is_closed), unicode(is_comment)))
|
||||
# logger.debug(u'> %s%s\n'%(rTag[0], rTag[1]))
|
||||
|
||||
if name in blockTags:
|
||||
|
||||
@@ -389,9 +389,14 @@ class Story(Configurable):
|
||||
elif self.getMetadataRaw('series') != None:
|
||||
self.setMetadata('seriesHTML',self.getMetadataRaw('series'))
|
||||
|
||||
# logger.debug("make_linkhtml_entries:%s"%self.getConfig('make_linkhtml_entries'))
|
||||
for k in self.getConfigList('make_linkhtml_entries'):
|
||||
# Assuming list, because it has to be site specific and
|
||||
# they are all lists.
|
||||
# they are all lists. Bail if kUrl list not the same
|
||||
# length.
|
||||
# logger.debug("\nk:%s\nlist:%s\nlistURL:%s"%(k,self.getList(k),self.getList(k+'Url')))
|
||||
if len(self.getList(k+'Url')) != len(self.getList(k)):
|
||||
continue
|
||||
htmllist=[]
|
||||
for i, v in enumerate(self.getList(k)):
|
||||
url = self.getList(k+'Url')[i]
|
||||
@@ -403,7 +408,7 @@ class Story(Configurable):
|
||||
url=removeAllEntities(url)
|
||||
v=removeAllEntities(v)
|
||||
|
||||
htmllist.append(linkhtml%('author',url,v))
|
||||
htmllist.append(linkhtml%(k,url,v))
|
||||
join_string = self.getConfig("join_string_"+k+"HTML",u", ").replace('\s',' ')
|
||||
self.setMetadata(k+'HTML',join_string.join(htmllist))
|
||||
|
||||
|
||||
+7
-11
@@ -48,13 +48,9 @@
|
||||
|
||||
<h3>fanfiction.net / fimfiction.net</h3>
|
||||
<p>
|
||||
Fanfiction.net appears to be blocking access from Google
|
||||
App Engine, which prevents this web service. There's
|
||||
nothing I can do about it. At the time of writing, the
|
||||
latest CLI and calibre plugin versions worked.
|
||||
</p>
|
||||
<p>It appears that FimFiction.net is also blocking access from Google
|
||||
App Engine now.
|
||||
As of Jan 13, 2014, fanfiction.net & fimfiction.net
|
||||
are working again. I'd ask that users limit the number of
|
||||
stories they download from those sites, thanks.
|
||||
</p>
|
||||
|
||||
{% if authorized %}
|
||||
@@ -68,9 +64,9 @@
|
||||
<h3>Changes:</h3>
|
||||
<p>
|
||||
<ul>
|
||||
<li>New Site: trekiverse.org/efiction/</li>
|
||||
<li>Fix for higher rated stories with not logged in on tthfanfic.org.</li>
|
||||
<li>Add prequels/sequels & fix groups site specific metadata for fimfiction.net.</li>
|
||||
<li>Change default encoding for finestories.com.</li>
|
||||
<li>Change default encoding and improve metadata for storiesonline.net, thanks davidfor.</li>
|
||||
<li>Fixes for hpfanficarchive.com changes.</li>
|
||||
</ul>
|
||||
</p>
|
||||
|
||||
@@ -82,7 +78,7 @@
|
||||
If you have any problems with this application, please
|
||||
report them in
|
||||
the <a href="http://groups.google.com/group/fanfic-downloader">FanFictionDownLoader Google Group</a>. The
|
||||
<a href="http://4-4-85.fanfictiondownloader.appspot.com">Previous Version</a> is also available for you to use if necessary.
|
||||
<a href="http://4-4-89.fanfictiondownloader.appspot.com">Previous Version</a> is also available for you to use if necessary.
|
||||
</p>
|
||||
<div id='error'>
|
||||
{{ error_message }}
|
||||
|
||||
+197
-175
@@ -507,6 +507,13 @@ extratags: FanFiction,Testing,HTML
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## archiveofourown.org stories allow chapters to be added out of
|
||||
## order. So the newest chapter may not be the last one. FFDL update
|
||||
## doesn't like that. If do_update_hook is uncommented and set true,
|
||||
## the adapter will discard all existing chapters from the newest one
|
||||
## on when updating to enforce accurate chapters.
|
||||
#do_update_hook:false
|
||||
|
||||
## AO3 adapter defines a few extra metadata entries.
|
||||
## If there's ever more than 4 series, add series04,series04Url etc.
|
||||
extra_valid_entries:fandoms,freeformtags,freefromtags,ao3categories,comments,kudos,hits,bookmarks,collections,series00,series01,series02,series03,series00Url,series01Url,series02Url,series03Url
|
||||
@@ -531,11 +538,6 @@ include_in_freefromtags:freeformtags
|
||||
#extra_subject_tags:fandoms,freeformtags,ao3categories
|
||||
|
||||
[ashwinder.sycophanthex.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Hermione Granger
|
||||
extraships:Severus Snape/Hermione Granger
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -543,6 +545,11 @@ extraships:Severus Snape/Hermione Granger
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Hermione Granger
|
||||
extraships:Severus Snape/Hermione Granger
|
||||
|
||||
[asr3.slashzone.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:The Sentinel
|
||||
@@ -553,9 +560,6 @@ extracategories:The Sentinel
|
||||
#is_adult:true
|
||||
|
||||
[bloodties-fans.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Blood Ties
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -568,6 +572,9 @@ extracategories:Blood Ties
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Blood Ties
|
||||
|
||||
[buffynfaith.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Buffy: The Vampire Slayer
|
||||
@@ -578,9 +585,6 @@ extracategories:Buffy: The Vampire Slayer
|
||||
#is_adult:true
|
||||
|
||||
[castlefans.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Castle
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -593,6 +597,9 @@ extracategories:Castle
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Castle
|
||||
|
||||
[chaos.sycophanthex.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
@@ -605,10 +612,23 @@ cover_exclusion_regexp:/images/.*?ribbon.gif
|
||||
|
||||
[dark-solace.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
extracategories:Buffy: The Vampire Slayer
|
||||
extracharacters:Buffy, Spike
|
||||
extraships:Spike/Buffy
|
||||
|
||||
[dramione.org]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -621,24 +641,11 @@ extraships:Spike/Buffy
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[dramione.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Draco Malfoy,Hermione Granger
|
||||
extraships:Draco Malfoy/Hermione Granger
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## some sites include images that we don't ever want becoming the
|
||||
## cover image. This lets you exclude them.
|
||||
cover_exclusion_regexp:/images/.*?ribbon.gif
|
||||
@@ -697,22 +704,19 @@ cliches_label:Character Cliches
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
[erosnsappho.sycophanthex.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## some sites include images that we don't ever want becoming the
|
||||
## cover image. This lets you exclude them.
|
||||
cover_exclusion_regexp:/images/.*?ribbon.gif
|
||||
|
||||
[fanfiction.mugglenet.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
@@ -725,10 +729,10 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[fanfic.potterheadsanonymous.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
@@ -741,16 +745,19 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[fanfiction.portkey.org]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extraships:Harry Potter/Hermione Granger
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[fanfiction.tenhawkpresents.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -795,18 +802,26 @@ dislikes_label:Dislikes
|
||||
## Clear FanFiction from defaults, site is original fiction.
|
||||
extratags:
|
||||
|
||||
extra_valid_entries:size,universe,codes
|
||||
#extra_titlepage_entries:size,universe,codes
|
||||
extra_valid_entries:size,universe,universeUrl,universeHTML,codes,notice
|
||||
#extra_titlepage_entries:size,universeHTML,codes,notice
|
||||
|
||||
size_label:Size
|
||||
universe_label:Universe
|
||||
universeUrl_label:Universe URL
|
||||
universeHTML_label:Universe
|
||||
codes_label:Codes
|
||||
notice_label:Notice
|
||||
|
||||
## Assume entryUrl, apply to "<a class='%slink' href='%s'>%s</a>" to
|
||||
## make entryHTML.
|
||||
make_linkhtml_entries:universe
|
||||
|
||||
## storiesonline.net stories can be in a series or a universe, but not
|
||||
## both. By default, universe will be populated in 'series' with
|
||||
## index=0
|
||||
universe_as_series: true
|
||||
|
||||
[grangerenchanted.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Hermione Granger
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -819,6 +834,10 @@ extracharacters:Hermione Granger
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Hermione Granger
|
||||
|
||||
## Extra metadata that this adapter knows about. See [dramione.org]
|
||||
## for examples of how to use them.
|
||||
extra_valid_entries:read,reviews
|
||||
@@ -855,18 +874,15 @@ extracharacters:Kirk,Spock
|
||||
extraships:Kirk/Spock
|
||||
|
||||
[lumos.sycophanthex.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[merlinfic.dtwins.co.uk]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Merlin
|
||||
extracategories:Harry Potter
|
||||
|
||||
[merlinfic.dtwins.co.uk]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -879,6 +895,9 @@ extracategories:Merlin
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Merlin
|
||||
|
||||
[national-library.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:West Wing
|
||||
@@ -890,24 +909,32 @@ extracategories:NCIS
|
||||
[netraptor.org]
|
||||
|
||||
[nfacommunity.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:NCIS
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:NCIS
|
||||
|
||||
[nha.magical-worlds.us]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Buffy: The Vampire Slayer
|
||||
extracharacters:Willow
|
||||
|
||||
[occlumency.sycophanthex.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape
|
||||
|
||||
[onedirectionfanfiction.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -915,10 +942,15 @@ extracharacters:Severus Snape
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
[onedirectionfanfiction.com]
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:One Direction
|
||||
|
||||
[pommedesang.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -931,40 +963,27 @@ extracategories:One Direction
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[pommedesang.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Anita Blake Vampire Hunter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
## defaults.ini.
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Some sites also require the user to confirm they are adult for
|
||||
## adult content. In commandline version, this should go in your
|
||||
## personal.ini, not defaults.ini.
|
||||
[ponyfictionarchive.net]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[ponyfictionarchive.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:My Little Pony: Friendship is Magic
|
||||
|
||||
[pretendercentre.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[pretendercentre.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:The Pretender
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[samdean.archive.nu]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Supernatural
|
||||
@@ -972,9 +991,6 @@ extracharacters:Sam,Dean
|
||||
extraships:Sam/Dean
|
||||
|
||||
[scarhead.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -987,6 +1003,9 @@ extracategories:Harry Potter
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[sg1-heliopolis.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
@@ -1017,9 +1036,6 @@ extracharacters:Draco Malfoy,Harry Potter
|
||||
extraships:Harry Potter/Draco Malfoy
|
||||
|
||||
[thehookupzone.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Criminal Minds
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1032,6 +1048,9 @@ extracategories:Criminal Minds
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Criminal Minds
|
||||
|
||||
[themasque.net]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -1046,9 +1065,6 @@ extracategories:Criminal Minds
|
||||
#is_adult:true
|
||||
|
||||
[thequidditchpitch.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1061,19 +1077,19 @@ extracategories:Harry Potter
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[tokra.fandomnet.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: SG-1
|
||||
extracategories:Harry Potter
|
||||
|
||||
[tokra.fandomnet.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[trekiverse.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Star Trek
|
||||
extracategories:Stargate: SG-1
|
||||
|
||||
[trekiverse.org]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1086,26 +1102,24 @@ extracategories:Star Trek
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Star Trek
|
||||
|
||||
extra_valid_entries:awards
|
||||
awards_label:Awards
|
||||
|
||||
cover_exclusion_regexp:art/.*Awards.jpg
|
||||
|
||||
[www.adastrafanfic.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Star Trek
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[www.dracoandginny.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Draco Malfoy,Ginny Weasley
|
||||
extraships:Draco Malfoy/Ginny Weasley
|
||||
extracategories:Star Trek
|
||||
|
||||
[www.dracoandginny.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1118,14 +1132,16 @@ extraships:Draco Malfoy/Ginny Weasley
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Draco Malfoy,Ginny Weasley
|
||||
extraships:Draco Malfoy/Ginny Weasley
|
||||
|
||||
[www.thealphagate.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: SG-1
|
||||
|
||||
[www.checkmated.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1133,6 +1149,9 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.destinysgateway.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
@@ -1140,11 +1159,6 @@ extracategories:Harry Potter
|
||||
#is_adult:true
|
||||
|
||||
[www.dokuga.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:Sesshoumaru,Kagome
|
||||
extraships:Sesshoumaru/Kagome
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1152,6 +1166,11 @@ extraships:Sesshoumaru/Kagome
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:Sesshoumaru,Kagome
|
||||
extraships:Sesshoumaru/Kagome
|
||||
|
||||
[www.dotmoon.net]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -1207,14 +1226,14 @@ dateUpdated_format:%%Y-%%m-%%d %%H:%%M:%%S
|
||||
[www.ficbook.net]
|
||||
|
||||
[www.fictionalley.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## fictionalley.org doesn't have a status metadatum. If uncommented,
|
||||
## this will be used for status.
|
||||
#default_value_status:Unknown
|
||||
@@ -1237,6 +1256,23 @@ extra_valid_entries:reviews,favs,follows
|
||||
#password:yourpassword
|
||||
|
||||
[www.fimfiction.net]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## fimfiction.net stories can be locked requiring individual
|
||||
## passwords. If fail_on_password is set, the downloader will fail
|
||||
## when a password is required rather than prompting every time.
|
||||
#fail_on_password: false
|
||||
|
||||
## fimfiction.net stories allow chapters to be added out of order. So
|
||||
## the newest chapter may not be the last one. FFDL update doesn't
|
||||
## like that. If do_update_hook is uncommented and set true, the
|
||||
## adapter will discard all existing chapters from the newest one on
|
||||
## when updating to enforce accurate chapters.
|
||||
#do_update_hook:false
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:My Little Pony: Friendship is Magic
|
||||
|
||||
@@ -1267,55 +1303,33 @@ keep_in_order_groupsUrl:true
|
||||
## make entryHTML.
|
||||
make_linkhtml_entries:prequel,sequels,groups
|
||||
|
||||
[www.harrypotterfanfiction.com]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## fimfiction.net stories can be locked requiring individual
|
||||
## passwords. If fail_on_password is set, the downloader will fail
|
||||
## when a password is required rather than prompting every time.
|
||||
#fail_on_password: false
|
||||
|
||||
## fimfiction.net stories allow chapters to be added out of order. So
|
||||
## the newest chapter may not be the last one. FFDL update doesn't
|
||||
## like that. If do_update_hook is uncommented and set true, the
|
||||
## adapter will discard all existing chapters from the newest one on
|
||||
## when updating to enforce accurate chapters.
|
||||
#do_update_hook:false
|
||||
|
||||
[www.harrypotterfanfiction.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[www.henneth-annun.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:The Hobbit
|
||||
|
||||
[www.hpfandom.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.hpfanficarchive.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.ik-eternal.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:InuYasha,Kagome
|
||||
extraships:InuYasha/Kagome
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1328,30 +1342,35 @@ extraships:InuYasha/Kagome
|
||||
## personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:InuYasha
|
||||
extracharacters:InuYasha,Kagome
|
||||
extraships:InuYasha/Kagome
|
||||
|
||||
[www.jlaunlimited.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:JLA
|
||||
|
||||
[www.libraryofmoria.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
[www.mediaminer.org]
|
||||
|
||||
[www.midnightwhispers.ca]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
## some sites include images that we don't ever want becoming the
|
||||
## cover image. This lets you exclude them.
|
||||
cover_exclusion_regexp:/stories/999/images/.*?_trophy.png
|
||||
@@ -1366,10 +1385,6 @@ extracategories:CSI
|
||||
extraships:Nick Stokes/Greg Sanders
|
||||
|
||||
[www.phoenixsong.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extraships:Harry Potter/Ginny Weasley
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1380,9 +1395,13 @@ extraships:Harry Potter/Ginny Weasley
|
||||
## phoenixsong.net, oddly, can have high rated chapters (login
|
||||
## required) in the middle of a lower rated story. Use this to force
|
||||
## FFDL to always login to phoenixsong.net so those stories download
|
||||
## correctly.
|
||||
## correctly. If you have a login, this is recommended.
|
||||
#force_login:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extraships:Harry Potter/Ginny Weasley
|
||||
|
||||
[www.potionsandsnitches.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
@@ -1405,17 +1424,15 @@ extracategories:Psych
|
||||
#is_adult:true
|
||||
|
||||
[www.qaf-fic.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Queer as Folk
|
||||
|
||||
[www.restrictedsection.org]
|
||||
extracategories:Harry Potter
|
||||
extragenres:Erotica
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1423,20 +1440,21 @@ extragenres:Erotica
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extragenres:Erotica
|
||||
|
||||
[www.scarvesandcoffee.net]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Glee
|
||||
extracharacters:Kurt Hummel,Blaine Anderson
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
[www.simplyundeniable.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1444,15 +1462,18 @@ extracategories:Harry Potter
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
[www.sinful-desire.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Supernatural
|
||||
extracategories:Harry Potter
|
||||
|
||||
[www.sinful-desire.org]
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Supernatural
|
||||
|
||||
[www.siye.co.uk]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
@@ -1460,25 +1481,26 @@ extracharacters:Harry Potter,Ginny Weasley
|
||||
extraships:Harry Potter/Ginny Weasley
|
||||
|
||||
[www.squidge.org/peja]
|
||||
# www.squidge.org/peja calls it Fandom <shrug>
|
||||
## www.squidge.org/peja calls it Fandom <shrug>
|
||||
category_label:Fandom
|
||||
# Remove numWords -- www.squidge.org/peja word counts are inaccurate
|
||||
|
||||
## Remove numWords -- www.squidge.org/peja word counts are inaccurate
|
||||
titlepage_entries: seriesHTML,category,genre,language,characters,ships,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,site,description
|
||||
|
||||
[www.squidge.org/peja:txt]
|
||||
## Add URLs since there aren't links.
|
||||
# Remove numWords -- www.squidge.org/peja word counts are inaccurate
|
||||
## Add URLs since there aren't links and remove numWords --
|
||||
## www.squidge.org/peja word counts are inaccurate
|
||||
titlepage_entries: series,seriesUrl,category,genre,language,status,datePublished,dateUpdated,dateCreated,rating,warnings,numChapters,site,storyUrl, authorUrl, description
|
||||
|
||||
[www.storiesofarda.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Lord of the Rings
|
||||
|
||||
[www.thepetulantpoetess.com]
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
@@ -1521,9 +1543,6 @@ collect_series: false
|
||||
extracategories:Twilight
|
||||
|
||||
[www.twilighted.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1531,13 +1550,13 @@ extracategories:Twilight
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## twilighted.net (ab)uses series as personal reading lists.
|
||||
collect_series: false
|
||||
|
||||
[www.twiwrite.net]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## Some sites require login (or login for some rated stories) The
|
||||
## program can prompt you, or you can save it in config. In
|
||||
## commandline version, this should go in your personal.ini, not
|
||||
@@ -1545,20 +1564,23 @@ extracategories:Twilight
|
||||
#username:YourName
|
||||
#password:yourpassword
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Twilight
|
||||
|
||||
## twiwrite.net (ab)uses series as personal reading lists.
|
||||
collect_series: false
|
||||
|
||||
[www.walkingtheplank.org]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Harry Potter
|
||||
extraships:Severus Snape/Harry Potter
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Harry Potter
|
||||
extracharacters:Severus Snape,Harry Potter
|
||||
extraships:Severus Snape/Harry Potter
|
||||
|
||||
[www.whofic.com]
|
||||
|
||||
[www.wizardtales.net]
|
||||
@@ -1580,14 +1602,14 @@ extracategories:X-Men Movie
|
||||
extracharacters:Wolverine,Rogue
|
||||
|
||||
[www.wraithbait.com]
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: Atlantis
|
||||
|
||||
## Some sites do not require a login, but do require the user to
|
||||
## confirm they are adult for adult content. In commandline version,
|
||||
## this should go in your personal.ini, not defaults.ini.
|
||||
#is_adult:true
|
||||
|
||||
## Site dedicated to these categories/characters/ships
|
||||
extracategories:Stargate: Atlantis
|
||||
|
||||
[overrides]
|
||||
## It may sometimes be useful to override all of the specific format,
|
||||
## site and site:format sections in your private configuration. For
|
||||
|
||||
+3
-7
@@ -53,13 +53,9 @@
|
||||
{% if fic.failure %}
|
||||
<h3>fanfiction.net / fimfiction.net</h3>
|
||||
<p>
|
||||
FYI, fanfiction.net appears to be blocking access from Google
|
||||
App Engine, which prevents this web service. There's
|
||||
nothing I can do about it. At the time of writing, the
|
||||
latest CLI and calibre plugin versions worked.
|
||||
</p>
|
||||
<p>It appears that FimFiction.net is also blocking access from Google
|
||||
App Engine now.
|
||||
As of Jan 13, 2014, fanfiction.net & fimfiction.net
|
||||
are working again. I'd ask that users limit the number of
|
||||
stories they download from those sites, thanks.
|
||||
</p>
|
||||
<span id='error'>{{ fic.failure }}</span>
|
||||
{% endif %}
|
||||
|
||||
Reference in New Issue
Block a user