Fix for sequels in fimf, remove prequels--not in code or on site anymore.

This commit is contained in:
Jim Miller
2015-01-27 11:08:36 -06:00
parent 4d6c1f36c1
commit d4e4880dea
3 changed files with 11 additions and 16 deletions
+2 -5
View File
@@ -1603,7 +1603,7 @@ extracategories:My Little Pony: Friendship is Magic
## Extra metadata that this adapter knows about. See [dramione.org]
## for examples of how to use them.
extra_valid_entries:likes,dislikes,views,total_views,short_description,groups,groupsUrl,groupsHTML,prequel,prequelUrl,prequelHTML,sequels,sequelsUrl,sequelsHTML,comment_count,coverSource,coverSourceUrl,coverSourceHTML
extra_valid_entries:likes,dislikes,views,total_views,short_description,groups,groupsUrl,groupsHTML,sequels,sequelsUrl,sequelsHTML,comment_count,coverSource,coverSourceUrl,coverSourceHTML
likes_label:Likes
dislikes_label:Dislikes
views_label:Highest Single Chapter Views
@@ -1612,9 +1612,6 @@ short_description_label:Short Summary
groups_label:Groups
groupsUrl_label:Groups URLs
groupsHTML_label:Groups
prequel_label:Prequel
prequelUrl_label:Prequel URL
prequelHTML_label:Prequel
sequels_label:Sequels
sequelsUrl_label:Sequel URLs
sequelsHTML_label:Sequels
@@ -1630,7 +1627,7 @@ keep_in_order_groupsUrl:true
## Assume entryUrl, apply to "<a class='%slink' href='%s'>%s</a>" to
## make entryHTML.
make_linkhtml_entries:prequel,sequels,groups,coverSource
make_linkhtml_entries:sequels,groups,coverSource
[www.harrypotterfanfiction.com]
## Some sites do not require a login, but do require the user to
@@ -275,12 +275,13 @@ class FimFictionNetSiteAdapter(BaseSiteAdapter):
self.story.addToList("groups",stripHTML(groupName).replace(',', ';'))
#sequels
sequelStoryHeader = soup.find('h1', {'class':'header-stories'}, text="Sequels")
if not sequelStoryHeader == None:
sequelContainer = sequelStoryHeader.parent.parent
for sequel in sequelContainer.find_all('a', {'class':'story_link'}):
self.story.addToList("sequelsUrl", 'http://'+self.host+sequel["href"])
self.story.addToList("sequels", stripHTML(sequel).replace(',', ';'))
for header in soup.find_all('h1', {'class':'header-stories'}):
# I don't know why using text=re.compile with find() wouldn't work, but it didn't.
if header.text.startswith('Sequels'):
sequelContainer = header.parent
for sequel in sequelContainer.find_all('a', {'class':'story_link'}):
self.story.addToList("sequelsUrl", 'http://'+self.host+sequel["href"])
self.story.addToList("sequels", stripHTML(sequel).replace(',', ';'))
#author last login
userPageHeader = soup.find('div', {'class':re.compile(r'\buser-page-header\b')})
+2 -5
View File
@@ -1603,7 +1603,7 @@ extracategories:My Little Pony: Friendship is Magic
## Extra metadata that this adapter knows about. See [dramione.org]
## for examples of how to use them.
extra_valid_entries:likes,dislikes,views,total_views,short_description,groups,groupsUrl,groupsHTML,prequel,prequelUrl,prequelHTML,sequels,sequelsUrl,sequelsHTML,comment_count,coverSource,coverSourceUrl,coverSourceHTML
extra_valid_entries:likes,dislikes,views,total_views,short_description,groups,groupsUrl,groupsHTML,sequels,sequelsUrl,sequelsHTML,comment_count,coverSource,coverSourceUrl,coverSourceHTML
likes_label:Likes
dislikes_label:Dislikes
views_label:Highest Single Chapter Views
@@ -1612,9 +1612,6 @@ short_description_label:Short Summary
groups_label:Groups
groupsUrl_label:Groups URLs
groupsHTML_label:Groups
prequel_label:Prequel
prequelUrl_label:Prequel URL
prequelHTML_label:Prequel
sequels_label:Sequels
sequelsUrl_label:Sequel URLs
sequelsHTML_label:Sequels
@@ -1630,7 +1627,7 @@ keep_in_order_groupsUrl:true
## Assume entryUrl, apply to "<a class='%slink' href='%s'>%s</a>" to
## make entryHTML.
make_linkhtml_entries:prequel,sequels,groups,coverSource
make_linkhtml_entries:sequels,groups,coverSource
[www.harrypotterfanfiction.com]
## Some sites do not require a login, but do require the user to