Page scrape story descriptions again for fimfiction.net due to API issues.

This commit is contained in:
Jim Miller
2013-03-02 15:46:47 -06:00
parent b76e50719b
commit abde9fdf8d
3 changed files with 12 additions and 7 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
# ffd-retief-hrd fanfictiondownloader
application: fanfictiondownloader
version: 4-4-45
version: 4-4-46
runtime: python27
api_version: 1
threadsafe: true
@@ -181,10 +181,15 @@ class FimFictionNetSiteAdapter(BaseSiteAdapter):
self.setCoverImage(self.url,coverurl)
# the fimfic API gives bbcode for desc, not html.
# btw, bbcode honors newlines, html doesn't. change newlines to br tags.
self.setDescription(self.url,
bbcodeparser().parse(storyMetadata["description"]).html(doDeepCopy=False).replace('\r','').replace('\n','<br />'))
self.setDescription(self.url,soup.find("div", {"class":"description"}))
# if "description" in storyMetadata and storyMetadata["description"]:
# # the fimfic API gives bbcode for desc, not html.
# # btw, bbcode honors newlines, html doesn't. change newlines to br tags.
# self.setDescription(self.url,
# bbcodeparser().parse(storyMetadata["description"]).html(doDeepCopy=False).replace('\r','').replace('\n','<br />'))
# elif "short_description" in storyMetadata and storyMetadata["short_description"]:
# self.setDescription(self.url,
# bbcodeparser().parse(storyMetadata["short_description"]).html(doDeepCopy=False).replace('\r','').replace('\n','<br />'))
# Dates are in Unix time
# Take the publish date from the first chapter posted
+2 -2
View File
@@ -57,7 +57,7 @@
<h3>Changes:</h3>
<p>
<ul>
<li>www.ficwad.com -&gt; ficwad.com. Ficwad changed their canonical domain. You'll need to change [www.ficwad.com] to [ficwad.com] in your configuration.</li>
<li>Change how fimfiction.net story descriptions are collected.</li>
</ul>
</p>
<p>
@@ -68,7 +68,7 @@
If you have any problems with this application, please
report them in
the <a href="http://groups.google.com/group/fanfic-downloader">FanFictionDownLoader Google Group</a>. The
<a href="http://4-4-44.fanfictiondownloader.appspot.com">Previous Version</a> is also available for you to use if necessary.
<a href="http://4-4-45.fanfictiondownloader.appspot.com">Previous Version</a> is also available for you to use if necessary.
</p>
<div id='error'>
{{ error_message }}