1. DLP Flash Christmas Competition + Writing Marathon 2024!

    Competition topic: Magical New Year!

    Marathon goal? Crank out words!

    Check the marathon thread or competition thread for details.

    Dismiss Notice
  2. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

DLP Archive - PDF Format

Discussion in 'General Discussion' started by Dryops, Jul 17, 2013.

  1. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    Hello all, putting this up here as an interest check. I've been getting some feedback on IRC from a few members, and wanted to see what the general interest level was for something like this.

    I've been working on a few conversion scripts, and came up with a general pdf (and now e-pub) theme for ff.net stories. I've been trying to loosely base it on the old html archives format.

    Current Features

    - Embedded Links to chapters in table of contents
    - Includes original cover image from ff.net, if one is set, for pdf only (otherwise, placeholder)
    - Summaries added to e-pub files (7/20)

    Edit 7/20: V1.5, added ~500 stories bringing the total up to just under 1100. Still have about 180 from the '07 archive to add manually. If anyone has a copy of the '08, so that I could compare / update anything that wasn't in the 07 or still in the library, please send me a pm.

    Get the Zips / Samples

    If you are interested in getting the updated versions automatically, via a shared dropbox folder, either leave a comment or send a PM with the following information:

    Dropbox e-mail , and whether you prefer pdf, epub, or both (separate folders to save you space)
     
    Last edited: Jul 20, 2013
  2. Joe's Nemesis

    Joe's Nemesis High Score: 2,058 ~ Prestige ~

    Joined:
    Jan 29, 2012
    Messages:
    1,192
    High Score:
    2,058
    You're probably familiar with FLAG, but honestly, it really does everything you're doing here, except for centering text in the story if you have text centered. Also allows for more than just PDFs.

    Don't mean to be a buzzkill, but not sure what this program has over what I'm already using.

    I Do appreciate the work however.
     
  3. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    Not a buzzkill at all, thanks for the feedback. I actually use flagfic fairly regularly for individual files. Unfortunately, FlagFic only allows for one conversion at a time, and then you have to go back in and paste the url later to get an updated copy. This tool was my attempt to overcome those limitations.


    I am essentially looking for input / feedback on the pdf format, before I convert ~600 stories (from the library), to zip and share.


    The idea behind the script I'm working on is that it can be fed some input, either the list of library fics, or for converting all fics from any community above a certain word count.

    The converter itself will not be publicly available, but just the finished pdf zip (at least for the moment, since it is hosted on my $200 Linux laptop, so not up for any kind of load).

    The converter was mainly developed as a personal archiving tool for communities that I enjoy (aka, converted all stories over 100k words from Top of the Mark earlier today), but the structure is such that I could easily have it create pdfs of all stories in the library that are hosted on ff.net, and then zip / share that file. I store chapters as entries in a database, so if I run the script again, it just has to look for new chapters of that story (aka, updating the zip would be much easier / faster).
     
    Last edited: Jul 17, 2013
  4. Joe's Nemesis

    Joe's Nemesis High Score: 2,058 ~ Prestige ~

    Joined:
    Jan 29, 2012
    Messages:
    1,192
    High Score:
    2,058
    Ahh, misunderstood then. Sorry.

    The green made it difficult to see, IMO. But other than that, I can't think of too many things that jumped out at me. Are you able to reflow text? That, I think, is a pretty big issue for some people. I'd love to pop a few PDFs on my phone or tablet and not have to worry about being connected anywhere, but without the text reflowing, it gets difficult.

    And yeah, one story at a time wouldn't be the best way to convert the library.
     
  5. Photon

    Photon Order Member

    Joined:
    Jul 15, 2012
    Messages:
    838
    Location:
    Poland
    Maybe it is just me but green on green is both unreadable and ugly. What is wrong with black on white?

    Why not?
     
  6. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    I am currently looking into that. (edit: Looked into epub creation with php, looks pretty simple. I'll probably run two copies of the archive, one in the formatted pdf, and one in epub for those who prefer it).


    I agree that it is harder to read, so I've been playing with some other color combinations. See attached.

    It's currently running on my lower end laptop. Don't really feel like hosting it at the moment, although I would be willing to share the source once things are cleaned up if anyone was interested. You would need a host that supports cURL and larger mysql databases, with the current build.


    --- Updated PDF Format --- (still need to fix the colour on cover photo)

    A-Bad-Week-at-the-Wizengamot-DisobedienceWriter
     
    Last edited: Jul 17, 2013
  7. melior

    melior Seventh Year

    Joined:
    Oct 25, 2011
    Messages:
    226
    What is the advantage of creating a pdf over, say, epub for phones/tablets/e-readers or just html for desktop reading?

    Most fanfiction I read is epub on my nook or phone. I prefer the fanficdownloader Calibre plugin (which also has a webapp version here) over FLAG. It's open source (FLAG is not), written in Python.

    If I had interest in doing this, that's what I'd do: write a script to use fanficdownloader (source here) and maybe xhtml2pdf (also open source python) if pdf is necessary.
     
  8. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    Thanks for the reply. I personally prefer pdf, but as I mentioned in my last reply, creating an epub from the same data should be pretty easy, so I plan to offer both formats (updated original post with that also, just now).

    I'm doing this as a coding exercise for myself, so I'm trying to limit the pre-built libraries to things like the pdf / epub generators.
     
  9. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    Version 1 is available, including pdf and epub format. (link is in the first post). I will update that link each time a new version is available, and reply to the thread here to notify you.
     
  10. Tommy

    Tommy The Green Ranger

    Joined:
    Nov 30, 2012
    Messages:
    202
    Location:
    In a bomb shelter, South Africa.
    Could you maybe create a dropbox folder so we can share and get the updates as you get them? That way, when your script updates fics, it'll mirror on our db folders? Don't mind downloading the link, but man, it'd be convenient...
     
    Last edited: Jul 20, 2013
  11. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    Sure! They're already on Dropbox now, not a big deal to extend that. Just shoot me a PM with your dropbox e-mail, and whether you want to get pdfs, epub, or both (I'll create separate folders for each to save you space).

    For those folders, would you prefer them still in zipped format, or would you prefer unzlpped?
    (basically, the thing I see happening is someone unzipping it inside the dropbox folder, and the rest of us getting spammed with 520 files)
     
    Last edited: Jul 19, 2013
  12. Tommy

    Tommy The Green Ranger

    Joined:
    Nov 30, 2012
    Messages:
    202
    Location:
    In a bomb shelter, South Africa.
    Shot you a PM - not sure if it sent, though... Not showing up in my sent folder.
     
  13. Dryops

    Dryops Second Year DLP Supporter

    Joined:
    May 29, 2007
    Messages:
    73
    Location:
    United States
    Update (added ~500 stories + story summaries)

    Major update, added ~500 stories, including the recycling bin and what I could find ff.net links for out of the '07 zip html archive. Also added the story summary to the epub files, so you can see them in Calibre without having to open the story (previous version had generic placeholder text).

    This still leaves around 180 stories I'm going to have to at least somewhat manually add in, but eventually they should all be in a standard format. I have updated the zip files in the link up top.

    I also added a text file with the titles of all of the stories that are currently in the zip, on the same link.

    Since this is all of the "easy" add ins, it will probably be a while until I release the next part, other than updates to the stories that are currently in. When I release all of the manually converted stories, I'll link both a full zip, with everything, and a zip that has just the ones added so that you do not have to re-download / add everything.


    Tommy, got it, and added you. You should get any epub updates automatically from now on.
     
    Last edited: Jul 20, 2013
Loading...