1. DLP Flash Christmas Competition + Writing Marathon 2024!

    Competition topic: Magical New Year!

    Marathon goal? Crank out words!

    Check the marathon thread or competition thread for details.

    Dismiss Notice
  2. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

Scryer: Fanfiction Search Database (Alpha)

Discussion in 'Announcements' started by Lord Ravenclaw, Feb 17, 2014.

  1. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,372
    Location:
    Denver, CO
    Announcing the newest in fiction search, Scryer. Scryer searches stories from FFN in a more powerful way than can currently be achieved with FFN's interface, and does so in a mobile-friendly manner. Scryer supports full-text search on title, author, and summary as well as filtering functions for characters, categories, and other fields.

    Please ensure noscript is turned off before attempting use.

    http://scryer.darklordpotter.net/


    Known issues
    None at the moment.

    Please leave feedback and bug reports in this thread.



    Future work
    • Chrome extension for reading tracking and analytics [soon]
    • Collaborative tagging StackOverflow style
    • Multiple site search (e.g. FicWad, Portkey, AO3)
    • Deeper integration into DLP & DLP Mobile
    • FFN Alert/Fav import for a better 'Update Feed'
    • Daily update summary emails based on the above

    Additionally, if you're a programmer, join us in DLP IRC #programming -- Scryer is open-source software built on top of the DLP API, contributions welcome. Scryer is Ruby on Rails and the DLP API is written in Java, and interfaces with ElasticSearch.
     
  2. Grubdubdub

    Grubdubdub Supreme Mugwump

    Joined:
    Feb 22, 2008
    Messages:
    1,604
    Other features that would be great are WIP/Completed classification, and a last updated/first published filters. The DLP Score integration is by far the best feature, it really is great. Huge thumbs up.
     
  3. Hero of Stupidity

    Hero of Stupidity Villain of Sensibility ~ Prestige ~ DLP Supporter

    Joined:
    Oct 5, 2010
    Messages:
    342
    Gender:
    Male
    Location:
    Hungary
    High Score:
    3,172
    I second the WIP/Completed. Maybe like FF the search function should fold itself up after the search is completed. Also I personalyl like to read with black background so maybe a Background change option would be good.

    Awesome job.
     
  4. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,372
    Location:
    Denver, CO
    Already covered in future work, it's 'in progress' but requires a full re-crawl to implement. Should be ready soon.

    I like that, I'll see what I can do.

    I've been looking at ways of making this easier to use and to make design sense. It won't be a fold-up, but I have other plans.

    You can use stylish if you want, but adding theme support to a search engine is really really far down my priorities list.
     
  5. FreakLord

    FreakLord Professor DLP Supporter

    Joined:
    Feb 13, 2011
    Messages:
    459
  6. Photon

    Photon Order Member

    Joined:
    Jul 15, 2012
    Messages:
    838
    Location:
    Poland
    What is indexed? All stories in Harry Potter category?
     
  7. Grubdubdub

    Grubdubdub Supreme Mugwump

    Joined:
    Feb 22, 2008
    Messages:
    1,604
    Running a search with no parameters shows that there are 520,000 stories indexed, about 150,000 less than ffn. Maybe the General genre stories make the difference?

    The Lucene Querey Syntax is working very well as far as I tested. Might be nice to add checkboxes for popular searches, such as Azkaban, Veela, -dumbles, Triwizard, and what not.
     
  8. Andrela

    Andrela Plot Bunny DLP Supporter

    Joined:
    Apr 19, 2012
    Messages:
    5,048
    Gender:
    Female
    Location:
    Silesia
    Thanks for doing this. Thumbs up.
     
  9. Klackerz

    Klackerz Bridgeburner

    Joined:
    Oct 22, 2009
    Messages:
    786
    Location:
    India
    Only the English stories are shown by default in Scryer. When you check the English stories in ffn, they are matching.

    As Raven said in the OP, currently only HP stories are indexed without any crossover support.
     
  10. KGB

    KGB Headmaster

    Joined:
    Jan 18, 2011
    Messages:
    1,067
    One feature I'm missing from the FFN options is the time range filters. Being able to filter out old abandoned fic's is a must have.
     
  11. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,372
    Location:
    Denver, CO
    01-17-14 Changelog
    • (Fix) Fulltext: Summary/title/author can now contain forward slashes. e.g. 'H/Hr' is now supported.
    • (Fix) Search URL with query params will now perform a search upon load.
    • (New) Sorts: Added a Popular & Recent sort. This is currently a little slow, but mixes activity (reviews, follows, favorites) and recency measures. Combined with filters, this makes it easier to find somewhat interesting fics.
    • (New) Filters: Added a chapter number filter, so you can limit it to say 5-10 chapters.
     
  12. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,372
    Location:
    Denver, CO
    I agree, I'll add this to my to-do.
     
  13. Spanks

    Spanks Chief Warlock

    Joined:
    Jun 8, 2007
    Messages:
    1,525
    Location:
    New Jersey
    Let me know if this is dumb or not. I see that it shows the ratings for stories DLP has reviewed and I was wondering if it's possible to rate the stories on Scryer too. That way as we are browsing through our search results we'll see how other readers rated a story. It might also help us find those few fics that are good time wasters, but not good enough to be put up for review on DLP.

    Anyway, awesome work, sir.
     
  14. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,372
    Location:
    Denver, CO
    You can currently see what reviews on DLP are for Scryer results, but I haven't decided if I'll allow rating / upvoting from the Scryer results.
     
  15. Lord Raine

    Lord Raine Disappeared DLP Supporter

    Joined:
    May 4, 2010
    Messages:
    1,038
    Just wanted to let you know that I've used it for all of thirty seconds and I already found a fic I've been trying to re-find after reading it about a year and a half ago.

    This is fantastic. I almost want to ask you to get around to making it work beyond the HP section of the site, if only because I've been looking for the very first piece of fanfiction I ever read for a long time, and this would probably find it if anything could.

    Amazing work.
     
  16. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,372
    Location:
    Denver, CO
    Which fandom are you looking for? I'm currently looking at indexing everything HP crosses with (both the crossovers and non-crossovers), starting with a few smaller ones.
     
  17. Tommy

    Tommy The Green Ranger

    Joined:
    Nov 30, 2012
    Messages:
    202
    Location:
    In a bomb shelter, South Africa.
    Perhaps the most wanted fandam besides HP is probably Dresden, so that could be a good place to start. Then trickle down to some popular manga, etc.
     
  18. PotterFan

    PotterFan High Inquisitor

    Joined:
    Aug 12, 2011
    Messages:
    575
    Location:
    The Road of Life
    Naruto should be the next fandom....Definitely Naruto. I need -mpreg and -slash to cut through all the BS.
     
  19. phazer11

    phazer11 Third Year

    Joined:
    Mar 26, 2011
    Messages:
    95
    Location:
    IN THE DEEP SouthEastern USA
    Dresden Files crossovers and regular fictions would be my next request for inclusion in Scryer.
     
  20. Lord Raine

    Lord Raine Disappeared DLP Supporter

    Joined:
    May 4, 2010
    Messages:
    1,038
    If you're starting small, it won't be on the list. The very first one I read was a Naruto fic. I remember almost every detail about the story, and by literally any and all standards whatsoever we have today, it was truly objectively horrible. But it was the first one I ever read (and honestly it was so long ago that a lot of the cliches really weren't cliches yet), and I've always been trying to find it again. Plus the author wrote a few other things that weren't bad either. Mostly slice of life and comedy stuff.

    I'll confess that I haven't actually looked, but I wouldn't have imagined there would have been enough Dresden on fanfiction.net to really be worth filtering. It might be good as a test to see how well Scryer works in other fandoms, but I'm not really sure Scryer would be necessary to search like it would be for some of the larger ones.
     
    Last edited: Mar 8, 2014
Loading...