1. DLP Flash Christmas Competition + Writing Marathon 2024!

    Competition topic: Magical New Year!

    Marathon goal? Crank out words!

    Check the marathon thread or competition thread for details.

    Dismiss Notice
  2. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

Scryer: Fanfiction Search Database (Alpha)

Discussion in 'Announcements' started by Lord Ravenclaw, Feb 17, 2014.

  1. Avierh

    Avierh First Year

    Joined:
    Jan 1, 2012
    Messages:
    36
    Location:
    US
    There's only about 837 fics for Dresden Files on fanfiction.net. I don't think that Dresden Files has that many writers outside of DLP for where you'd need a filter.
     
  2. Snarf

    Snarf Squanchin' Party Bro! ~ Prestige ~ DLP Supporter

    Joined:
    Apr 27, 2007
    Messages:
    18
    Gender:
    Male
    Location:
    Forty-Six & 2
    High Score:
    1,832
    So the newest results I have been receiving are from six days ago. This is happening on both my phone and my laptop, so I'm pretty sure that this isn't the fault of browser settings. Is there something I can due to alleviate this as a personal issue, or is this happening to others as well?
     
  3. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    This has been fixed as of a few days ago.
     
  4. Photon

    Photon Order Member

    Joined:
    Jul 15, 2012
    Messages:
    838
    Location:
    Poland
    I have mu own personal fanfiction database, with quite complicated sets of rules.

    One of them is detector of slash stories, smarter than just "no word slash in description" - and it seems quite good at hiding slash.

    I wonder - would somebody be interested in my code and use it to implement "no slash" checkbox (it uses rules that cannot be currently set in Scryer, like "genre contains <romance> AND both first character and second characters are male")?
     
  5. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    These kinds of rules are possible in Scryer, but not exposed via the UI yet. It is open source, I welcome additions.
     
  6. Imperius

    Imperius First Year

    Joined:
    May 9, 2008
    Messages:
    39
    Location:
    Nowhere Important
    Hi, I haven't been getting any results from Scryer for last one week. Is there an indexing problem?

    I have some experience in building search engines & web scraping. Let me know if you need a hand.
     
  7. Oz

    Oz For Zombie. Moderator DLP Supporter

    Joined:
    Jan 31, 2008
    Messages:
    9,027
    Gender:
    Female
    Location:
    Baile Átha Cliath
    Raven's been migrating host and a few things have broken along the way. Last I heard I think he said secondary services like Scryer and the API could be down for a while until he gets some free time.
     
  8. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Good call -- it's back up now. DLP API on the new server supports SSL, and browsers forced it (HSTS headers), but the frontend was attempting HTTP only, which results in a cross-origin error.
     
  9. SilverOtter

    SilverOtter Seventh Year

    Joined:
    Feb 20, 2011
    Messages:
    263
    Location:
    USA
    Is there any way to save my options between searches?

    Eg: I have a list of male characters that are commonly used in slash fics, and I have to manually add them each time I go to search.

    I tried bookmarking the search page after I entered them and searched, but it doesn't auto-fill in the characters. So if I want to modify the search at all I have to manually enter them all again.
     
  10. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Not yet, but this is something I'll fix in the next release.
     
  11. SilverOtter

    SilverOtter Seventh Year

    Joined:
    Feb 20, 2011
    Messages:
    263
    Location:
    USA
    The last couple of days I've been getting the error "No results found. Try another query."

    No matter what I try I always get this error. Is this a known bug?
     
  12. Spanks

    Spanks Chief Warlock

    Joined:
    Jun 8, 2007
    Messages:
    1,531
    Location:
    New Jersey
    I'm getting the same issue. Also, it won't open in Opera for me, but it's fine in other browsers.
     
  13. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Hey guys, resolved the issue with Scryer. Another issue with runaway processes. It shouldn't happen again.
     
  14. Radmar

    Radmar Disappeared

    Joined:
    Feb 3, 2014
    Messages:
    339
    Location:
    Czech Republic
    Scryer doesn't work in Chrome (Ver 38.0.2125.111 m). I use win 7. I disabled all extensions and tried restarting computer. Nothing works. I used another laptop (with same version of chrome, with extensions and win 8.1) and it works just fine. It also works fine in Firefox and IE. It looks like this: [​IMG]

    edit: It works now. Thanks Lord Ravenclaw.
     
    Last edited: Nov 3, 2014
  15. Jpzh2d

    Jpzh2d Disappeared

    Joined:
    May 11, 2014
    Messages:
    117
    Location:
    The Nethersphere
    Just a suggestion but could you please make it possible to include something that has a space in it. I know I'm not explaining well so I'll just give an example.

    If I want to search for a story that includes time travel in the summary I have to search for:
    '+time +travel' as a search for '+timetravel' will only return stories that have the word 'timetravel' in the summary.

    This isn't exactly a problem but it is kind of annoying and confusing at first. Maybe you could make escape sequences work (e.g. '+time\ travel') or use underscores to represent spaces (e.g '+time_travel'). Again its not a problem, but it is me being a nitpicking twat.

    If you need any Python (3.0 +) or Ruby scripting help you can contact me.
     
  16. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Use the syntax +"time travel"

    Lucene tokenizes query strings without quotes to group the tokens together.

    Appreciate the offer, I'll keep it in mind. We're usually in #programming on IRC, which is the easiest way to get into DLP OSS projects. The majority of DLP's projects are in PHP (vBulletin or legacy) or Java/Scala (new projects). That being said, I may be moving Scryer to be a Rails project (vs. just an Angular app talking to the DLP API)


    Clear your cache and try again. Also Ctrl+shift+R a few times.
     
  17. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Good news everyone!

    Progress has finally been made on Scryer. I'd honestly hit a wall with AngularJS in developing new features for Scryer. It was simpler to rewrite Scryer on Rails than to continue with AngularJS. It's allowed me to aggressively ship features at a rapid pace the last few days, notably HP crossover support. Multi-fandom support is also there, but disabled while we backfill the search index from FFN.

    The new version of Scryer can be found https://scryerdev.darklordpotter.net/ here, it may be unavailable occasionally as features are shipped and deployments are tweaked. That said, it should be relatively stable and iterating rapidly.

    This Scryer is friendly to JS-less browsers (and those with NoScript enabled). It's written in Ruby on Rails vs. a pure frontend JS framework like before. The design is slightly different as well. It's a little rough, and needs some more work, but IMO it's already superior to the existing Scryer.

    On the near-term roadmap, I'm expecting to also ship saved searches.
     
  18. golan

    golan Temporarily Banhammered DLP Supporter

    Joined:
    Oct 11, 2014
    Messages:
    567
    Location:
    Central Europe, for now.
    js-less browsers? Ain't that rather switched-off JS with NoScript or similar? And does the new interface go well on cellphones and tablets?

    Sent from my ALCATEL ONE TOUCH 997D using Tapatalk
     
  19. Oz

    Oz For Zombie. Moderator DLP Supporter

    Joined:
    Jan 31, 2008
    Messages:
    9,027
    Gender:
    Female
    Location:
    Baile Átha Cliath
    Screen-readers and text-only browsers like Lynx will trip over javascript.
     
  20. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    As The Wizard said, text-only browsers like links rarely have a JS engine. But yes, the primary case is NoScript.
     
Loading...