1. DLP Flash Christmas Competition + Writing Marathon 2024!

    Competition topic: Magical New Year!

    Marathon goal? Crank out words!

    Check the marathon thread or competition thread for details.

    Dismiss Notice

  2. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice
DLP Story Competition Guest User Registration Nag SPS + NTT

Scryer: Fanfiction Search Database (Alpha)

Discussion in 'Announcements' started by Lord Ravenclaw, Feb 17, 2014.

5/5, 6 votes
Page 3 of 6
  1. SilverOtter

    SilverOtter Seventh Year

    Joined:
    Feb 20, 2011
    Messages:
    263
    Location:
    USA
    Nov 10, 2014 #41
    Now that Crossover searches are possible, would it be possible to do multiple crossover searches?

    There are 2 'Star Wars' and 1 'Star Wars: The Clone Wars' Fandoms and it would be nice if I could search all 3 of those at once
     
  2. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 10, 2014 #42
    I'm actually already working on this :) It requires a few changes with how queries are done currently, but I'm hoping to roll it out today or tomorrow.
     
  3. SilverOtter

    SilverOtter Seventh Year

    Joined:
    Feb 20, 2011
    Messages:
    263
    Location:
    USA
    Nov 11, 2014 #43
    Excellent! :D

    Is it possible to get the number of stories in each Fandom, like it shows on the fanfiction.net page? That's another thing I would like to see in the drop down menu.

    Here: https://www.fanfiction.net/crossovers/Harry-Potter/224/

    So in the drop down menu on Scryer, could you do like 'Star Wars (302)', instead of just 'Star Wars'?
     
  4. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 11, 2014 #44
    I've rolled out multi-crossover support. The counts are coming, but will take a bit more work.

    Additionally, old Scryer has been removed. https://scryer.darklordpotter.net/ is now Scryer 2.0.
     
    Gizmore
  5. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 11, 2014 #45
    Also done. It's sorted descending by count of fics.


    Gauging interest: I'm thinking of building a Chrome extension / FF extension that beaconed what you were reading back to Scryer. This way Scryer could keep bookmarks for you and tell you if you've already read something. How many would actually use it?

    I'd use it, so I'll probably build it regardless.

    There's a lot of other potential opt-in analytics that might be interesting. e.g. plotting reading speed per hour, and number of words read per week + stories read per week.
     
    Scarty, Oz, Sauce Bauss, Reign, Nuit, Brukel, readerboy7
  6. Reign

    Reign Fourth Year

    Joined:
    Oct 27, 2008
    Messages:
    124
    Gender:
    Male
    Nov 11, 2014 #46
    I would be extremely interested in any analytics you could provide. There are many features I would love to see but in doing so it might turn Scryer into more of a pensieve. So I wont mention them.

    For the last few months I've started a project to record any Worm fan fiction I've read in a Google spreadsheet (Title/Link, word count, rating, last updated, date started, ability's, etc), as a trial before I do the same for Harry Potter. I've come across many annoying problems.

    One such problem is wasting time filling out a form for a story I only spent less than a minute reading, but with Scryer it could easily be saved, marked, blacklisted, etc so I would know not to interact with that story again.

    Any aggregated data you could get on a personal level or even as a group would be interesting and even fun to have. Being opt-in is just icing on the cake. If this is implemented it would be the first time I'd ever willing allow browser tracking.
     
  7. KGB

    KGB Headmaster

    Joined:
    Jan 18, 2011
    Messages:
    1,067
    Nov 12, 2014 #47
    Yes we need a blacklist/dislike feature. I have a script that hides authors/stories i've disliked. It saves a lot of time when searching for something to read.

    An integrated feature, that wouldn't produce blank page after a blank page would be even better.

    Not super sure about the analytics, but if you make it opt in I don't really have any problems.
     
    KrzaQ
  8. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 12, 2014 #48
    Good idea. I've added that to the list, along with default search settings.

    You'll have to install the extension, which would be the "opt in".
     
  9. SilverOtter

    SilverOtter Seventh Year

    Joined:
    Feb 20, 2011
    Messages:
    263
    Location:
    USA
    Nov 13, 2014 #49
    The ability to login and have a tagging system where you could opt-in to viewing tags from every user, or a certain set of users who have similar tastes to you would be nice, but I think that would require a lot of work to do.
     
  10. FreakLord

    FreakLord Professor DLP Supporter

    Joined:
    Feb 13, 2011
    Messages:
    459
    Nov 13, 2014 #50
    This was what I once proposed to Raven. But it was more of a DLP sponsored tags.
     
  11. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 14, 2014 #51
    Definitely on my TODO list. This feature will require some planning. Until traffic increases though, I'm not certain the value for effort is there yet.

    The viability of Scryer is really dependent on 1) getting more data and 2) building personalization / assistant-style features. Scryer has to be more than a search engine.

    --------------------------

    Minor update tonight, pairings (as much as FFN tells us, I haven't done any extra summary inference) are now shown in search results. Also squashed some various bugs.

    Edit: disabled pairings, there was an unfortunate bug in the parser that requires a reindex of FFN.
     
  12. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 21, 2014 #52
    Pairing view (which characters the author claims are paired) has been reenabled. Additionally, sorting / searching by DLP scores has been fixed, which was broken by a bug in a new version of ElasticSearch.

    I've also brought back an older sorting mode broken in the past by another ElasticSearch upgrade called Popular and Recent. This mode ranks stories by update date and blends in reviews, follows, and favorites. It's most powerful when used with a more specific search (e.g. -slash + your favorite characters) or a smaller crossover category.

    Next up: 3 new fandoms (which ones? You decide.), user accounts + DLP account linkage to then lead to automatic FFN bookmarking via Chrome extension + rating/note taking.
     
    Sauce Bauss, KrzaQ, Nuit, readerboy7
  13. readerboy7

    readerboy7 Fourth Year

    Joined:
    Jan 11, 2013
    Messages:
    126
    Location:
    GMT + 12
    Nov 21, 2014 #53
    Would it be possible to have permantent searches? e.g. +fred, +george, -Mpreg, that sort of thing.

    As to new fandoms, I would prefer Naruto and Worm, although Worm doesn't really seem to be poular on FFnet, so...
    Whatever fandoms you think are best.

    Edit: I was browsing Scryer and I found that there was atleast one story by Perfect Lionheart on the list. Would it be possible to have blacklisted authors as well?
     
    Last edited: Nov 21, 2014
  14. Sauce Bauss

    Sauce Bauss Second Year ~ Prestige ~ DLP Supporter

    Joined:
    Apr 4, 2008
    Messages:
    61
    High Score:
    1411
    Nov 21, 2014 #54
    The problem with using Scryer to find good Worm fics is the assumption that there are good Worm fics to be found.
    There aren't any.
     
  15. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 21, 2014 #55
    After user accounts are implemented, I'm planning on adding saved searches, blacklisted authors, and default searches (your template for searches).
     
  16. Lium

    Lium First Year

    Joined:
    Jul 1, 2011
    Messages:
    43
    Location:
    The Greatest of Britains.
    Nov 21, 2014 #56
    Fandomwise, I'd like to see ATLA/Teen Titans.
     
  17. FreakLord

    FreakLord Professor DLP Supporter

    Joined:
    Feb 13, 2011
    Messages:
    459
    Nov 21, 2014 #57
  18. Zombie

    Zombie Black Philip Moderator DLP Supporter ⭐

    Joined:
    Apr 28, 2007
    Messages:
    6,036
    Nov 21, 2014 #58
    Naruto, Bleach I would like to see on the new Fandom. I can't think of a third. I have no third.
     
  19. Oz

    Oz For Zombie. Moderator DLP Supporter

    Joined:
    Jan 31, 2008
    Messages:
    9,027
    Gender:
    Female
    Location:
    Baile Átha Cliath
    Nov 22, 2014 #59
    Needs moar Twilight.
     
  20. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,373
    Location:
    Denver, CO
    Nov 25, 2014 #60
    I've deployed seven new fandoms and their respective crossovers into the search index.

    • Dresden Files
    • Fate/stay night
    • Naruto (still backfilling, ~50% done)
    • Stargate SG-1
    • Stargate Atlantis
    • Star Trek: TNG
    • Star Wars

    Crossovers have been loaded as well.

    Next up:
    • X-Men?
    • Doctor Who?
    • Dragon Age
    • ?

    I'll be rolling out new fandoms by request and popularity. Currently been looking at fandoms often crossed with HP.

    Edit:

    • Avatar: The Last Airbender
    • Legend of Korra
    • Halo
    • LOTR
    • Bleach
    • ASOIAF
    • Buffy
    • Mass Effect
    • Avengers
    • X-Men the Movie

    Are now all in progress backfilling. I'll enable access to them tomorrow once initial indexing has completed.
     
    Oz, Sauce Bauss, KrzaQ, Klackerz, Nuit, Nae, readerboy7
Page 3 of 6
Loading...
Similar Threads
  1. Lord Ravenclaw

    Second Annual DLP Fanfiction Awards of 2007

    Lord Ravenclaw, Jan 7, 2007, in forum: Announcements
    Replies:
    24
    Views:
    12,046
    Erotic Adventures of S
    Jan 29, 2007