1. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  2. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

Scryer - DLP's FF Search Engine

Discussion in 'General Discussion' started by Lord Ravenclaw, Mar 31, 2016.

  1. Nuit

    Nuit Dark Lord

    Joined:
    Feb 14, 2010
    Messages:
    1,926
    Location:
    The Peach State
    I'd like to request Warcraft be added.
     
  2. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,353
    Location:
    Denver, CO
    Added. Check back in an hour or two and it should be indexed.
     
  3. Snarf

    Snarf Squanchin' Party Bro! ~ Prestige ~ DLP Supporter

    Joined:
    Apr 27, 2007
    Messages:
    15
    Gender:
    Male
    Location:
    Forty-Six & 2
    High Score:
    1,832
    All searches are no longer populating new content from about 16 hours ago.
     
  4. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,353
    Location:
    Denver, CO
    Every night, 1/30th of all indexed fandoms are selected for complete reindexing. When HP is reindexed, it tends to take a while at the slow pace I allow the crawler to make requests. That completed about an hour after your post and the hourly delta fetches resumed.
     
  5. dmacx

    dmacx Groundskeeper

    Joined:
    Jun 29, 2008
    Messages:
    374
    Is there a way to exclude certain fandoms when searching for crossovers?
     
  6. Zombie

    Zombie Black Philip Moderator DLP Supporter

    Joined:
    Apr 28, 2007
    Messages:
    5,999
    I thought you had to specify the crossover fandom A/B so how is the fandom you want to be excluded being included?
     
  7. dmacx

    dmacx Groundskeeper

    Joined:
    Jun 29, 2008
    Messages:
    374
    The way it is now, let's say I select Mass Effect on the left side. On the right, I can select individual fandoms to include, or (all fandoms). What I'd like to be able to do is specify multiple fandoms on the right side that I don't want. I'm not seeing a way to do that.
     
  8. Zombie

    Zombie Black Philip Moderator DLP Supporter

    Joined:
    Apr 28, 2007
    Messages:
    5,999
    Wouldn't they be excluded if they're not included? I'll let Raven weigh in more on that, but that's how I interpreted that function to work.
     
  9. lopeck

    lopeck Seventh Year

    Joined:
    Oct 22, 2013
    Messages:
    240
    Gender:
    Male
    Location:
    Germany
    Yes, but if you want to see all crossovers except a with a specific few fandoms you are currently screwed.
    If you, for example, want to exclude Naruto and Avengers from the crossover list, you can't just toggle exclude and add those two. You have to manually add all others. In the case of Harry Potter that would be over 800.

    Obviously most other fandoms have a far smaller list of crossovers, but the option to exclude would still remove a lot of hassle in such edge cases.
     
  10. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,353
    Location:
    Denver, CO
    Adding an "exclude" toggle wouldn't be hard, I'll see what I can do.
     
  11. dmacx

    dmacx Groundskeeper

    Joined:
    Jun 29, 2008
    Messages:
    374
    You said it far better than I did.

    Thank you. This would make an already good tool even better.
     
  12. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,353
    Location:
    Denver, CO
    I haven't forgotten the request above, I have some work in progress there. :)

    I did want to announce that I've made a dark mode for Scryer. It's not 100%, and I'll be tweaking it as I go, but it's now available. Just click the icon next to "Queue" to enable.

    Screen Shot 2020-04-14 at 21.16.28.png
     
  13. muthulx

    muthulx Muggle

    Joined:
    Oct 27, 2018
    Messages:
    3
    Location:
    United States
    Would it be too difficult to add another Fandom? If not, Can "prince of tennis" be added?
    Thanks in advance!
     
  14. Silirt

    Silirt Headmaster DLP Supporter

    Joined:
    Sep 19, 2018
    Messages:
    1,191
    Gender:
    Male
    Location:
    Georgia
    I was about to make a joke but fuck, looks like you've got something there. What's weird is it appears in the crossover fuzzy search.
    upload_2020-4-17_1-18-39.png
     
  15. muthulx

    muthulx Muggle

    Joined:
    Oct 27, 2018
    Messages:
    3
    Location:
    United States
    I usually make do with this Firefox Addon, it adds some filtering on top of the archaic ffnet's filtering.
    https://addons.mozilla.org/en-US/firefox/addon/fanfiction-tools/
    It's pretty good for many purposes like avoiding certain keywords but prince of tennis fandom is pretty huge, and scryer is way more advanced than all of these tools.. I did notice this fandom in crossover search.. Could it be because of HP fandom indexing? ..as prince of tennis's most popular crossovers are Harry Potter and Inuyasha.
    (It's just sad there are very few ffnet scraper tools out there when AO3 has so many :
    https://github.com/search?q=ao3
    as an aside as a very basic python programmer , I find many of this very interesting)
     
  16. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,353
    Location:
    Denver, CO
    Scryer indexes a certain list of fandoms, when it does it indexes all of that fandom's crossovers with at least 3 stories. So you'll see crossover fandoms which aren't actually indexed themselves. I consider those a part of the "source fandom".

    I've added "Prince of Tennis" to the list of indexed "source" fandoms. Depending on whether I managed to get the database update in before the indexer starts, it'll show up in an hour or two.
     
  17. muthulx

    muthulx Muggle

    Joined:
    Oct 27, 2018
    Messages:
    3
    Location:
    United States
    Thanks a lot!
     
  18. Lord Ravenclaw

    Lord Ravenclaw DLP Overlord Admin DLP Supporter

    Joined:
    Apr 2, 2005
    Messages:
    4,353
    Location:
    Denver, CO
    It's done, though honestly there's some kind of nasty edge cases around character loading that I need to work out. Loading all characters across all fandoms is apparently (for Rails) fairly expensive, and Chosen (frontend UI) is also poorly optimized for long lists of items. It comes grinding to a halt after every crossover addition for big fandoms.

    I have a good way to improve that situation, but until then it'll be a little annoying to use. There also might be bugs -- if you find any, let me know. :)

    Capture2.PNG
     
Loading...