1. DLP Flash Christmas Competition + Writing Marathon 2024!

    Competition topic: Magical New Year!

    Marathon goal? Crank out words!

    Check the marathon thread or competition thread for details.

    Dismiss Notice
  2. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

Yet Another Fanfiction Recommendation Engine

Discussion in 'Fanfic Discussion' started by Dark Minion, Jul 22, 2015.

  1. Dark Minion

    Dark Minion Bright Henchman DLP Supporter Retired Staff

    Joined:
    Sep 22, 2006
    Messages:
    2,231
    Like JordanL did some years ago (here) I tried my hand at coding a fanfiction.net search engine that ranks stories based on how often they appear in user-favorites and communities. Not just in any favorites and communities, but specifically in those favs and communites that are most similar to your own profile.

    The basic idea actually isn't that new and Lord Ravenclaw did something similar for the DLP-library.

    Sadly that can't be done by just accessing the ff.net database. One has to manually copypaste every user profile into a spreadsheet and... well... yeah... you get the picture. So for the last three weeks I was kind of busy and couldn't care for the library...


    The engine can be found here.

    Two examples:
    Some Random Story Search
    Some Random Small Fandom Search


    It starts with you submitting either your fanfiction.net user-id or a story-id (or several) of stories you like. The engine will then look up which users and communities list the same stories and return the rest of their list - ranked by how many of these communites / users list it.

    Right now the result is based on 16,000 communities and 50,000 users. With about 100 stories per user-favorites the database has to search through about 5 million user-story relations and might be a bit slow.

    There still is some work to be done. I have to tweak the indices, especially concerning the text search options and I mangled the utf8 enconded text when moving the database - so don't bother to filter for some weird weeaboo characters.

    Nevertheless I proved for myself that the entire problem is solveable. For me as mathematician that means from now on the problem is boring and not worth further time. Thus I might never update nor enhance it.

    For now I'll sticky this thread. Perhaps it actually is of use for some people here.
     
    Last edited: Jul 22, 2015
  2. dmacx

    dmacx Groundskeeper

    Joined:
    Jun 29, 2008
    Messages:
    375
    I put in the story id for one of my favorite Worm fics.

    The results tell me that peoples' tastes are really, really varied :)

    Thanks for making the effort to do this.
     
  3. Nuit

    Nuit Dark Lord

    Joined:
    Feb 14, 2010
    Messages:
    1,934
    Location:
    The Peach State
    A suggestion, maybe something a little closer to this if possible.

    [​IMG]

    [​IMG]

    Code:
    <ol>
    <li> Users: 7, StoryId: <a href="ffsearch.php?storyid=4451412">4451412</a>, Category: Naruto<br>
    <br>
    <a href="https://www.fanfiction.net/s/4451412/1/" target="_blank">Houses of the Holy</a> by HowdyU<span class="hidden" style="display: inline; margin-left: 1em;"><br>
    <br>
    	Spirited away from Konoha at an early age, Naruto finds a new family amid the strife of a battlefield as old as the shinobi nations. But what will he do when his new allies’ ambition is to change the world through a baptism by fire? An Akatsuki!Naruto fic<br><br>Naruto - Rated: M - English - Adventure - Chapters: 12 - Words: 127,994 - Reviews: 1045 - Favs: 2,054 - Follows: 1,849 - Updated: 1/15/2009 - Published: 8/5/2008 - Naruto U., Yugito N.</span></li><br>
    
    <li> Users: 7, StoryId: <a href="ffsearch.php?storyid=4532323">4532323</a>, Category: Naruto<br>
    <br>
    <a href="https://www.fanfiction.net/s/4532323/1/" target="_blank">Failing Twilight</a> by Shadow Rebirth<span class="hidden" style="display: inline; margin-left: 1em;"><br>
    <br>
    	ABANDONED. Naruto had learnt from a young age that life could get hectic, but all this chaos with Biju, daimyos, and psychotic missing-nin was going a bit too far. Now just throw in a fourth Great Shinobi War and you've got one pissed off seal master. [AU, no pairings.]<br><br>Naruto - Rated: T - English - Adventure - Chapters: 14 - Words: 75,508 - Reviews: 1121 - Favs: 2,625 - Follows: 2,742 - Updated: 6/3/2010 - Published: 9/11/2008 - Naruto U.</span></li><br>
    </ol>
    Edit: Also, when did you become Dark Friend ?
     
    Last edited: Jul 23, 2015
  4. Dark Minion

    Dark Minion Bright Henchman DLP Supporter Retired Staff

    Joined:
    Sep 22, 2006
    Messages:
    2,231
    :(

    Jonicon violated my profile.

    About the formatting of the summary and stats - the way I grab them I can't format them differently. It's one huge block beginning with the title and ending with the 'complete'

    I never bothered (read: actually forgot) to finetune the grabber. For now I would need to pass it through some regular expressions first to clean that mess up. I might take a look into it next weekend.
     
Loading...