1. DLP Flash Christmas Competition + Writing Marathon 2024!

    Competition topic: Magical New Year!

    Marathon goal? Crank out words!

    Check the marathon thread or competition thread for details.

    Dismiss Notice
  2. Hi there, Guest

    Only registered users can really experience what DLP has to offer. Many forums are only accessible if you have an account. Why don't you register?
    Dismiss Notice
  3. Introducing for your Perusing Pleasure

    New Thread Thursday
    +
    Shit Post Sunday

    READ ME
    Dismiss Notice

Recommendation Engine for FF.net

Discussion in 'Fanfic Discussion' started by JordanL, Nov 24, 2010.

  1. b0b3rt

    b0b3rt Backtraced

    Joined:
    Sep 28, 2010
    Messages:
    252
    Lol, that suggestion is like a T(n) = xFavorites(n) recursive function - it would be insane. On the other hand, you would probably only have to do it a few times to get every single author who is favorited at all...
     
  2. Beonid

    Beonid Seventh Year DLP Supporter

    Joined:
    Jul 12, 2008
    Messages:
    201
    Location:
    Melbourne
    I'm getting so many stories around 10% that I don't think it'd make much of a difference.

    Maybe you could expand the filters available at the search page, instead?
     
  3. SmileOfTheKill

    SmileOfTheKill Magical Amber

    Joined:
    Mar 24, 2007
    Messages:
    1,219
    Location:
    Florida, Sigh...
    Some things to note. It should say Completed on the story list if the story is completed as that is something easy to have. Might also want a completed only query, but I don't know how that would be handled.

    Anyways, JordanL is a hero.
     
  4. silverlasso

    silverlasso Minister of Magic DLP Supporter

    Joined:
    Dec 7, 2007
    Messages:
    1,302
    Location:
    San Francisco
    If you add suport for an argument to the suggest users url (e.g. http://fanfictionrecs.net/suggest_user.php?userid=1239654), then people could just use a bookmarklet to easily submit profiles.

    The code for the bookmarklet would be something like this:

    Code:
    javascript: (function () {
        if (location.href.match(/fanfiction\.net\/u/)) {
            var id = document.getElementsByTagName("link")[0].href;
            id = id.match(/\d{1,10}/);
            location = 'http://fanfictionrecs.net/suggest_user.php?userid=' + id;
        } else if (location.href.match(/fanfictionrecs\.net/) && document.referrer.match(/fanfiction\.net\/u/)) {
            history.back();
        }
    })();
    
    If you're on a user page, the bookmarklet will load the submit page on fanfictionrecs with the URL argument, and you can click it again to get back to where you were.

    There's a version I have that would work right now if I could figure out how to use global variables with bookmarklets, but it would probably be easier to add the URL argument. Also, I really wish bookmarklets could access different domains; I understand why XSS is blocked by browsers, but it's a PITA, sometimes.
     
    Last edited: Nov 26, 2010
  5. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    It does support that formatting for the variable.

    If you get a bookmarklet working, I will certainly post it up on the page for people to use. :)

    I'm not sure if I grabbed the completed aspect of the stories... If it's not in the database I'd have to recrawl all 300,000 stories.

    I'll get around to it tomorrow. Next on the list is actually a bunch of formatting and usability stuff, then some more filters.

    ---------- Post automerged 11-26-2010 at 11:52 AM ---------- Previous post was 11-25-2010 at 11:28 PM ----------

    I implemented your bookmarklet silverlasso. :)

    And gave you credit of course.

    EDIT: Also, I put in the "jump to page" feature that was requested.
     
    Last edited: Nov 26, 2010
  6. Stenstyren

    Stenstyren Professor

    Joined:
    Jan 9, 2009
    Messages:
    465
    Would it be possible to have a blacklist? I think the ability to filter away unwanted/already read stories would make this tool much more useful since I could then simply get to the goddies without having to go through the stuff I've already read.

    Will be very intense on the server though, perhaps you could make it so that my own favorite list is my blacklist? So that you don't have to save the data on your servers.
     
  7. Trig

    Trig Unspeakable

    Joined:
    Jan 27, 2010
    Messages:
    708
    Location:
    Germany
    Or simply Like/Dislike buttons similar to YouTube, where people are able to rate their search results so that someone who does the same search later on could possibly get better results.

    Another benifit of this would be the ability to filter out or downrate stories like Methods of Rationality/An Aunt's Love etc. that everybody and their mums got on their favourite lists.
     
  8. Stenstyren

    Stenstyren Professor

    Joined:
    Jan 9, 2009
    Messages:
    465
    The problem with that would be the huge use of server place I guess. Although I agree, a thumb up/down next to the story name would be awesome.
     
  9. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    I went ahead and added the "filter by my favorites" feature.

    NOTE: Your profile must be submitted to the database for that feature to work.

    It will greatly help reduce the duplication of stories, but was by far the least server intensive way to accomplish it that had been suggested. Great idea, BTW.

    EDIT:

    As I noted on the index page of the site, ( http://fanfictionrecs.net ) I have no intention of ever charging anything to use the site, and right now I don't need any money for hosting either. But if this site experiences heavy usage, what would everyone's reception be to a donation jar?

    This is a hobby for me, as I noted, and I did this, essentially, over a single weekend because I felt like it. But if in the future I need a dedicated server... I don't know that I'd do it to be honest. Not without some financial help. I mean I could do it... I make enough at my day job to support that... but it would be the difference between me getting a bus pass once a month or not.

    Anyway, we're a ways off from that actually becoming an issue, just thought I'd feel people out.
     
    Last edited: Nov 26, 2010
  10. Stenstyren

    Stenstyren Professor

    Joined:
    Jan 9, 2009
    Messages:
    465
    It's easy enough for me to say that I would donate, and i probably would for a few months.

    The problem is that I'm a poor student (I have like 40$ to spend on fun stuff every month:p) so my donations would probably be sporadic.

    That's usually the way it goes, first month you will get lot's of money but then people will say to themselves "Well, I've already donated X to this, it's someone else problem now".

    For example, the entire DLP website costs $214 a month to run and that amount is not covered every month (but some months too much is paid so I guess it evens out). It would be hard for you to get the exposure that the DLP donate bar gets so I think it would be tough.


    Protip: Give authors the option to pay $100 for their stories to get better rankings :awesome
     
  11. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    Naw, wouldn't want to taint the results. :p

    I'm sure I'd figure something out if it didn't work out. But like I said, that's a ways off yet.
     
  12. Trig

    Trig Unspeakable

    Joined:
    Jan 27, 2010
    Messages:
    708
    Location:
    Germany
    What about advertisements that net you a very small bit of money for every pagevisit? They wouldn't be a problem at all considering most people use adblock and/or noscript, but you would still profit.

    I don't know if those even exist anymore or how easily accessable they are but this was the first thing that came to mind.
     
  13. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    No, internet users have made it very clear: they don't want to be advertised to.

    If it comes down to it, and I can't afford to keep the site online, people will pay for it via donation or it will be taken down. AdBlock and similar software ensures that the people who use the site will end up paying for it.

    Dealing with advertisers when people simply block the ads is just a giant headache for me with little or no gain at all.

    EDIT: (And that's my opinion from professional experience...)
     
    Last edited: Nov 26, 2010
  14. b0b3rt

    b0b3rt Backtraced

    Joined:
    Sep 28, 2010
    Messages:
    252
    I'm not sure if you'd be able to find enough people to donate $100/month for dedicated hosting, but this may be a possibility if your user base grows more. I'm sure DLP users alone could pitch in $25-$50/month easily.
     
  15. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    Well, the idea is that if I needed dedicated hosting, the userbase should be large enough. :)

    It's good to know a few would already be willing if it were necessary though.

    I've actually been slacking off on programming some of the new features because I've been reading stories as I've been testing it. :eek:
     
  16. Sesc

    Sesc Slytherin at Heart Moderator

    Joined:
    Dec 20, 2007
    Messages:
    6,216
    Gender:
    Male
    Location:
    Blocksberg, Germany
    The remove all own favourites is a good idea, but are you sure it's working? I submitted one of my stories, and my user-id, and I still got a few stories that were on my favourite list. The same if I put in the entire profile url. Am I doing something wrong?
     
  17. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    It was tested and working for me... except for a single story that I added to my favorites since the first time I submitted myself to the database.

    The rolling update hasn't gotten to me yet.

    If that's your problem as well, you can force the DB to update by resubmitting your UID using the add form. If you do, it should say "Updated & Processed".

    EDIT:

    Side question: do all these thumbs up represent cool points I get to spend somewhere? :p
     
    Last edited: Nov 27, 2010
  18. b0b3rt

    b0b3rt Backtraced

    Joined:
    Sep 28, 2010
    Messages:
    252
    I hadn't been finding anything using this tool, but after seeing Delenda Est so high so many times, I finally gave in and read it... and was pleasantly surprised. So the tool's already worth my time =D
     
  19. Sesc

    Sesc Slytherin at Heart Moderator

    Joined:
    Dec 20, 2007
    Messages:
    6,216
    Gender:
    Male
    Location:
    Blocksberg, Germany
    @Jordan:
    I found the error. Somehow, the database didn't have my profile, although I had submitted it the day before yesterday. I re-submitted it, and now it's working. Cool 8D

    What format does the profile url have to be in? Does it matter whether it's
    Code:
    http://www.fanfiction.net/u/1232425/
    or
    http://www.fanfiction.net/u/1232425/SeriousScribble
    ? I used the former this time, and the latter the first time.

    And naw, the thumbs up only work as e-peen size, nothing else really. But for that, they're good enough :p


    Edit: Rofl. Now I get some stories I remember reading and forgetting to add to my favourites. Your tool helps me organise my shit on top of everything, Jordan :D
     
    Last edited: Nov 27, 2010
  20. JordanL

    JordanL Third Year

    Joined:
    Jan 17, 2009
    Messages:
    98
    Either format works the same. Your profile must have been one of the 30 that were accidentally lost when my updater script went rogue for three hours without me noticing. I tried to reinsert all of the profiles but must have missed yours.

    EDIT:

    Personally, it's actually helped me get more in tune with various fandoms. I hadn't read Indomitable or An Aunt's Love because I honestly hadn't gone looking. But it came up in three different searches so I gave both a try and were pleasantly surprised.

    I also was very pleased to find that two of the top 10 A:TLA stories by favorites had been on my favorites for a long time. Second Nature and Ashes to Ashes are both fantastic stories.

    ---------- Post automerged at 05:27 AM ---------- Previous post was at 01:25 AM ----------

    Shit... I'm starting to get intermittent SQL errors because of too many connections, which means too much load on the servers...

    Time to solve THAT one...

    ---------- Post automerged at 06:15 PM ---------- Previous post was at 05:27 AM ----------

    Well, it's become popular enough that I've had to disable to stats page in order to keep the database load lower.

    ---------- Post automerged 11-28-2010 at 02:46 PM ---------- Previous post was 11-27-2010 at 06:15 PM ----------

    Well... this is fairly incredible.

    It's pushing almost 30,000 hits per day on its seventh day. I just started experiencing MySQL connection errors, which means it's starting to hit the hard limit on users it can serve...

    So I actually do have to move to dedicated hosting. Oh well.

    I threw up a donation button and went ahead and put in one ad anyway, even though I don't think that'll help much.

    In the meantime, lets see if I can get my host to let me change some MySQL settings to improve performance...
     
    Last edited: Nov 27, 2010
Loading...