Enquirer Home Page | Twitter | Back to Improbable Island

 Forum Index > Season Two > Development New Topic Post Reply
 Commentary expiration and server performance
 |  Printable Version
CavemanJoe
 Sunday, January 31 2010 @ 07:22 PM UTC (Read 5378 times)  
Forum Admin
Admin

Status: offline

Registered: 02/24/08
Posts: 2281

There are 575 pages of commentary in Common Ground. Common Ground is among the most heavily-accessed pages on the site, and the query that grabs the commentary is the biggest drain.

Given that we have the HTML export function for interesting scenes, would anybody object if commentary over 100 pages were deleted? The speed increase would be noticeable. Right now we're peaking at about 25 pages served per second.

Thoughts?


 
Profile Email Website
Quote
Chimental
 Sunday, January 31 2010 @ 07:27 PM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 06/30/09
Posts: 371

No objections here. (And apparently no witty remark from me either)


I make the many models of a mutant individual. To make them I use vegetables, animals, and minerals. From robot bugs to zombie bears to many singing barnacles.
 
Profile Email
Quote
FunnyMan
 Sunday, January 31 2010 @ 07:29 PM UTC  
Forum Contestant
Contestant

Status: offline

Registered: 12/03/09
Posts: 29

Makes sense to me. I never go back more then twenty or thirty pages.

Edit: Also, slightly offtopic, but did you catch my email a week and a half or so ago? 25/s seems like way less than a decent server should be capable of, but finding the bottlenecks mostly-blind is much harder.


 
Profile Email
Quote
CavemanJoe
 Sunday, January 31 2010 @ 08:33 PM UTC  
Forum Admin
Admin

Status: offline

Registered: 02/24/08
Posts: 2281

@FunnyMan, yes, got your mail, still mulling it over. Sorry to keep you waiting, you'll see why.

(also we're not going entirely blind - the test server is kitted out with XDebug/KCacheGrind and a debug report on the live server gives us the times of each individual query and modulehook (although it only displays the info to the admin, rather than storing it - that'd be pretty server-intensive). It's not terribly comprehensive, but it's better than what you get with standard LotGD (IE nowt)) Mr. Green


 
Profile Email Website
Quote
FunnyMan
 Sunday, January 31 2010 @ 08:45 PM UTC  
Forum Contestant
Contestant

Status: offline

Registered: 12/03/09
Posts: 29

Actually, I was referring to myself. I'm sitting here prodding II Labs' comment retrieval query(/-ies), trying to figure out why it would be a limiting factor. I've got a 60K row dummy commentary table, and right now I can run the main query 100 times in 0.37s, which seems plenty fast for 25 pages/sec.

Probably means my dummy table needs work to simulate II's actual usage.


 
Profile Email
Quote
CavemanJoe
 Sunday, January 31 2010 @ 09:12 PM UTC  
Forum Admin
Admin

Status: offline

Registered: 02/24/08
Posts: 2281

Remember about datacaching.

The biggest lookups are the initial author lookup just after a new comment is posted, and the clan info, both of which are (off the top of my head, without checking) cached. More distinct authors, longer query time.


 
Profile Email Website
Quote
Zolotisty
 Sunday, January 31 2010 @ 10:09 PM UTC  
Forum Moderator
Moderator

Status: offline

Registered: 07/25/08
Posts: 570

No complaints here.

The CG is, uh. Hm.

Well, I can tell you that no one reads back through those archives.


BARK BARK BARK.
 
Profile Email
Quote
SicPuess
 Sunday, January 31 2010 @ 10:32 PM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 08/10/08
Posts: 327

Ditto.


 
Profile Email
Quote
Bakemaster
 Monday, February 01 2010 @ 12:32 AM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 12/08/09
Posts: 226

Only "over 100"? As in, between 100 and 200?

Because hey, the practical difference between reading back through 575 pages and reading back through 376 pages is... well, I don't think there really is one.

As far as I'm concerned, you could go ahead and permanently truncate the CG at 5 pages, and I wouldn't miss a thing. Of course, plenty of people spend more time in the CG than I do; still, I can't imagine anyone having a really pressing need to look back more than 50 pages, at the outside.


Unofficial Improbable Island out-of-character chat is at irc.foonetic.net, channel #iisland - come on by!
 
Profile Email Website
Quote
Stanlygirl
 Monday, February 01 2010 @ 12:54 AM UTC  
Forum Contestant
Contestant

Status: offline

Registered: 06/23/09
Posts: 30

Sounds good to me. 100 pages is plenty.


 
Profile Email
Quote
CavemanJoe
 Monday, February 01 2010 @ 01:22 AM UTC  
Forum Admin
Admin

Status: offline

Registered: 02/24/08
Posts: 2281

OK, looks like nobody's going to grumble if we wipe 400-odd pages of commentary. Now to figure out an unobtrusive place to put the delete instruction - there's already a lot going on at system-newday, and obviously erasing one comment every time one more is posted isn't going to help things much.

However. Note that the current commentary expiration is set to seven days. When I proposed a two-day commentary timeout, folks really didn't like that idea. Right now there's 559 pages of commentary in Common Ground (weekends are quiet) - 559/7 = just under 80 pages of commentary posted on Common Ground every day. This will have the effect of a just-over-one-day commentary timeout in CG.

Are we still sure? I don't want you guys to think I'm using crafty sleight-of-hand to push the two-day timeout thing I suggested a few months ago. Wink


 
Profile Email Website
Quote
SicPuess
 Monday, February 01 2010 @ 01:59 AM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 08/10/08
Posts: 327

Sure! And in said discussion nobody would've minded a two-day limit on the CG - but very much everywhere else.


 
Profile Email
Quote
Zolotisty
 Monday, February 01 2010 @ 02:18 AM UTC  
Forum Moderator
Moderator

Status: offline

Registered: 07/25/08
Posts: 570

Quote by: SicPuess

Sure! And in said discussion nobody would've minded a two-day limit on the CG - but very much everywhere else.



Yup, this exactly. Scenes worth saving don't happen in the CG? The CG is just a constant flow of stuff, which is read as it occurs, and then is not read again.

People will rear up spitting (like last time) if commentary elsewhere takes a hit.


BARK BARK BARK.
 
Profile Email
Quote
FunnyMan
 Monday, February 01 2010 @ 03:01 AM UTC  
Forum Contestant
Contestant

Status: offline

Registered: 12/03/09
Posts: 29

I think the best solution would be if you could finagle it so that the timeout went away and every comment area just kept the last N pages of commentary. In something like my clan chat, where the volume is really low, a 7-day timeout is really annoying, because the conversations weren't very long ago relative to the pace of discussion. In common grounds, anything over a few hours is positively ancient.


 
Profile Email
Quote
Bakemaster
 Monday, February 01 2010 @ 04:29 AM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 12/08/09
Posts: 226

So if I understand correctly, there's currently one commentary timeout that handles all commentary areas, and it refers to a constant that's set to 168 hours/7 days.

Can you throw in a switch where the constant is read, that runs through cases associated with the database key identifying the location where the commentary line is displayed, in order to return different constants for comments in different locations? All you'd need immediately is a case for the CG and a default to handle everything else, but this way if need be you could add a case for clan chats without any extra work, or for natters, or what have you.

That's assuming rather a lot about your database structure, though. So feel free to ignore me, I'm just avoiding working on this damn paper...

(For the record though, I would also really dislike a "global" timeout of 24-48 hours... Particularly as a clan leader, I feel I need to be able to see what people have been saying in the clan chatspace even if I don't get the opportunity to log in for the better part of a week.)


Unofficial Improbable Island out-of-character chat is at irc.foonetic.net, channel #iisland - come on by!
 
Profile Email Website
Quote
NotAgain
 Monday, February 01 2010 @ 08:07 AM UTC  
Forum Contender
Contender

Status: offline

Registered: 10/27/09
Posts: 35

Speaking as someone who does spend a bit of time in the CG, even a hard limit of 24 hours would be plenty. Sad, but true.

The CG's not totally... stream of consciousness and there are the occasional small gems of RP, which I'll go hunting back for. However, with the export button it's now easy enough to archive 'em as they occur... or within the day, anyway.

So, I'm another vote for "go for it." ...so long as it doesn't affect other commentary areas.


 
Profile Email
Quote
K.K. Victoria
 Monday, February 01 2010 @ 12:40 PM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 12/17/08
Posts: 498

Yeah. As I recall no one really cared if the Common Ground limit was cut down, they only cared if the limits everywhere else was.

(In fact, I think I suggested just lowering the Common Ground archive and seeing how that worked, but yeah.)

No arguments here. I don't think I even set foot in the Common Ground anymore...


"You saved Pineapple!"
 
Profile Email
Quote
Ada
 Tuesday, February 02 2010 @ 03:29 AM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 12/07/09
Posts: 428

I see something has been done! I'm curious about what exactly causes the pages to be cut off - you said not at newday, so when does it happen?


 
Profile Email
Quote
CavemanJoe
 Tuesday, February 02 2010 @ 09:33 PM UTC  
Forum Admin
Admin

Status: offline

Registered: 02/24/08
Posts: 2281

It ended up going on newday-runonce anyway. Razz


 
Profile Email Website
Quote
Ada
 Thursday, March 11 2010 @ 03:14 PM UTC  
Forum Improbable Badass
Improbable Badass

Status: offline

Registered: 12/07/09
Posts: 428

I'm confused! I thought this discussion was just about the CG, but I just had a look at NewHome, and the oldest comment there is from just over 2 days ago.

Did the new rule get applied to NewHome, too?


 
Profile Email
Quote
Content generated in: 1.66 seconds
New Topic Post Reply



 All times are UTC. The time is now 12:52 AM.

Normal Topic Normal Topic
Locked Topic Locked Topic
Sticky Topic Sticky Topic
New Post New Post
Sticky Topic W/ New Post Sticky Topic W/ New Post
Locked Topic W/ New Post Locked Topic W/ New Post
View Anonymous Posts 
Anonymous users can post 
Filtered HTML Allowed 
Censored Content