More Server Crashes

Unread post by **CitroJim** » 09 Jan 2010, 17:51

I'm aware that the forum server is now throwing it's toys with a bit of worrying regularity.

I'm busy investigating why. It looks like a memory leak in IIS7 stemming from a known issue between it and PHP.

As a temporary "fix" I'll be setting the IIS worker threads to recycle on a regular basis to clear down the memory they are holding.

I have made some other configuration changes to IIS and I'm now monitoring memory usage and event logs.

Grateful if all apparent crashes can be reported here just in case I miss any.

spider · Unread post by **spider** » 10 Jan 2010, 18:27

It was down for me about 6pm GMT for a short time. Just getting the usual index.php download after it timed out.

Anyway about 5 mins ago, there was a problem with the sessions table (duplicate key) although that appears to have settled down now, at least that's the best one to go if any are going to act up.

[off topic]Am surprised to read you are using IIS though rather than using a 'nix box with apache

[/off topic]

Unread post by **CitroJim** » 10 Jan 2010, 18:37

Thanks Spider,

I saw it go down

Webfusion had a report of connectivity issues but this was a crash in the good old style. Strangely, nothing in the NT Event logs as there never is

It came back up of it's own accord but the phpbb_sessions tables on both the FCF and BXC were corrupt. They repaired OK.

I'm a dyed in the wool Windows person and know diddly-all about 'nix in comparason

We were on a W2K3/IIS6 platform until not so long ago and that was relatively stable, now on a W2008/IIs7 platform and I don't like it. Vista for servers. need I say more

About as stable as quicksand at present.

I was watching the IIS memory leak build. I need to recycle the IIS application pools for the forums a bit more frequently to see if that helps.

If this continues, I may try a 'nix box....

HDI · Unread post by **HDI** » 10 Jan 2010, 18:49

Yeah , I've had a couple of google , "ooops , this link appears to be broken" events recently.

Unread post by **CitroJim** » 10 Jan 2010, 18:59

Yes, I'm sorry about all this. I shall be having carefully considered words with the hosting company in the not too distant.

This level of service is not good enough...

spider · Unread post by **spider** » 12 Jan 2010, 13:33

As requested:

Was down (usual timeout) about 12.30 for a while

Came back up (I got to view a forum) a short time after that.

Session table crashed (now fixed I see)

I don't want to stick my nose in (but having seen this happen before on other software as the session table is the most used, given how it works) , you could probably change it to memory (assuming mysql) which would stop that in most cases, server load may go up though...

/ Random thought, mainly just posting the times for you as requested anyway...

Unread post by **CitroJim** » 12 Jan 2010, 13:40

Thanks Spider

I've raised a support ticket with the hosting company to look into this issue as it should not happen as frequently as it does. Luckily they're taking it seriously...

I've just repaired the sessions table for the nth time

I can do it blindfold now from the mysql command pronpt. Biggest problem was finding an internet machine where I presently am that would let me do an RDP session into the server....

Please bear with me on this, hopefully a proper resolution very soon. I have (to keep the) faith...

spider · Unread post by **spider** » 12 Jan 2010, 13:45

Yes, understand the problems

Changing it to memory (just that table!) will stop it crashing, obviously the contents of it would be lost on a restart, not the structure but as it does not contain anything critical as far as I know etc. Not used phpbb for ages. In memory it can't crash (much!) as its not having to keep mysql up with the constant disc writes. Only problem is memory usage goes up

But I agree, should see what the host says really. I'd personally expect that to only crash about once a year to be honest. If its any help, mines crashed once in 3 years but I have been lucky...

Unread post by **KennyW** » 13 Jan 2010, 21:17

Jim,

The program is throwing another wobbly for me at thuis time and not allowing me access to the other parts of the site.
Will log off and log back on again

Kenny

spider · Unread post by **spider** » 13 Jan 2010, 21:21

Couple of issues today, once the members table was crashed (earlier today)

Couple more timeouts during the day, although I'll be honest I've been that busy I did not take note of the times...

EDIT... I'm getting odd intermittant cannot modify header errors along with warnings about unsaved queries, I think your tables are in need of a repair

, error specifies table "u"

Unread post by **CitroJim** » 13 Jan 2010, 21:29

I'm doing my best to sort it. Host being no help at all. Getting very frustrating....

Please bear with me...

Stress levels are building somewhat...

Xaccers · Unread post by **Xaccers** » 13 Jan 2010, 21:37

CitroJim wrote:I'm doing my best to sort it. Host being no help at all. Getting very frustrating....

Please bear with me...

Stress levels are building somewhat...

Jim I hope you've got a cup of tea next to you.
There is no rush to get things up and running, we know you work your socks off repairing the forum when it goes down, we can wait as long as it takes.
Deep breath, count to ten.

I'm currently fighting with a sharepoint install at home so you have my sympathy.

addo · Unread post by **addo** » 14 Jan 2010, 03:13

I don't find these crashlets a problem. What we still get, is far ahead of resources available to likeminded people fifteen years ago. It's the efforts in maintaining an overall good website, that I appreciate.

Xaccers · Unread post by **Xaccers** » 14 Jan 2010, 09:03

Jim, when a company I hosted a forum with decided to let it's service plummet, I used sitescope to record whenever the forum went down, and used that to show them I knew very well how bad their server was.
They moved me to a different server and gave me a month free hosting.
It's a handy tool anyway.
Hmm I might check it out again myself, I know it can run through web sites automatically, but I can't remember off the top of my head if it could be used to run a script (to repair the tables) if certain text strings appear on the website.

Unread post by **CitroJim** » 14 Jan 2010, 15:50

Xac wrote:Jim, when a company I hosted a forum with decided to let it's service plummet, I used sitescope to record whenever the forum went down, and used that to show them I knew very well how bad their server was.

Thanks Xac

I've been doing same by hand and never fear I've not told the host! I'll have a look at sitescope as it looks promising.

I've also told them in no uncertain terms that their level of service is not acceptable and that if the server remains unstable. I want a new one. End of.

I have a support ticket open and last evening I asked for an update. fair enough. The 1st line desk informed me that yes, the ticket had been looked at by the sysadmins but they'd not updated the progress log so there was no update to give

Now, I work on a help desk, I action support tickets. If I had handled a ticket and NOT updated the progress log I'd be into big trouble. Makes me wonder just what sort of operation they run there. I did thell them my thoughts.

Strangely, and maybe not co-incidentially, the server has now been stable for 18 hours...

Again, I apologise for the crashlets (lovely term Adam!) and my absence from here whilst I have been investigating every angle. On the plus-side, my knowledge of W2008 Server is now very much enhanced