Site running in 'limp' mode following major failure

[xantia_v6]

On Sunday evening this site was moved to a new hardware node to fix some long term and short term performance problems.
On Monday night the server suffered a major disk corruption which (as far as we know) was due to some problem introduced when the server was moved (this was done by the hosting company, not me).

After spending some hours trying to recover the disk, it was deemed necessary to start again with a clean server installation, reverting to the database backup taken on Monday morning. This means that all posts made during Monday have been lost and are not coming back.

The forum is now running again, but some features, particularly image hosting, have not been enabled yet, because it takes many hours to upload the image files from the backup.

So you are free to start using the forum again, and we hope to have all the bells and whistles back by the end of tomorrow. Anything posted from now on will be kept in the database, we are not planning to lose anything more.

The BXC site and wiki will be restored tomorrow.

[Hell Razor5543]

I have noticed that the bookmark tab (up at the top, alongside the 'your posts, tec.' tabs) is missing.

[xantia_v6]

I have noticed that the bookmark tab (up at the top, alongside the 'your posts, tec.' tabs) is missing.

Thanks for pointing that out. I had missed it, but it should be part of the file restore that is pending on the upload of a 7GB backup file via an ADSL link...

[NewcastleFalcon]

Pleased FCF hasn't disappeared, the messages started with the mild connection is not secure,as previously posted up, then at one stage turned into frenchcarforum.co.uk...URL does not exist!

Hats off to you chaps who voluntarily sort this stuff out.

Maybe when such crises occur when from a user point of view the site just disappears... some reassurring message like you used to get on the Telly...."Normal Service will be resumed as soon as possible!" could be put up. If thats a problem with the website being inaccessible, maybe a message on a FCF twitter/FB platform which members could consult if all attempts to connect to the FCF itself fail.

Course it dosen't happen very often but you are left wondering whether its your own machinery/internet or a problem at the FCF end.

Regards Neil

[xantia_v6]

We did previously have a backup/test server that we could use to give a test-card message, but it was decommissioned a few months ago, as it was costing us money whether used or not.

I will try to implement something else for next time, but this is only the second time in 3 years that the site has gone down for more than a couple of hours, and the previous time was a software bug that also affected our backup site.

[Paul-R]

Oh dear, my sympathies. Obviously the forum doesn't like its new home!

[Zelandeth]

Sorry to hear you've had problems, glad to see we're nearly back on our feet though. Hats off to those involved.

Always the way isn't it with hardware like this...

I've only been peripherally involved in running anything like this once, and it was for something like ten years run on truly ancient freely obtained "life expired" ex-corporate hardware out of someone's basement (no, really!) without a hitch. Sure, we had the occasional hard drive physically die, but the RAID array and backup system always saw to that with no fuss.

Then we moved to all new, warrantied hardware and moved to a proper co-location facility. Was a complete and utter pain...we had power supplies fail, we had more hard discs fail than we could count, and the remote management console at one point developed a truly maddening habit of going unresponsive - necessitating a three hour round trip to the facility to figure out what the heck it was doing. Usually approximately two minutes before someone got there it would spring back to life again as though nothing happened.

I was barely even involved, but that server took years off my life I reckon! We were pretty convinced that the thing was quite simply possessed by the time we saw the back of it. ...or more likely it had some unspecified and hard to pin down intermittent motherboard fault. Though we never did tie that up with the mortality rate of the hard drives.

I seem to recall that for the last couple of years before the project was finally canned as the mainstream Amiga usergroups died away it returned to its ancient basement server and the barely used new one was sold on to some unsuspecting big company...wonder if their IT guys had as much "fun" with it as we did...

Fingers crossed this isn't setting the scene here!

[CitroJim]

Mike, a tremendous effort to get the forum this far in such a short time from 'bare metal'

Well done! We are most fortunate to have you at the technical helm

[Hell Razor5543]

There are still issues. I just experienced a spell where I was told I did not have permission to access anything on the site. I have also noticed that the 'like' function has alignment problems. For example, on THIS post I see the '0', 'X' and the heart next to each other in that sequence, while on the other posts I see the heart and the '0' next to each other, and the 'X' below them.

Oh, and before I appear to be complaining, thanks for getting the forum back up and running. I was getting withdrawal symptoms!

[xantia_v6]

I am aware that there are still some issues, as I did not have access to all of the backup files yesterday, the site was put on-line with a cobbled together mix of old debug files that happened to be on my PC.

I have now applied the full backups from Monday morning, so most things should be back in order, but there are still some configuration issues and some peripheral software issues to sort out.

To resolve the formatting issues, please press ctrl-f5 or clear your browser cache.

I think that all of the images and pictures should be back now. If no-one reports any missing, then I will re-enable attachment uploads later today.

[Hell Razor5543]

Erm, there may be issues with the pictures as well. The following link has 5 pictures, but the first and last are from different posts;

viewtopic.php?f=3&t=58722&p=552722#p552722

The bookmarks are back, so progress is being made.

[xantia_v6]

I thought that I had disabled attachment uploads, but apparently not, so the post mentioned above was made before the file restoration was complete and 2 of the attachments had the same name as restored files. I have deleted the attachments from that post, and will check for any other attachments added since Monday.

[xantia_v6]

The emails sent by the forum do not currently have signed headers, so some mail systems may consider them to be spam. I will correct this as soon as I can.

[Peter.N.]

Welcome back! and thanks for all your hard work.

Peter

[ragger]

Very big thanks to Mike and everyone else who works so hard to maintain this wonderful resource.