Server downtime

log in

Advanced search

Message boards : News : Server downtime

Author Message
Profile TJM
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 25 Aug 07
Posts: 716
Credit: 21,425,526
RAC: 9,183

Message 1532 - Posted: 9 Mar 2010, 10:25:45 UTC
Last modified: 9 Mar 2010, 10:28:48 UTC

During this weekend the server will be taken down for O/S upgrade.
The old distro (Debian 4) will be replaced by Debian 5 amd64, since the older version has a limited support and probably slows down the hardware a bit.
It's hard to estimate how long the upgrade process will take, but I'd say that 8-12 hours minimum, because I have to do a complete backup first, so in the worst case I'll be able to rollback everything to the previous state.
____________
M4 Project homepage
M4 Project wiki

Tom_unoduetre
Send message
Joined: 18 Sep 07
Posts: 7
Credit: 144,908
RAC: 404

Message 1533 - Posted: 9 Mar 2010, 16:41:36 UTC

Good luck and hopefully all goes well.

thinking_goose
Send message
Joined: 12 Nov 07
Posts: 110
Credit: 1,090,928
RAC: 503

Message 1534 - Posted: 9 Mar 2010, 18:04:50 UTC

I hope everything goes well. I've upped the work unit cache to a whole day on a couple of machines that I expect to be running through the weekend.
____________

Profile TJM
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 25 Aug 07
Posts: 716
Credit: 21,425,526
RAC: 9,183

Message 1535 - Posted: 14 Mar 2010, 18:15:04 UTC
Last modified: 14 Mar 2010, 20:10:44 UTC

So far everything looks fine.
The upgrade process took longer than I expected, mainly due to lots of minor configuration differences between etch and lenny.
Also, I had to rebuild the server software, so I decided to upgrade all the way to the latest trunk. As a result, the server runs on revision 20902 which is the latest one and might be unstable.
So far only one bug has been found and it's just a minor glitch on the main site - missing character encoding.

EDIT: There was a bug in work generator and it crashed soon after I restarted the server. I didn't notice but someone left me a message at google chat - thanks.
____________
M4 Project homepage
M4 Project wiki

Profile TJM
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 25 Aug 07
Posts: 716
Credit: 21,425,526
RAC: 9,183

Message 1595 - Posted: 7 May 2010, 18:16:46 UTC
Last modified: 7 May 2010, 18:18:14 UTC

07.05.2010

The server is going down for database cleanup, extended backup, software upgrade and maintenance. Later today new ciphertext will be added to the server's queue.
____________
M4 Project homepage
M4 Project wiki

Profile TJM
Volunteer moderator
Project administrator
Project developer
Project scientist
Avatar
Send message
Joined: 25 Aug 07
Posts: 716
Credit: 21,425,526
RAC: 9,183

Message 1598 - Posted: 10 May 2010, 23:26:58 UTC

11.05.2010

The server is back up after a critical database failure.
I'm not sure what exactly happened, the mysql daemon just dissapeared and couldn't be restarted because one of the ib_logdata files got damaged.
I tried to avoid restoring the DB from backup, instead I used the realtime copy provided by replication slave. Unfortunately the slave is always few minutes behind the main server, so probably there was some (hopefully minor) data loss.

Right now everything seems to be running, but if you think that something is wrong with the database, please let me know immediately.
____________
M4 Project homepage
M4 Project wiki


Post to thread

Message boards : News : Server downtime


Return to Enigma@Home main page


Copyright © 2017 TJM