|
|
| Author |
Thread Statistics | Show CCP posts - 11 post(s) |
|
CCP Red Button
   |
Posted - 2010.12.13 22:04:00 -
[1]
Thought I'd post some pictures of StuffÖ that happened today for those interrested.
Burned bladecenter
Burned blade 1
burned blade 2
As you can see we had a decent bonfire burning in our datacenter today. While not the primary reason for the delayed patch tomorrow this was definitely a pretty big contributing factor. This morning we had an alarm triggered on a powersupply in one of the bladecenters containing the Singularity testcluster. When reseating the powersupply which is part of a routine troubleshooting procedure before replacing with a spare part it somehow mysteriously, even magically (cause yet undetermined, and this is the first time we have this happen) managed to cause an internal short circuit in the bladecenter, completely fry one of the blades and seriously singe a few of the others. As you can imagine this caused us to immediately pull the bladecenter out of service and thus shutting down our primary public test server while we were juggling hamsters between cages to make StuffÖ work again.
Having Singularity, which is our primary staging server, shut down for several hours on the day before a big deployment is far from ideal, in fact it is critical as we use every hour right up until the last minute to test, retest and test again that everything is working. So add this to other issues that we were dealing with today and the prudent thing to do was to delay the patch and have a second dress rehearsal tomorrow, this time hopefully without incident.
Oh and by the way, the hamsters are still alive if somewhat visibly shaken.
------
Hmmm... wonder where this cable goes 0_o |
|
|
CCP Zulu
|
Posted - 2010.12.13 22:07:00 -
[2]
 Originally by: Crumplecorn
Someone needs to train Thermodynamics higher.
Ooh, is it bad pun night? Can I try?
"Did anyone order the fried hamster?"
|
|
|
CCP Wrangler
|
Posted - 2010.12.13 22:09:00 -
[3]
Let me guess, we're going to have a thread filled with bad jokes.
Wrangler
Senior Community Manager
CCP Hf, EVE Online
Contact us
If it's stupid but works, it isn't stupid. |
|
|
CCP Wrangler
|
Posted - 2010.12.13 22:16:00 -
[4]
Originally by: Abdiel Kavash
CCP, prepare to get flamed for this.
I think we already did... 
Wrangler
Senior Community Manager
CCP Hf, EVE Online
Contact us
If it's stupid but works, it isn't stupid. |
|
|
CCP Zulu
|
Posted - 2010.12.13 22:42:00 -
[5]
If you don't stop with the bad puns I fear the forum blades will set themselves on fire in protest.
|
|
|
CCP Red Button
|
Posted - 2010.12.13 22:47:00 -
[6]
Only when I'm hard pressed ;)
------
Hmmm... wonder where this cable goes 0_o |
|
|
CCP Explorer
|
Posted - 2010.12.13 23:05:00 -
[7]
Originally by: Konoch
quote=SlayerOfArgus]TTTUUUUXXXXFFFOOOORRRRRDDDDDDDDD!!!!!!!!!!!!!!
WHY DID YOU HAVE TO TOUCH THE RED BUTTON!? WHYYYYYYYYYYYYY?!
Indeed very inappropriate...
Erlendur S. Thorsteinsson
Software Director
EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.13 23:06:00 -
[8]
Originally by: Garreth Vlox
was thinking of putting a day long skill in the extended DT que now im thinking more like a month long skill.
What could possibly go (more) wrong?
Erlendur S. Thorsteinsson
Software Director
EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.13 23:28:00 -
[9]
Originally by: Brennivargur
Originally by: Sporked
Originally by: Maniakas
Well thats hot news.
To bad they delayed new patch 24 hours.. but a good reason.
seems not all places are cold there in Iceland 
Good luck with it 
Iceland had nothing to do with it, the servers are in London.
London's burning, London's burning....
http://www.geoiptool.com/en/?IP=87.237.38.50 
SiSi (and that IP) is nevertheless in London.
Erlendur S. Thorsteinsson
Software Director
EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.13 23:33:00 -
[10]
Originally by: Akita T
Originally by: Brennivargur
Originally by: Sporked
Originally by: Maniakas
Well thats hot news. To bad they delayed new patch 24 hours.. but a good reason. seems not all places are cold there in Iceland Good luck with it
Iceland had nothing to do with it, the servers are in London. London's burning, London's burning....
http://www.geoiptool.com/en/?IP=87.237.38.50
The physical location of the server cluster _IS_ in London. As far as we know. They never claimed they moved it to Iceland. Just because the "main IP" for EVE in general is listed as belonging in Iceland doesn't mean that's where the server is. Heck, geoip databases hardly ever claim anywhere near 100% accuracy anyway.
P.S. At least TQ is in London, but I assume that also goes for SiSi and Multiplicity. Although that's not guaranteed. SiSi and Multiplicity might be physically located in Iceland.
Nearly all our test servers are located in London, including in particular Singularity and Multiplicity.
Erlendur S. Thorsteinsson
Software Director
EVE Online, CCP Games |
|
|
|
|
CCP Xhagen
|
Posted - 2010.12.14 10:18:00 -
[11]
Originally by: Konoch
Originally by: SlayerOfArgus
TTTUUUUXXXXFFFOOOORRRRRDDDDDDDDD!!!!!!!!!!!!!!
WHY DID YOU HAVE TO TOUCH THE RED BUTTON!? WHYYYYYYYYYYYYY?!
I couldn't help it. The Blade witnessed it and the pics show the results.
____________________________
CSM Project Manager
EVE Online
CCP Games |
|
|
|
| |
|