|
Author |
Thread Statistics | Show CCP posts - 23 post(s) |
|
CCP Wrangler
|
Posted - 2007.09.15 19:49:00 -
[1]
This Monday it was brought to our attention that an alliance couldn't get Sovereignty 4 in their capital system even though they should. Looking into it we concluded that their capital station was a conquerable station which shouldn't be possible to set as a capital station. Fixing this required changing procedures in the database so attempting to set a conquerable station as capital station causes an error.
Unfortunately, due to unforeseen behavior in the server code a certain argument was passed to the procedure, which did not have anticipated default value, causing the procedure to error on every call from conquerable stations that were set as capital stations, of which there are 10 of on Tranquility.
While hotfixing this we failed to notice that the error flag code we copied was bad, that error condition can possibly have caused other crashes, in the sense that it did not close the transaction. This in turn led to that every call to this procedure would lock up a session to the database as it waited indefinitely for the transaction to be either rolled back or closed. The sessions locking up created a plethora of different locks all over the database, causing us to chase for a ghost of the problem. Eventually this led to a database fail-over which meant Tranquility crashed.
This oversight has been located in two other procedures as well, but none of them are called often enough on Tranquility to cause any real problems.
To sum it up, a series of unfortunate exceptional cases set up an environment on TQ wherein 10 stations on the entire server could repeatedly lock sessions in the DB until it had to do a failover to free them up. As a result of this, our hotfixing protocols are being revised to prevent this from happening again in the future.
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP Atropos
|
Posted - 2007.09.15 19:51:00 -
[2]
\o/
|
|
|
CCP Prism X
|
Posted - 2007.09.15 19:53:00 -
[3]
Working on Saturdays is so much more giving when it results in something.
~ Prism X EvE Lead Database Developer, Relocating your character to a cozy, giant secure container since 2006. |
|
|
CCP Mindstar
|
Posted - 2007.09.15 19:57:00 -
[4]
|
|
|
CCP Wrangler
|
Posted - 2007.09.15 20:11:00 -
[5]
Originally by: Pirate Tom
Originally by: "CCP Wranglermuffin" ...wherein 10 stations on the entire server could repeatedly lock sessions in the DB until it had to do a failover to free them up...
So in a roundabout sort of way, it really was BoB and Goonfleet (et al) causing the crash!
No, they were not involved. I'd also like to clarify that this problem was not caused by any Alliance, we messed up and not any player(s).
Originally by: torswin http://www.yonkis.com/mediaflash/1000000.swf
thankyou! (same as wrangler (i think) posted but without the fleshlight ad).
You people will never let me forget that, will you?
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP John Proctor
|
Posted - 2007.09.15 20:12:00 -
[6]
I can go sleep now
Dont go 72 hours with no sleep... you get sent to the hospital and have to answer emails with your black berry from the emergency room
|
|
|
CCP Wrangler
|
Posted - 2007.09.15 20:19:00 -
[7]
Originally by: Pirate Tom Wrangler I was going for the obvious poke at the 'OMG BOB' and 'OMG GOONS' crowd. I know it was a code error. You explained that pretty clearly.
I figured, I just wanted to make sure people understood that it was no Alliances fault that this happened.
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP Gangleri
|
Posted - 2007.09.15 20:34:00 -
[8]
Originally by: Sylia Guess sharkbait is sleeping after a few sleepless nigths now eh :)
Grats CCP on getting it fixed though, we players appreciate it
If I can save some iskies I'll buy a sexy dev or 2 a beer at fanfest
All devs are sexy, you might want to start saving up.
|
|
|
Kaemonn
Forum Moderator
|
Posted - 2007.09.15 20:51:00 -
[9]
\o/
forum rules | [email protected]| Eve-CCG
Originally by: kieron: off duty You dont have to swallow!
|
|
|
Rauth Kivaro
Forum Moderator
|
Posted - 2007.09.15 20:53:00 -
[10]
Great stuff! Good job to everyone on the team who put in the hours to track this down.
Oh, and next time, get some sleep, CCP John Proctor. ;)
forum rules | [email protected] | Our Website
|
|
|
|
Eldo Davip
Forum Moderator
|
Posted - 2007.09.15 20:58:00 -
[11]
Get some sleep guys. Well done. ___
|
|
|
IC Tsuki Kiyuu
Caldari ISD Interstellar Correspondents
|
Posted - 2007.09.15 21:03:00 -
[12]
I'd been thinking all along that my screwy html tags in the latest IC article were to blame. I'd been having these nightmares that I'd slowly set off a chain reaction that would eventuate in the deletion of Jita and the unlocking of the gate back to Earth, out of which poured a vast army of epileptic Jovian furriers that enslaved us all.
Imagine my relief. Turns out we are safe from those bloodthirsty monsters for another day! \o/
Good work on delivering so quickly on the issue, that hint in the news article was right!
|
|
|
CCP Wrangler
|
Posted - 2007.09.15 21:08:00 -
[13]
Originally by: ViolenTUK Edited by: ViolenTUK on 15/09/2007 20:47:59 In this crash the entire "3asy company" Alliance was lost of the database. All of the alliance members were kicked from the alliance and 3asy company doesnt show in the rankings list. Please CCP give us back our alliance.
I'm sorry to hear that, can your CEO create a petition on it, or someone else within that alliance? I'll send off some emails too so we can get it handled.
Originally by: Pistonbroke Edited by: Pistonbroke on 15/09/2007 20:55:15 Clarification please.... does this mean that conquerables can not ever be set as capital station?
I believe that was never the intention.
Originally by: Kalissa Does this mean if you've set a conquerable station to be a capital already but it's not kicked in yet (you've not done the 30 days) that it wont kick in when the 30 days are up?
That is something I'll have to check into.
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP Explorer
|
Posted - 2007.09.15 21:39:00 -
[14]
Originally by: Larieen Yeah good job guys,but as it stands i have been with out eve for 7 days and every day i looked for the 36800 patch and its not available.Would really like to update my game so i can play with out the Incompatible (Protocol) message!It is nowhere on this site!
The patches are here.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Prism X
|
Posted - 2007.09.15 22:28:00 -
[15]
Edited by: CCP Prism X on 15/09/2007 22:28:33 Answers to some questions: According to design conquerable stations should not be able to become capital stations. They do count towards constellation sovereignty but cannot be capitals. They wont be allowed to become capital stations in the future either. Owners of such stations should already have been contacted by GMs but what resolution will be offered is not up to me.
~ Prism X EvE Lead Database Developer, Relocating your character to a cozy, giant secure container since 2006. |
|
|
CCP Explorer
|
Posted - 2007.09.15 22:53:00 -
[16]
Originally by: Swamp Ziro
Originally by: CCP Explorer
Originally by: CCP John Proctor Dont go 72 hours with no sleep... you get sent to the hospital and have to answer emails with your black berry from the emergency room
I wasn't sure which worried me more, that you were sent to the hospital or that you kept answering your e-mails from the ER. Thank you for keeping TQ up this week.
Wait.. seriously?
Yes, seriously.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Prism X
|
Posted - 2007.09.15 23:40:00 -
[17]
True, YouWhat suffers from this mistake of ours. However, this topic is not about that. I believe your alliance is in contact with our GMs to try and resolve the issue and I hope that resolution will be fair and square. I'm not trying to diminish the problem you have in any way at all. I just don't want this issue to become derailed. We're on your case to the extent that I've known about it for almost a week now, no worries it will be resolved one way or the other.
~ Prism X EvE Lead Database Developer, Relocating your character to a cozy, giant secure container since 2006. |
|
|
CCP Prism X
|
Posted - 2007.09.16 00:09:00 -
[18]
Originally by: Emmerlaine
Originally by: Apoc77
Originally by: CCP Prism X Edited by: CCP Prism X on 15/09/2007 22:28:33 Stuff.
Stuff.
Heh, sounds like CCP fooled 10 different alliances into putting a lot of work into trying to get sov 4. Guess the joke's on us players again.
I've answered this already. Please refrain from derailing this topic. If you feel my previous answer was inadequate create your own topic.
~ Prism X EvE Lead Database Developer, Relocating your character to a cozy, giant secure container since 2006. |
|
|
CCP Wrangler
|
Posted - 2007.09.16 13:27:00 -
[19]
When it comes to reimbursing ship losses and similar during this period of crashes that's not something we can comment on in this thread. As with all reimbursement it will be handled by our Customer Support people on a case by case basis.
As has been said, it was never the intention that conquerable stations should be able to be set at capital stations. In the cases where that has been done we ask the alliances affected to work with out Customer Support team to resolve this issue.
The 3asy company alliance appears not to have paid its bill, the crash revealed that and the alliance was disbanded. You will have to recreate the alliance to fix this, however we will help you out of you have any problems recreating the alliance and in case you had a alliance logo made we will make sure you get it back as soon as we can. That will be handled through the petition you made though.
I hope this has answered the recent questions in this thread.
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP Wrangler
|
Posted - 2007.09.16 14:01:00 -
[20]
Originally by: Guvnor RBM Edited by: Guvnor RBM on 16/09/2007 13:49:30 Thanks for that, I checked but we didn't get the bill...
The cheque is in the post ect ect :)
Any way, as long as it can be sorted
If some one could contact me in game or 101 commander to get it running again it would great
Thanks
The bill is sent to the CEO of the Executor Corporation for the Alliance.
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
|
CCP Wrangler
|
Posted - 2007.09.16 14:42:00 -
[21]
Originally by: Marcus DeMon The biggest issue we have is not the ISK as such... BUT THERE IS NO BILL ! for us to pay :(
The bill wasn't paid, and because of that the alliance disbanded. However, Guvnor RBM has a petition in and this will be dealt with there.
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP Wrangler
|
Posted - 2007.09.16 14:46:00 -
[22]
"Powered by Stella Artois"??
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
CCP Wrangler
|
Posted - 2007.09.16 18:14:00 -
[23]
Originally by: Guvnor RBM The sig maker has disaperred, but yeah, I didnt notice my self for almost 6 weeks
Probably due to Stella...
Wrangler Community Manager EVE Online
Contact Support - Contact Moderators - Report Bug - Submit News Leads - Knowledge Base Player Guide - Policies - Join ISD - Fan Submissions - DevFinder LiteÖ |
|
|
|
|