EVE Search - Why TQ is down

All Channels

EVE General Discussion

Why TQ is down

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 [2] 3 :: one page
Author	Thread Statistics \| Show CCP posts - 27 post(s)
Kehmor Caldari The Movement	Posted - 2007.03.05 01:27:00 - [31] Originally by: Sharkbait cluster is starting now in VIP mode with full proxy servers and all sol servers. will keep you updated. does this allow BoB members on?
Grant Smith Infinitus Odium Curse Alliance	Posted - 2007.03.05 02:04:00 - [32] http://evevault.ign.com/fullstory.php?id=27114 - dunno if im allowed to put url's in here but there it is anyway. <--- Very lost over this whole BoD business.
Cadiz Caldari No Quarter. C0VEN	Posted - 2007.03.05 02:24:00 - [33] I trust the ratting money I technically made during those 40 minutes of pre-server-reboot brokenness will happily appear in my wallet when the journals get fixed? ------ Director, No Quarter "There is no problem that cannot be solved by the judicious application of violence."
Jonaaka Kaliin Caldari	Posted - 2007.03.05 02:30:00 - [34] Well since this thread is going places, should we ask just who has "VIP" access? Might explain how out of 1000 online, 200 were BoB in local...
Dahak2150 Chaos Monkeys	Posted - 2007.03.05 02:33:00 - [35] Very nice to see the updates, makes me happy. ---------------- If you're on IE, pretend this is transparent while you get a browser that supports .png.
Boliknar The Shadow Order Hydra Alliance	Posted - 2007.03.05 03:01:00 - [36] Originally by: Jonaaka Kaliin Well since this thread is going places, should we ask just who has "VIP" access? Might explain how out of 1000 online, 200 were BoB in local... So I guess they would just have us believe out of the 300 fastest login guys in all of EVE 200 plus were in one alliance and in the same system? Its only paranoia if the accustations are unfounded.
Ealiom Blackhole Entry Point	Posted - 2007.03.05 03:02:00 - [37] Now that ladies and gents is a helluva professional update. Congrats on the handling and speedy response. You guys should get some sleep now. Space Cookie warping its way to you. Executioner Model Blackbird Model

Sharkbait	Posted - 2007.03.05 09:28:00 - [38] Edited by: Sharkbait on 05/03/2007 09:25:37 Originally by: Jonaaka Kaliin Well since this thread is going places, should we ask just who has "VIP" access? Might explain how out of 1000 online, 200 were BoB in local... well, basically the server has roles much like corporations do within eve. the only difference is that only CCP, Customer Support and Volunteer accounts can have these roles. The roles will simply not work on player accounts. When a player has a ROLE on the server, thats when he/she has different color text. There are many roles, ROLE_VIP is just 1 of them and it allows us to login when the cluster is running in VIP mode. QA use it after every patch todo a basic smoke test and when poo happens to TQ like it did, we offen start in VIP to make sure everything is ok before allowing everyone to login and causing more problems. As it stands only CCP and Customer Support accounts have this role, there are no Volunteer accounts with it yet, but it's something i'm planning on getting done. As regards the problem on TQ, i beleive a primary ID key in the transactions log filled which is not good, mkay. I will ask Valar to post abit more info if he has time as i don't fully know about the stuff that happened.

Par'Gellen Low Grade Ore	Posted - 2007.03.05 09:38:00 - [39] Awesome work on the updates Sharkbait! Thank you for keeping us informed during the downtime. Two thumbs up! --- CCP : Save my mousewheel!

Admiral Chamrajnagar	Posted - 2007.03.05 09:44:00 - [40] Originally by: Sharkbait As regards the problem on TQ, i beleive a primary ID key in the transactions log filled which is not good, mkay. I will ask Valar to post abit more info if he has time as i don't fully know about the stuff that happened. As Valar's personal slave I can answer that question so the masta doesnt have too... The problem was a table called accTransactions filled up, this table has a primary index that is a number of type int, an int can only go as high as "2,147,483,647" Or 2.1 billion transactions. The system is set to auto-incriment each transaction, we use these unique transaction numbers to track a lot of database calls within the EvE database. Markets, Agents, PvPTrading, all use these transaction numbers. When we got to the 2,147,483,646 transaction the stack overflowed, or went back to "0". As there was already an entry in there sharing the same number it caused the calls to fail, and thusly everything went offline. For those that may ask why are we not using BigInt, it is because bigint require very large and bloated index tables, but plans are in the works to move this particular table over to bigint. Hopefully that clears up some questions

Scilent Enigma Minmatar	Posted - 2007.03.05 09:56:00 - [41] Thanx for the explanation, and the updates, when things go down like this it's a relif to see that there is a line of communication about what is going on. Much appreciated. /Happy subscriber
ApaKaka Minmatar	Posted - 2007.03.05 10:24:00 - [42] Originally by: Boliknar So I guess they would just have us believe out of the 300 fastest login guys in all of EVE 200 plus were in one alliance and in the same system? Its only paranoia if the accustations are unfounded. Did you log in when the post was made that the server was up, or before? Because I could log in long before that. I got into rens and most my corpmates got into their systems before there were more than 20 people online in any systems local, without any waiting time. I'm pretty sure I'm not a VIP though :) Sounds like your FC was just slow on the uptake and ordered you all to log in too late, while BoB had continually tried to login, regardless of the status.
Splagada Minmatar Tides of Silence	Posted - 2007.03.05 10:29:00 - [43] nice realtime info, congratulations on this we can accept anything as long as we KNOW what happens it's ignorance that makes our anger, anger our bad posting habits, bad posting habits our bans wait i got lost in that loop ------ Relaxed corp recruiting
Rutoo Gallente Interstellar eXodus R0ADKILL	Posted - 2007.03.05 11:24:00 - [44] So quick question... So i was NPCin a triple 1.7 million spawn, Killed the First BS got the isk... Killed the Second BS no isk... Killed the thrid BS no isk... Will i get the isk at downtime? my wallet is missing 3 million :P _________________________________________________________ My Second EvE Video Club Seals Not Sandwichs
Ealiom Blackhole Entry Point	Posted - 2007.03.05 11:39:00 - [45] Please tell me your joking. I thought you were okay, please tell me your joking Rut! Executioner Model Blackbird Model
Col Entreri	Posted - 2007.03.05 11:43:00 - [46] With the wallet problem is it likely that we will see an extended downtime today? and good job with the communication much less stressful
Saldun Zexu	Posted - 2007.03.05 11:56:00 - [47] Originally by: Rutoo So quick question... So i was NPCin a triple 1.7 million spawn, Killed the First BS got the isk... Killed the Second BS no isk... Killed the thrid BS no isk... Will i get the isk at downtime? my wallet is missing 3 million :P I hope u'll get money. Happened to me too when I tried to buy battleship. I bought one but nothing happened. my wallet didn't change and no battleship in ship array. I tried 3 times - still nothing. after DT I got 3 battleships in array and wallet changed it was long time ago.
Emofilia	Posted - 2007.03.05 12:25:00 - [48] Originally by: Admiral Chamrajnagar The problem was a table called accTransactions filled up, this table has a primary index that is a number of type int, an int can only go as high as "2,147,483,647" Or 2.1 billion transactions. ... When we got to the 2,147,483,646 transaction the stack overflowed, or went back to "0". As there was already an entry in there sharing the same number it caused the calls to fail, and thusly everything went offline. so if ia buy 2 bil of units of tritanium and i start to sell it one buy one im near to break the server ?
jamesw Omniscient Order	Posted - 2007.03.05 12:30:00 - [49] <3 GM Retrofire and Sharkbait for keeping us up to date! -- Latest Vid: Domination!
Obsequious Woe	Posted - 2007.03.05 12:31:00 - [50] Or they archive out old records / re-index the table ... and move on ... they are right though whilst INT is a small number at 2.1 billion ... bigint will possibly have performance hits that have to be investigated. I applaud CCP on an actual downtime thread that has honest to goodness content. Whether this is due to an internal policy change or just awesome actions on Sharkbait's behalf I'm pleasantly surprised. /me is more then happy to wait the extra DT out since we have actual reasons. OW
Zartmann	Posted - 2007.03.05 12:46:00 - [51] Originally by: Splagada nice realtime info, congratulations on this we can accept anything as long as we KNOW what happens it's ignorance that makes our anger, anger our bad posting habits, bad posting habits our bans wait i got lost in that loop /Signed 100% Please CCP, use this episode as a perfect example of what the community need/desearve from u. Quick response + Fast and continous situation updades + Names and roles of those involved + Feedbacks + Comments... I just can say more about how PERFECT & PROFESSIONAL Style this event was dealt with! WAY TO GO !!!! Please make it STANDARD PROCEDURE !! We¦ll prove u with our 100% support that this "DEV - Run an Extra Mile posting on foruns - Realtime Feedback Procedure" is the truly and one way to go! My english sucks? I bet that u can also beat me in <my_native_language>? BTW: hate my spelling too? Go play Scrabble !

Sharkbait	Posted - 2007.03.05 12:51:00 - [52] Originally by: Zartmann Originally by: Splagada nice realtime info, congratulations on this we can accept anything as long as we KNOW what happens it's ignorance that makes our anger, anger our bad posting habits, bad posting habits our bans wait i got lost in that loop /Signed 100% Please CCP, use this episode as a perfect example of what the community need/desearve from u. Quick response + Fast and continous situation updades + Names and roles of those involved + Feedbacks + Comments... I just can say more about how PERFECT & PROFESSIONAL Style this event was dealt with! WAY TO GO !!!! Please make it STANDARD PROCEDURE !! We¦ll prove u with our 100% support that this "DEV - Run an Extra Mile posting on foruns - Realtime Feedback Procedure" is the truly and one way to go! this is not really a new Procedure, just sometime i always wanted to see before i started at CCP. i decided to do it last night as best i could. and i plan to do it for future problems should there be any. hopefully not tho

Anon I'muss	Posted - 2007.03.05 12:51:00 - [53] Server is up? Perhaps you would like to have a word with my clients, EVEMon and EVELauncher who all agree that server is offline... _________________________________________________________ My main forum alt...
ElfeGER Black Nova Corp Band of Brothers	Posted - 2007.03.05 12:58:00 - [54] Originally by: Admiral Chamrajnagar Originally by: Sharkbait As regards the problem on TQ, i beleive a primary ID key in the transactions log filled which is not good, mkay. I will ask Valar to post abit more info if he has time as i don't fully know about the stuff that happened. As Valar's personal slave I can answer that question so the masta doesnt have too... The problem was a table called accTransactions filled up, this table has a primary index that is a number of type int, an int can only go as high as "2,147,483,647" Or 2.1 billion transactions. The system is set to auto-incriment each transaction, we use these unique transaction numbers to track a lot of database calls within the EvE database. Markets, Agents, PvPTrading, all use these transaction numbers. When we got to the 2,147,483,646 transaction the stack overflowed, or went back to "0". As there was already an entry in there sharing the same number it caused the calls to fail, and thusly everything went offline. For those that may ask why are we not using BigInt, it is because bigint require very large and bloated index tables, but plans are in the works to move this particular table over to bigint. Hopefully that clears up some questions hmmm int is a 32 bit value on most databases which results in 4.2 billion different primary keys (-2.1 to 2.1 if signed int is used but then you wouldn't need to return to 0) db tools should allow monitoring of primary key usage and trigger an alarm when a certain amount is passed (prewarnings, automatic cleanup) I doubt that within 6 hours on sunday the transaction numbers spiked that much compared to the total of last weeks sunday
Lora Decksaimer	Posted - 2007.03.05 13:25:00 - [55] Originally by: Obsequious Woe I applaud CCP on an actual downtime thread that has honest to goodness content. Whether this is due to an internal policy change or just awesome actions on Sharkbait's behalf I'm pleasantly surprised. /me is more then happy to wait the extra DT out since we have actual reasons. OW /me will happy wait as long as is needed -no matter how long- just keep me realtime posted and I¦ll be in the clear Originally by: Zartmann WAY TO GO !!!! Please make it STANDARD PROCEDURE !! We¦ll prove u with our 100% support that this "DEV - Run an Extra Mile posting on foruns - Realtime Feedback Procedure" is the truly and one way to go! S.O.P (Standard Operational Procedure) NOW FTW /me says 100 times = This must to be implemented, This must to be implemented, This must to be implemented, This must to be implemented, This must to be implemented...
Gnulpie Minmatar Miner Tech	Posted - 2007.03.05 13:34:00 - [56] Excellent handling of the situation. The playerbase was informed of the progress all the time with some info while the gurus were working to solve the problem very fast which they succeeded with. And most important afterwards, when the experts had some time, the playerbase was informed in detail about what the problem was and got lots of info. That I call a good service!

Admiral Chamrajnagar	Posted - 2007.03.05 13:53:00 - [57] Edited by: Admiral Chamrajnagar on 05/03/2007 13:50:01 Originally by: El*ER hmmm int is a 32 bit value on most databases which results in 4.2 billion different primary keys (-2.1 to 2.1 if signed int is used but then you wouldn't need to return to 0) db tools should allow monitoring of primary key usage and trigger an alarm when a certain amount is passed (prewarnings, automatic cleanup) I doubt that within 6 hours on sunday the transaction numbers spiked that much compared to the total of last weeks sunday You would need a programing change to deal with negative numbers. Secondly, we can not run triggers on the database as it would degrade performance. The EVE database handles over 150 million transactions per day. We also run without domain checks on tables, and most contrainst types, due to once again a performance hit.

Roshan longshot Gallente Ordos Humanitas FREGE Alliance	Posted - 2007.03.05 13:56:00 - [58] Sharkbait your the man...I dont think I have ever seen any CCP employee communicate to us like you do. Great Job. Free-form Professions, ensure no limetations on professions. Be a trader, fighter, industialist, researcher, hunter *pirate* or mixture of them all. [i]As read from the original box.
SadisticTheKing	Posted - 2007.03.05 14:03:00 - [59] I hope my Aura is still going to be with me once i reconnect

Eldo Davip Forum Moderator Interstellar Services Department	Posted - 2007.03.05 14:07:00 - [60] Originally by: Emofilia Edited by: Emofilia on 05/03/2007 12:34:42 Originally by: Admiral Chamrajnagar The problem was a table called accTransactions filled up, this table has a primary index that is a number of type int, an int can only go as high as "2,147,483,647" Or 2.1 billion transactions. ... When we got to the 2,147,483,646 transaction the stack overflowed, or went back to "0". As there was already an entry in there sharing the same number it caused the calls to fail, and thusly everything went offline. so if i buy 2 bil of units of tritanium and i start to sell it one by one im near to break the server ? edited : some errors while typing Do that and you'll be on the bad end of Sharkbait's wrath. ___ Email Us \| Forum Rules \| Our Website EvE +NLINE - T+TALHELLDEATH SUPPORTER


Pages: 1 [2] 3 :: one page
First page \| Previous page \| Next page \| Last page

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.