EVE Search - New Dev Blog: Full Suite of Upgrades to the Tranquility DB

All Channels

EVE Information Portal

New Dev Blog: Full Suite of Upgrades to the Tranquility DB - Yippie!

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 2 3 4 5 :: [one page]
Author	Thread Statistics \| Show CCP posts - 14 post(s)

CCP Zymurgist Gallente C C P	Posted - 2011.03.24 16:29:00 - [1] CCP Yokai has a new blog in our technical series about the upgrades we have applied to the Tranquility Database. Read more about it here. Zymurgist Community Representative CCP NA, EVE Online Contact Us

Lothros Andastar Gallente	Posted - 2011.03.24 16:41:00 - [2] IBC! So, at first I was liek D: but then I saw the graphs (especially the Disk Busy one) and was liek :D
Walextheone The Red Circle Inc. Red Shift Alliance	Posted - 2011.03.24 16:56:00 - [3] Wow :-D
Dansels Friend	Posted - 2011.03.24 17:01:00 - [4] That's some serious hardware! Now go out there and take some pictures! Want to see them in action so to speak.
Moraguth Amarr Paragon Fury Cascade Imminent	Posted - 2011.03.24 17:03:00 - [5] omg... itsbeautiful.jpg ps. can we buy the old pieces of TQ that you aren't frankensteining over to the new setups? good game Hoc filum tradit - This thread delivers.
Vuk Lau 4S Corporation Morsus Mihi	Posted - 2011.03.24 17:05:00 - [6] :fapfapfap:
Zhentar Minmatar The 5th Freedom Wildly Inappropriate.	Posted - 2011.03.24 17:08:00 - [7] Can I have your [old] stuff? :)
Marconus Orion S.E.G.W.A.Y.	Posted - 2011.03.24 17:11:00 - [8] I'm not very tech savy so for us laymen non-super geeks, please using a car reference, what is the comparison? We going from a Fiesta to a Mustang?

CCP Yokai	Posted - 2011.03.24 17:16:00 - [9] Originally by: Marconus Orion I'm not very tech savy so for us laymen non-super geeks, please using a car reference, what is the comparison? We going from a Fiesta to a Mustang? We had a Ferrari 1985 288 GTO We have a Ferrari 2011 599 GTO Both great... both fast... the new one is just magic. make sense?

chainedtofate Caldari Fleet of Doom RaVeN Federation	Posted - 2011.03.24 17:18:00 - [10] Originally by: Marconus Orion I'm not very tech savy so for us laymen non-super geeks, please using a car reference, what is the comparison? We going from a Fiesta to a Mustang? think of it more going from a 96 mustang with 200k miles to a 2011 mustang with 10 miles
momma like	Posted - 2011.03.24 17:20:00 - [11] Edited by: momma like on 24/03/2011 17:20:30
Crux Croupier Locutus	Posted - 2011.03.24 17:22:00 - [12] confirming my laptop has 128GB of ram also pics plz so the faping can begin
Chihiro Ma	Posted - 2011.03.24 17:26:00 - [13] Amazing... so now you dont have to tune those SQL's for a while ;)
Dziu	Posted - 2011.03.24 17:28:00 - [14] Well i guess someone has new toys. We better see some improvements in clusters so we dont die on grid every time there 600+ pilots in a nod. :)
Commander Azrael Red Federation	Posted - 2011.03.24 17:28:00 - [15] Epic hardware is epic
DTson Gauur Caldari Association of Commonwealth Enterprises R.A.G.E	Posted - 2011.03.24 17:30:00 - [16] Tech** the best ** ( in dev blogs you PERVS! ) \o/ Quite simply epic step forward , finally after what? 6 years the DB Failovers actually WORK as they're supposed to :)
EliteSlave Minmatar Macabre Votum Morsus Mihi	Posted - 2011.03.24 17:32:00 - [17] I wish i could post that screen cap of Stan after sneaking into that trailer with internet.... But, I have so many questions about the hardware... like specific model numbers, firmware. More so to do with possible integration here at my office as we have a pretty large database and are looking to scale up our hardware as we are approaching saturation of it. so any ** is good **

Chribba Otherworld Enterprises Otherworld Empire	Posted - 2011.03.24 17:32:00 - [18] zomg. Secure 3rd party service \| my in-game channel 'Holy Veldspar'


CCP Yokai	Posted - 2011.03.24 17:34:00 - [19] Originally by: EliteSlave I wish i could post that screen cap of Stan after sneaking into that trailer with internet.... But, I have so many questions about the hardware... like specific model numbers, firmware. More so to do with possible integration here at my office as we have a pretty large database and are looking to scale up our hardware as we are approaching saturation of it. so any ** is good ** Ask and I'll try and answer... better yet if you are at Fanfest... I present all that tomorrow.

Nayette Ellis Dalogne	Posted - 2011.03.24 17:34:00 - [20] Holy whopping cow! And here I was thinking I already had all my nergasms this week.
Neramore	Posted - 2011.03.24 17:34:00 - [21] Awesome stuff; I has an envy! However, would you mind translating some of that to units we can all relate to, like Library of Congresses per Second (LoC/S), maybe? "Actually, I enjoy this in the same vague, horrible way I enjoy watching the A-Team." - P. Opus
EliteSlave Minmatar Macabre Votum Morsus Mihi	Posted - 2011.03.24 17:43:00 - [22] Originally by: CCP Yokai Originally by: EliteSlave I wish i could post that screen cap of Stan after sneaking into that trailer with internet.... But, I have so many questions about the hardware... like specific model numbers, firmware. More so to do with possible integration here at my office as we have a pretty large database and are looking to scale up our hardware as we are approaching saturation of it. so any ** is good ** Ask and I'll try and answer... better yet if you are at Fanfest... I present all that tomorrow. Wow, didnt expect a response like that.. Well I guess the first few questions are we are currently doing the FC/IP (Fibre Chan over IP) and we are kinda limited in the I/O factor of around 30 that we have currently attached to 12 Dell Powervault 3610's ( ISCSI / FC/IP and FC ) and we are already maxing out the hardware and we are trying to get the next bang for out buck with going full FC but since this will be our first foray into the "Enterprise" level we are reading "blah blah blah" and dont really understand what we should be looking for of sorts. Now im not expecting you to give me a visio flowchart of the equipment or how its setup. But if you can say broad terminology that is well acceptable for growth of the next 2-3 years and allows for 1000-1500 ( ideally would like to have hardware that supports 2500-3000 to allow for growth) users concurrently hitting the database at any given time would mucho appreciated.
Emrys Ap'Morgravaine Caldari Minmatar Ship Construction Services Ushra'Khan	Posted - 2011.03.24 17:43:00 - [23] Confirming that I've just had multiple "personal" moments in my pants, and may now die of dehydration... -=-=-=-=- Reformed Carebear. Much bear, zero care. -=-=-=-=-
Ban Doga	Posted - 2011.03.24 17:46:00 - [24] Edited by: Ban Doga on 24/03/2011 17:46:25 I remember that someone (CCP Explorer ?) said not so long ago, that the DB is not anywhere near its performance limit. So I'm really looking forward to seeing no performance improvements from this one. I'm happy that you are happy, tho...
Psihius Caldari Anarchist Dawn U N K N O W N	Posted - 2011.03.24 17:52:00 - [25] Edited by: Psihius on 24/03/2011 17:53:32 BEST GEEK ** EVAR! Most of us, geeks, would work for food to get our hands on that hardware, to be able to touch it with some very nasty thoughts ;) :D I'll blow my head with a shotgun if I will not make it to the next Fanfest! -------------------------------------------------- Originally by:** Blacksquirrel This is EVE. PVE can happen anywhere at anytime. Be prepared.

CCP Yokai	Posted - 2011.03.24 17:54:00 - [26] Originally by: EliteSlave Originally by: CCP Yokai Originally by: EliteSlave I wish i could post that screen cap of Stan after sneaking into that trailer with internet.... But, I have so many questions about the hardware... like specific model numbers, firmware. More so to do with possible integration here at my office as we have a pretty large database and are looking to scale up our hardware as we are approaching saturation of it. so any ** is good ** Ask and I'll try and answer... better yet if you are at Fanfest... I present all that tomorrow. Wow, didnt expect a response like that.. Well I guess the first few questions are we are currently doing the FC/IP (Fibre Chan over IP) and we are kinda limited in the I/O factor of around 30 that we have currently attached to 12 Dell Powervault 3610's ( ISCSI / FC/IP and FC ) and we are already maxing out the hardware and we are trying to get the next bang for out buck with going full FC but since this will be our first foray into the "Enterprise" level we are reading "blah blah blah" and dont really understand what we should be looking for of sorts. Now im not expecting you to give me a visio flowchart of the equipment or how its setup. But if you can say broad terminology that is well acceptable for growth of the next 2-3 years and allows for 1000-1500 ( ideally would like to have hardware that supports 2500-3000 to allow for growth) users concurrently hitting the database at any given time would mucho appreciated. First off... nothing over anything if you can do it... FC is the best bet for "Enterprise" like you said because of the session based communications and the potential for synchronous protection to redundant SAN controller. I am biased... this is my opinion but I avoid iSCSI unless forced but sharp object. Next... direct connection or true SAN need to be decided on early. Direct connecting the disks is faster (like nano seconds only) but cheaper because you don't have to buy switches. BUT!!! When you run out of host ports and you need to get another system attached or accessing the LUNs, boy are you gonna miss those switches. Next... SAS unless you know better. SAS is great! I love this stuff... cheaper than FC-SCSI and much better than SATA... best mid-ground disks. SSD's rock my socks but you will pay massively for them or you will get cheap ones and hate what you did later. Last... Find the right performance metric. Concurrent users means lots to lots of people. I would suggest looking at your IOPS. A 12 drive tray of SAS usually nets 5-10K IOPS on normal workloads for a DB. Hope that helps a little. All I can toss at you between beers at Fanfest :) CCP Yokai

EliteSlave Minmatar Macabre Votum Morsus Mihi	Posted - 2011.03.24 17:55:00 - [27] Originally by: Psihius BEST GEEK **** EVAR! Most of us, geeks, would work for food to get our hands on that hardware, to be able to touch it with some very nasty thoughts ;) :D I would work just for boarding for CCP for the experience alone. I would work for free ( just pay for my housing / dont mind me going to the chef with a big take home tupperware box)
lceman Gallente Destructive Influence Northern Coalition.	Posted - 2011.03.24 17:55:00 - [28] impressive wall of text, but unfortunely i can only come up with. Try harder, Try again, u fail. Why cause eve is still broken by lag, by unfair advantages, biased gm/dev. And to the next obvious question why do i still play eve, well thats easy question Cause of the friends i have trough the community i'm in, in eve simple as that. I don't play it be cause its a cool awsome game that exciting and thrilling or cause i'm building up a corp or an alliance, all that is more or less broken either by faulty game mechanic's or simple bad choices by gm/dev@ccp public relation. To put it simple i play eve to be able to sit on teamspeak and talk to my eve@buddies wich some of them have become irl@buddies, but eventually like everything else this group is becomimg smaller and smaller and then when there is no left ill quit too. If what not is, what is ?.

CCP Yokai	Posted - 2011.03.24 17:56:00 - [29] Originally by: Ban Doga Edited by: Ban Doga on 24/03/2011 17:46:25 I remember that someone (CCP Explorer ?) said not so long ago, that the DB is not anywhere near its performance limit. So I'm really looking forward to seeing no performance improvements from this one. I'm happy that you are happy, tho... Check the disk busy... CPU... um and other graphs in the blog... We dug deeper and found some bottlenecks recently.

Ban Doga	Posted - 2011.03.24 18:00:00 - [30] Edited by: Ban Doga on 24/03/2011 18:02:45 Originally by: CCP Yokai Originally by: Ban Doga Edited by: Ban Doga on 24/03/2011 17:46:25 I remember that someone (CCP Explorer ?) said not so long ago, that the DB is not anywhere near its performance limit. So I'm really looking forward to seeing no performance improvements from this one. I'm happy that you are happy, tho... Check the disk busy... CPU... um and other graphs in the blog... We dug deeper and found some bottlenecks recently. I didn't say the DB is not running better now (it certainly is). I was just repeating what a CCP official said: the DB is not a bottleneck itself (I'll try to find the original statement on eve-search). If your SOL nodes hit 100% CPU load your improved DB performance won't matter that much... EDIT Reduced stress for the machines is great (especially for the ones in charge of the health of those servers), but how does this translate into perceived performance for the users?
gtiness Sick Tight Controlled Chaos	Posted - 2011.03.24 18:10:00 - [31] Edited by: gtiness on 24/03/2011 18:10:39 What is the size of the EVE database? Edit: to properly snipe page2.
theocratis	Posted - 2011.03.24 18:11:00 - [32] Originally by: Ban Doga Edited by: Ban Doga on 24/03/2011 18:02:45 Originally by: CCP Yokai Originally by: Ban Doga Edited by: Ban Doga on 24/03/2011 17:46:25 I remember that someone (CCP Explorer ?) said not so long ago, that the DB is not anywhere near its performance limit. So I'm really looking forward to seeing no performance improvements from this one. I'm happy that you are happy, tho... Check the disk busy... CPU... um and other graphs in the blog... We dug deeper and found some bottlenecks recently. I didn't say the DB is not running better now (it certainly is). I was just repeating what a CCP official said: the DB is not a bottleneck itself (I'll try to find the original statement on eve-search). If your SOL nodes hit 100% CPU load your improved DB performance won't matter that much... EDIT Reduced stress for the machines is great (especially for the ones in charge of the health of those servers), but how does this translate into perceived performance for the users? "We have also seen another positive, albeit unplanned, side effect of the increased performance of the new database systems. Previously if our SQL Server cluster needed to fail over to the redundant server, every node in the cluster died and all players were disconnected.We recently failed over to our secondary database server on the new system and only 3 nodes out of 208 died! This means with some tweaks we may be able to fail the servers, storage, and switching environment without a single disconnect!"

CCP Yokai	Posted - 2011.03.24 18:11:00 - [33] Originally by: gtiness What is the size of the EVE database? Lots of "stuff" in there but somewhere around 1.3TB

EliteSlave Minmatar Macabre Votum Morsus Mihi	Posted - 2011.03.24 18:14:00 - [34] Originally by: CCP Yokai Edited by: CCP Yokai on 24/03/2011 17:57:15 Originally by: EliteSlave Originally by: CCP Yokai Originally by: EliteSlave I wish i could post that screen cap of Stan after sneaking into that trailer with internet.... But, I have so many questions about the hardware... like specific model numbers, firmware. More so to do with possible integration here at my office as we have a pretty large database and are looking to scale up our hardware as we are approaching saturation of it. so any ** is good ** Ask and I'll try and answer... better yet if you are at Fanfest... I present all that tomorrow. Wow, didnt expect a response like that.. Well I guess the first few questions are we are currently doing the FC/IP (Fibre Chan over IP) and we are kinda limited in the I/O factor of around 30 that we have currently attached to 12 Dell Powervault 3610's ( ISCSI / FC/IP and FC ) and we are already maxing out the hardware and we are trying to get the next bang for out buck with going full FC but since this will be our first foray into the "Enterprise" level we are reading "blah blah blah" and dont really understand what we should be looking for of sorts. Now im not expecting you to give me a visio flowchart of the equipment or how its setup. But if you can say broad terminology that is well acceptable for growth of the next 2-3 years and allows for 1000-1500 ( ideally would like to have hardware that supports 2500-3000 to allow for growth) users concurrently hitting the database at any given time would mucho appreciated. First off... nothing over anything if you can do it... FC is the best bet for "Enterprise" like you said because of the session based communications and the potential for synchronous protection to redundant SAN controllers. I am biased... this is my opinion but I avoid iSCSI unless forced buy sharp object. Next... direct connection or true SAN need to be decided on early. Direct connecting the disks is faster (like nano seconds only) but cheaper because you don't have to buy switches. BUT!!! When you run out of host ports and you need to get another system attached or accessing the LUNs, boy are you gonna miss those switches. Next... SAS unless you know better. SAS is great! I love this stuff... cheaper than FC-SCSI and much better than SATA... best mid-ground disks. SSD's rock my socks but you will pay massively for them or you will get cheap ones and hate what you did later. Last... Find the right performance metric. Concurrent users means lots to lots of people. I would suggest looking at your IOPS. A 12 drive tray of SAS usually nets 5-10K IOPS on normal workloads for a DB. Hope that helps a little. All I can toss at you between beers at Fanfest :) CCP Yokai Hey thanks for the tidbit of advice, (Give me a virtual server farm and I can do wonders... give me a database make me cry...but im a masochist and would love the time learn it) I definitely agree with the anything over anything is going to be a hassle and try to avoid it, but the CTO before me well... had X budget and if he spent only Y he got a certain commission on that and the company finally learned that well being cheap and rewarding cheap only got them deeper in the hole later on and finally fired him and now going into compliance and looking to stay ahead of the curve. Can you reccomend any classes to take and which to avoid as you think they are a waste of time and or just nothing to gain from? PS: If i find you at fanfest I will throw a beer your way and plus my resume ( even tho i know you prolly cant hire me ) hahha

CCP Valar	Posted - 2011.03.24 18:17:00 - [35] Perhaps the perceived performance for the users won't change much with the new database hardware, but it gives us a lot of room to grow and makes us able to perform more online maintenance without affecting users and prevents us from having to schedule extended downtimes for things we needed to do offline before. Also, a major part of the decision to upgrade the hardware was to increase availability and options for disaster recovery. ---- Senior Virtual World Database Administrator Virtual World Operations CCP Games

Leet Magician Evolution IT Alliance	Posted - 2011.03.24 18:20:00 - [36] maybe now the logs will actually show something!!
AnonyTerrorNinja Minmatar Atomic Geese	Posted - 2011.03.24 18:24:00 - [37] If I had to be told "you are going to work in the server room" I swear I would take a sleepingbag, one of those funny inch-thick mattress things, a perpetual coffee machine, a portable shower and never leave. Just being around that kind of awesome is enough. Besides all of this, you just might be reading my signature.
Sarmatiko	Posted - 2011.03.24 18:25:00 - [38] fap fap fap Thanks!
Ban Doga	Posted - 2011.03.24 18:27:00 - [39] Originally by: theocratis Originally by: Ban Doga Edited by: Ban Doga on 24/03/2011 18:02:45 Originally by: CCP Yokai Originally by: Ban Doga Edited by: Ban Doga on 24/03/2011 17:46:25 I remember that someone (CCP Explorer ?) said not so long ago, that the DB is not anywhere near its performance limit. So I'm really looking forward to seeing no performance improvements from this one. I'm happy that you are happy, tho... Check the disk busy... CPU... um and other graphs in the blog... We dug deeper and found some bottlenecks recently. I didn't say the DB is not running better now (it certainly is). I was just repeating what a CCP official said: the DB is not a bottleneck itself (I'll try to find the original statement on eve-search). If your SOL nodes hit 100% CPU load your improved DB performance won't matter that much... EDIT Reduced stress for the machines is great (especially for the ones in charge of the health of those servers), but how does this translate into perceived performance for the users? "We have also seen another positive, albeit unplanned, side effect of the increased performance of the new database systems. Previously if our SQL Server cluster needed to fail over to the redundant server, every node in the cluster died and all players were disconnected.We recently failed over to our secondary database server on the new system and only 3 nodes out of 208 died! This means with some tweaks we may be able to fail the servers, storage, and switching environment without a single disconnect!" I didn't miss that, but it's not about performance. It's about stability. You don't get dropped, but you're not getting improved performance while staying online. Also found the original statement I was referring to: Originally by: CCP Atlas We only have a single database and it's easier to scale that up than the sol nodes and we're already ahead of the curve in terms of what the DB can deliver. We do cache very aggressively on the server though and consolidating these character node calls onto a half a dozen nodes rather than servicing them throughout the cluster does remove a bit of the DB load since we get more cache hits, but like I said, the DB is not a big issue in this regard today. http://www.eveonline.com/ingameboard.asp?a=topic&threadID=1371750&page=2#39
Thunderf00t	Posted - 2011.03.24 18:32:00 - [40] Was wondering about the Windows side of things. Can/Do you disable the FS buffer cache for particular FS ( don't know if it's possible ) so you don't have the same data, 2 times in the memory? Is there a option to disable the FS file locking mechanics in windows, or maybe the SQL can open the database files with some option so it disables the buffer cache for the particular file that the DB process opens and maybe the locking of the said file, so if you have more SQL processes accessing the same file they can write to it concurrently? Does the storage system support active-active mode, or is it passive-active to the storage processors/controllers? What SAN switches do you use? Brocade...Cisco? I suppose the DB cluster is some sort of active-active setup ( something like Oracle RAC maybe)?
J Kunjeh Gallente	Posted - 2011.03.24 18:35:00 - [41] Yet another sultry Dev Blog for those of us who love the tech pron. So enlightening to read more in-depth about the Eve architecture. Just finished another article over at Gamasutra that went into some depth on the architecture as well (here for those who are interested). Keep up the good work CCP! ~Gnosis~
Batolemaeus Caldari Free-Space-Ranger Morsus Mihi	Posted - 2011.03.24 18:41:00 - [42] I came.
Ariane VoxDei	Posted - 2011.03.24 18:41:00 - [43] Originally by: CCP Yokai Check the disk busy... CPU... um and other graphs in the blog... We dug deeper and found some bottlenecks recently. Yes, the "disk before" graphs is scary. Like really really scary. If that translates to something like the similar graph in windows, it gives peen-shrinking shivers of diskwaits - making even the mightiest cpu/ram/gfx combo seem like stoneage implements choking any game into a stuttering slideshow while it desperately waits for IO requests to complete. (memories of logging into lagaran come to mind). Interesting graph of online-players you had about fall/fail-over to redundant SQL server. That smaller spike coincides well with the recent mass disconnent many of us suffered, where we got repeated disconnect after logging back in for quite some time. Think it was about 2 weeks ago. Was that it or something else? Quote: Ask and I'll try and answer... better yet if you are at Fanfest... I present all that tomorrow. Looking forward to watching that. Anyway, if that failover was the cause, could you try to talk about that on the presentation? And talk about it anyway, so we know what to expect when it does happen. The ugly thing about that problem was that it only dropped "some". If it drops everyone, well no worries, your enemies dropped too. Partial drops are nastier. I am not a titan pilot, but I think you can get the picture, and thats just one of the ugly scenarios. Match that with the revision of the reimburse policy, as per GM blog, and that can suddenly be very expensive.
Andrea Griffin	Posted - 2011.03.24 18:42:00 - [44] So, how much did this all cost (roughly)? Being a nerd girl I love hearing about this stuff. I really enjoy CCP's tech blogs. I'd love to play with that hardware, but being able to play ON it is almost as fun (and I don't get called at 4am when it breaks). - "When I nerf something, it takes 2-3 months for your dreams to be crushed." - CCP Big Dumb Object
Celebris Nexterra Gallente Lowsec Static	Posted - 2011.03.24 18:46:00 - [45] Man, you guys have been churning out devblogs like it's your freaking JOB these past two weeks! You- wait...yes...OK, I'm being told it is in fact your job to post devblogs. Nonetheless! It is still awesome! Keep up the great work, and give me more fapping material like the screen cap of 16 hyper-threaded CPU cores =D!!!
Ciaa Gallente The Executives	Posted - 2011.03.24 18:48:00 - [46] Nice blog, good to see some tech love/**** :D Any chance of some photos around the office and server room? DON'T PANIC!
Rambobinette	Posted - 2011.03.24 18:52:00 - [47] Is it really a single database? if it is, it means you have a active-passive cluster configuration which is a waste of computing resources because when you have 2 or more, you can balance the databases on each nodes giving you an active/active cluster configuration. You can also add more nodes thus reducing the work load.
Charles37	Posted - 2011.03.24 18:54:00 - [48] Those are some incredibly sexy graphs. Thank you! This also makes my trusty computer feel... rather inadequate. But then again, spec sheets aren't everything, right...? Right?
Aldariandra Gallente MunsterMunch The 0rphanage	Posted - 2011.03.24 18:57:00 - [49] Edited by: Aldariandra on 24/03/2011 19:04:21 This is very interesting. At our company we are currently trying out a Whiptail SSD SAN (rated theoretically up to 250000 IOp/s) and we get about 65000 I/ops out of it on account of being limited to 4Gb FC Blade switches (Brocom). This is to run VMware storage on btw. Didn't you guys use Blades aswell (IBM)? If so, seems like a lot of ports being taken up for both network and HBA? What kind of SAN switches do you use? Maybe I am picky, but average disk queue of about 3 still seems high to me. The fact that your storage still seems to get to 100% disk use also explains the queuing probably. I would not be happy with ever seeing disks bottleneck on 100% use, its something I would see as a definite problem to solve still. I find it very interesting that you run Eve on MS SQL. We have a lot of performance problems with some of our SQL servers and our DBA's are quite inexperienced. I would very much like to know how its set up and how you spread the load out. What kind of IOP/s does the database eat?
Myobi Rush	Posted - 2011.03.24 19:00:00 - [50] This effect every system or just Jita? :Unamused:
Rambobinette Caldari Angels of Death Corp	Posted - 2011.03.24 19:05:00 - [51] Originally by: Aldariandra I find it very interesting that you run Eve on MS SQL. We have a lot of performance problems with some of our SQL servers and our DBA's are quite inexperienced. I would very much like to know how its set up and how you spread the load out. What kind of IOP/s does the database eat? thre is a limit a DBA can fix. Even with the appropriate index and optimizations, if the app do a full table scan because the developer doesn't know SQL rules, well you will have problems. I had great experiences with MSsql. Luc R http:://www.lucraymond.net MCSE&MCDBA
Koshiko Murakami	Posted - 2011.03.24 19:20:00 - [52] So how many TPS does the current database hit? How do you see this expanding?
Soldarius Caldari Northstar Cabal R.A.G.E	Posted - 2011.03.24 19:23:00 - [53] Wait. A business is using its income from subscribers to actually improve the business? Impressive numbers. Great job, CCP. Keep it up. Originally by: CCP Shadow ...I cannot guarantee (my) sobriety or decency.
ORCACommander	Posted - 2011.03.24 19:25:00 - [54] Originally by: Vuk Lau :fapfapfap: this ^^^^
DiaBlo UK ZDK	Posted - 2011.03.24 19:33:00 - [55] ball park figure on the cost of the upgrade??? Originally by: CCP Navigator Pretty sure someone is selling tinfoil hats. You should buy one Originally by: CCP Zulupark Trollin' with my homies!
Ishina Fel Caldari Terra Incognita Intrepid Crossing	Posted - 2011.03.24 19:42:00 - [56] Originally by: Ban Doga Also found the original statement I was referring to: Originally by: CCP Atlas We only have a single database and it's easier to scale that up than the sol nodes and we're already ahead of the curve in terms of what the DB can deliver. We do cache very aggressively on the server though and consolidating these character node calls onto a half a dozen nodes rather than servicing them throughout the cluster does remove a bit of the DB load since we get more cache hits, but like I said, the DB is not a big issue in this regard today. http://www.eveonline.com/ingameboard.asp?a=topic&threadID=1371750&page=2#39 That quote is from August 2010... I'm sure it was compeltely true back then. But since then, they released Incursions, activated resource depletion on planets, overhauled the whole inventory system, and did other code improvements... I'm pretty sure that it is especially the latter two things that trouble the database. Imagine - they just released a blog where they state that they can allow for Jita's maximum population to grow by over 1000 additional people, because the new efficient inventory code allows the node CPU to handle that many more inventory operations per second. But where do all these inventory operations go? Well, they hit the database. And now there's going to be a whole lot more of them in the same amount of time. Not only in Jita, but in every system that saves CPU cycles due to this coding change. So the very database that ended up sitting around bored because TQ couldn't generate enough requests to saturate it, suddenly had to scramble to keep up, approaching its limits. So an upgrade made sense. (This is of course pure guesswork, I have no idea what really happened. I only know that more often than not when you improve one part of a complex system, you end up stressing a different part without even meaning to.) And on topic: that is a beautiful database server you got there. As a system integrator myself, consider me jealous - I can't find a single thing I would have done different! - Signature? What signature?
Diomedes Calypso	Posted - 2011.03.24 19:50:00 - [57] cool stuff.. just a thumbs up to let you know its being read and enjoyed even by people like me who have little clue about what some of the stuff means and use it as a learning experience.
Dian'h Might Minmatar Cash and Cargo Liberators Incorporated	Posted - 2011.03.24 20:14:00 - [58] Awesome blog. Technical details like that are great and give me an excuse to read eve forums at work - - - Dian'h Might - C&Ps resident "internet kleptomaniac"
Mr LaForge	Posted - 2011.03.24 20:21:00 - [59] Whoah....Dude.. So I herd u got new hamsters...
Shandir Minmatar EVE University Ivy League	Posted - 2011.03.24 20:22:00 - [60] Originally by: DiaBlo UK Edited by: DiaBlo UK on 24/03/2011 19:54:23 ball park figure on the cost of the upgrade??? I mean, do I need to win the Saturday night jackpot or wait around for a Euro millions double roll-over? I suspect they already have to put a lot of effort into cooling, although this is an idea for reinforced nodes. If CCP currently is looking into multicore as they cannot push single-core processing as much as they'd like - what is stopping you from taking the highest clock-speed rating CPU commercially available, and then overclock it under heavy cooling for the max performance reinforced nodes? - Vote Trebor Daehdoow for CSM and Chairman of CSM. Trebor's Campaign Manifesto
Ban Doga	Posted - 2011.03.24 20:54:00 - [61] Originally by: Ishina Fel Originally by: Ban Doga Also found the original statement I was referring to: Originally by: CCP Atlas We only have a single database and it's easier to scale that up than the sol nodes and we're already ahead of the curve in terms of what the DB can deliver. We do cache very aggressively on the server though and consolidating these character node calls onto a half a dozen nodes rather than servicing them throughout the cluster does remove a bit of the DB load since we get more cache hits, but like I said, the DB is not a big issue in this regard today. http://www.eveonline.com/ingameboard.asp?a=topic&threadID=1371750&page=2#39 That quote is from August 2010... I'm sure it was compeltely true back then. But since then, they released Incursions, activated resource depletion on planets, overhauled the whole inventory system, and did other code improvements... I'm pretty sure that it is especially the latter two things that trouble the database. Imagine - they just released a blog where they state that they can allow for Jita's maximum population to grow by over 1000 additional people, because the new efficient inventory code allows the node CPU to handle that many more inventory operations per second. But where do all these inventory operations go? Well, they hit the database. And now there's going to be a whole lot more of them in the same amount of time. Not only in Jita, but in every system that saves CPU cycles due to this coding change. So the very database that ended up sitting around bored because TQ couldn't generate enough requests to saturate it, suddenly had to scramble to keep up, approaching its limits. So an upgrade made sense. Well August 2010 is not that much time IMO. 7 months won't totally change the landscape. Yes, they released Incursions and a lot of other changes but I don't expect the overall ratio "stress for the CPU" vs "stress for the DB" to change radically in such a short time. Furthermore there has been no statement that the situation is suddenly reversed and the DB needed an upgrade. On top of that CCP Valar basically confirmed that an improvement in perceived performance is not probable. Originally by: Ishina Fel (This is of course pure guesswork, I have no idea what really happened. I only know that more often than not when you improve one part of a complex system, you end up stressing a different part without even meaning to.) Yes, it is all guesswork we're doing here. Good to keep that in mind. It is also true that there is always a limiting factor and addressing one will usually make something else a limiting factor. However it is also a good idea to stick to the information actually available and not guess too much. If the last info was "DB is not stressed" and no one said "this has changed" why assume it has changed? As an excercise: try assuming it has changed so that the DB is even less stressed than before - after all no one said this would be the case either.
Ishina Fel Caldari Terra Incognita Intrepid Crossing	Posted - 2011.03.24 20:55:00 - [62] Originally by: Shandir I suspect they already have to put a lot of effort into cooling, although this is an idea for reinforced nodes. If CCP currently is looking into multicore as they cannot push single-core processing as much as they'd like - what is stopping you from taking the highest clock-speed rating CPU commercially available, and then overclock it under heavy cooling for the max performance reinforced nodes? Probably the fact that they will lose all warranty support from their hardware supplier for anywhere from a single machine to the entire cluster, and depending on the terms of use, might be found guilty of breach of contract? Then there's stability concerns. You do not run servers outside specifications, period. If you overclocked desktop CPU botches a single calculation a day because it is at its limit, you probably won't even see anything happen at all for months to come. If each of TQ's CPUs botches a single calculation a day, then the server will become borderline inoperable - and CCP will have to foot every single cent of the bill itself. Never, ever run a production environment that your entire business depends on outside of specifications. That's like building a nuclear power plant in Japan and forgetting the flood walls. And despite how utterly minimal the chances of the worst case scenario were, we all know how THAT ended up, don't we. - Signature? What signature?
Tinkanium	Posted - 2011.03.24 20:58:00 - [63] Originally by: Ishina Fel Imagine - they just released a blog where they state that they can allow for Jita's maximum population to grow by over 1000 additional people, because the new efficient inventory code allows the node CPU to handle that many more inventory operations per second. But where do all these inventory operations go? Well, they hit the database. And now there's going to be a whole lot more of them in the same amount of time. Not only in Jita, but in every system that saves CPU cycles due to this coding change. So the very database that ended up sitting around bored because TQ couldn't generate enough requests to saturate it, suddenly had to scramble to keep up, approaching its limits. So an upgrade made sense. (This is of course pure guesswork, I have no idea what really happened. I only know that more often than not when you improve one part of a complex system, you end up stressing a different part without even meaning to.)] I was planning to drip sarcasm all over this thread in a post about how I witnessed performance go down in light of hardware increases and code optimization. Thankfully in a rare fit of wisdom I read the entire thread before posting to avoid looking like a total ass. I suspect that Ishina Fel is quite correct. Immediately after the patch dealing with the new inventorying system (designed to nerf the soul-crushing lag caused by missile swarms) I personally witnessed a very noticeable hit to performance when loading new grids, entering systems, initiating jumps, docking/undocking, loading planetary interaction (sometimes a crash, but maybe that's client hardware specific) and most annoyingly, when selecting something from the overview and then trying to click buttons in the Selected Item window. At one point I was even presented with the buttons for a ship after clicking a stargate. Instead of being able to warp to this gate, I was presented with greyed out approach and warpto buttons, as well as a button for enter ship. Interesting choice when it comes to gates. I think this will fix it. I am looking forward to experiencing gameplay on the new hardware. Especially the backend stuff for all of you server-jockeys. Fewer extended downtimes is better! It's already a rather impressively short downtime when compared to other online games. It's certainly not five-nines but let's face it games don't seem to have a hope in hell of achieving that any time soon. To end on a funny note in response to the post regarding the use of profits to improve the service which garnered said financial gains, looks like the sandbox is not the only thing that CCP is unique for! And yes. I am insanely jealous of the shiny new hardware you guys are playing with!
Tinkanium	Posted - 2011.03.24 21:06:00 - [64] Originally by: Ban Doga It is also true that there is always a limiting factor and addressing one will usually make something else a limiting factor. Yeah man. In fact based on what you yourself said you're already wrong. There's always a limiting factor. If you fix that, and it is no longer the limiting factor, then by the logic that there's always a limiting factor something else must become the limiting factor. Even in the dev blogs there's always someone. Seriously. Incredibly long post containing nothing but quotes and seemingly intelligent logic that basically amounts to nothing. Please do save us all some time and just read along k?
Korerin Mayul Amarr	Posted - 2011.03.24 21:19:00 - [65] Resident EVE Storage nerd checking in with some questions, comments and suggestions: ...so you have four 8g ports per server, rolled into 2 trunks of 16g? - sweet!! what is managing the host level trunking? I havent seen this done in anger before or you have two 8g ports per server, rolled into 1 trunk of 16g - less sweet, in this case how do you handle link failure and are you still doing two paths to two fabrics? Might i recommend a cross-grade to Brocade HBSs? ÆRumour has itÆ That they will be first to market with ratified 16gb FSPs / HBAs / Switches û super plasma fast if you want on that shizzell early. Also, as a shot in the dark, go and take a look at the EMC Greenplum appliance û its a weird use-case but a big sack of ETL capabilities may be useful for something up there. A fully kitted, massively parallel VSP might outperform that IBM box, but im only mentioning it so i donÆt sound like an EMC shill :P Additionally, be really careful with thin provisioning, I speek form bitter, bitter experience that using that stuff in prod will cause many tears and late night clickeyfests. Finally, if you have any more money spare, you could give it to me, or buy yourselves some Data Domain kit, those boxes can handle 3gb/s sustained throughput per controller. If that doesnÆt back your face right up, i donÆt know what will.
Hawk TT Caldari Bulgarian Experienced Crackers RED Citizens	Posted - 2011.03.24 21:27:00 - [66] CCP, now you have to look at Intel Xeon X5698 Dual-Core 4.4GHz and X569X @ 4.66GHz... I suppose those would be nice match for the SOL nodes... ___________________________________ BECKS
Akita T Caldari Navy Volunteer Task Force	Posted - 2011.03.24 21:28:00 - [67] Always nice to see YARRRrrrrdware p0rn blogs _ Make ISK\|\|Build\|\|React\|\|1k papercuts _
Commander TGK Gallente The Deep Space Armada	Posted - 2011.03.24 21:36:00 - [68] Now that is our subscriber money at work! Excellent job CCP! Those are gigantic improvements over the old.
Vaerah Vahrokha Minmatar Vahrokh Consulting	Posted - 2011.03.24 21:43:00 - [69] Quote: If I had to be told "you are going to work in the server room" I swear I would take a sleepingbag, one of those funny inch-thick mattress things, a perpetual coffee machine, a portable shower and never leave. Just being around that kind of awesome is enough I don't get why a server room would be so exciting. I have worked for 6 years next and inside a datacenter with loads of fiber optic connections, super SANs, 7-8 different OSes and databases, web servers... ... after the first days of amusement at retina scans and similar, it gets old quick. Best use? We stored good seasoned wine under the modular floor tiles, the air conditioning kept them at optimal temperature. Another time I worked in and out of a controlled air (I don't know the exact English word) datacenter and the only thing that got me was how boring is to constantly dress the white suit and do that cleaning shower. Auditing \| Research \| 3rd Party \| Collateral Holding \| EvE RL Charity
Kalia Masaer Amarr Border Defense Consortium	Posted - 2011.03.24 21:45:00 - [70] While I greatly appreciate the effort and cost of the hardware being installed to reduce lag, it will not be capable of staying ahead of players who will simply bring bigger blobs until the new hardware has reached its limit and crashes just like they currently do, it will just take more players. As important though is a way to make it so simply bringing a bigger blob is not the only way to win. It is sadly where I feel we are being let down. The battle to make the software and hardware more capable seems to have people working head over heels to do the impossible but really it is a pointless gestures as ever bit of gain made is gone within weeks of as more people come to fights. I'd really like to see some development effort spent on moving forward with a rework of SOV. Sad as it maybe the old tower spamming sov method was in many ways better at spreading out battles. CCP needs to give the guys fighting lag a break by making it so we the players do not keep taxing the system to the limit, which we will do so long as more is better.
Justin Cody Caldari Instant Annihilation Everto Rex Regis	Posted - 2011.03.24 22:11:00 - [71] so much beautiful YARRRDWARE!!! Can I haz one server? Remind people that profit is the difference between revenue and expense. This makes you look smart. Scott Adams
Patient 2428190 DEGRREE'Fo'FREE Internet Business School	Posted - 2011.03.24 22:14:00 - [72] Well I'll be damned, a tech blog that isn't "HURRRR WE THINK ITS WURKIN LOL" Very nice, very sexy ...Then when you stopped to think about it. All you really said was Lalala.
Svizac Marmotov Caldari	Posted - 2011.03.24 22:50:00 - [73] Is Tranquility going to become self-aware?
ModeratedToSilence	Posted - 2011.03.24 23:05:00 - [74] Originally by: CCP Yokai Originally by: Marconus Orion I'm not very tech savy so for us laymen non-super geeks, please using a car reference, what is the comparison? We going from a Fiesta to a Mustang? We had a Ferrari 1985 288 GTO We have a Ferrari 2011 599 GTO Both great... both fast... the new one is just magic. make sense? Attempting to align computer technology development with automotive development is stupidity. Computer hardware tends to double in its basic attributes every 18-24 months. If we had the same development in motor vehicles we would not be playing internet spaceships - we would be in real life spaceships.
Prince aikka Grand River Trades Intrepid Crossing	Posted - 2011.03.24 23:27:00 - [75] I got wood reading that. starts printer for a bathroom copy
Mihara Shiharu	Posted - 2011.03.24 23:46:00 - [76] Tech **** indeed!!!!!!
GRIEV3R Gallente Galactic Defence Consortium STR8NGE BREW	Posted - 2011.03.24 23:48:00 - [77] Awesome dev blog. My nerd boner is the size of an Avatar. CCP wins like Charlie Sheen.
Karajishi Caldari Caldari Provisions	Posted - 2011.03.25 00:04:00 - [78] Great job on the upgrades, looks sweeeet! Loving these blogs to, a great insight into how you guys do things. Keep it up CCP
Dawn Harbinger	Posted - 2011.03.25 00:50:00 - [79] If I understood half the things in this blog I'd have a much cooler job that I do now ;) Sounds epic though, hail our robot overlords!! WINNING!
LeeBaak	Posted - 2011.03.25 01:00:00 - [80] Originally by: Andrea Griffin So, how much did this all cost (roughly)? Curious about this, too. Also, since you won't be needing all of that "old junk" anymore, feel free to ship it my way...it's so out-dated...but I'm sure I can find a use for it.
Jason Edwards Autistic Sharks Test Alliance Please Ignore	Posted - 2011.03.25 01:19:00 - [81] Meanwhile my most important production server has 256mb ram, freebsd 4, 2 core xeon, 80 gig hd. ------------------------ To make a megathron from scratch, you must first invent the eve universe.
MidKnight Reign	Posted - 2011.03.25 01:21:00 - [82] Edited by: MidKnight Reign on 25/03/2011 01:24:05 Edited by: MidKnight Reign on 25/03/2011 01:23:22 Edited by: MidKnight Reign on 25/03/2011 01:22:22 Win upgrade is win! Also hardcore tech **** is hardcore. (Kosher in a dev blog but not on the forum? Oh, I see how it is...) Congratulations on subduing the dreaded disk I/O bottleneck (for now), as well as royally beating the snot out of the approaching CPU bottleneck. Remember kids: More threads > more cycles Would you rather do a few tasks simultaneously at 3Ghz or many tasks at 2Ghz? Or rather, would you rather have a few crazy fast turrets or a crazy number of decently fast turrets?
Lev Aeris b.b.k Fidelas Constans	Posted - 2011.03.25 01:58:00 - [83] Will this new hardware make Team Gridlock more successful? Cool blog. Thanks for sharing your new tool specs with us.
Wollari Phoenix Industries	Posted - 2011.03.25 02:13:00 - [84] Can I've your old hardware, pleeeeease :-)
freshspree	Posted - 2011.03.25 02:17:00 - [85] I lold. Anyway, let's see the difference when I log on today.
Naomi Wildfire Amarr Men Who Stare At Gates	Posted - 2011.03.25 02:23:00 - [86] Can i have your old suff? :(
ivar R'dhak Minmatar	Posted - 2011.03.25 03:31:00 - [87] Edited by: ivar R''dhak on 25/03/2011 03:31:23 Niiice. Yardware p÷rn. Ooh .. I¦m getting all tingly, and, uh .. THERE ya go. ______________ Mal-¦Appears we got here just in a nick of time. What does that make us?¦ Zoe-`Big damn heroes, sir.` Mal-¦Aint we just.¦
Essenz	Posted - 2011.03.25 03:44:00 - [88] Only 2 words @ CCP: HELL YEAH!
Dr BattleSmith PAX Interstellar Services	Posted - 2011.03.25 04:39:00 - [89] Quote: One day we woke up and our super cool database servers... well... weren't super cool anymore. ROFL! They actually once considered MS SQL to be "super cool"? LOL
Sith LordXXX	Posted - 2011.03.25 05:28:00 - [90] Edited by: Sith LordXXX on 25/03/2011 05:35:33 Edited by: Sith LordXXX on 25/03/2011 05:30:49 Opteron 6174's would have been a much better upgrade as apose to 8 fake cores in the intel and intel costing twice as much going for $2400, the AMD's have 12 real cores giving far more processing power then a intel chip with only 8 real cores with a marketing gimmic of 8 fake cores. 16 threads don't equal twice the processing power. The performance boost 2 threads give is fake. Like smoke and mirrors you can't get power from something that isn't there. As apose to 12 real physical cores with 12 threads. Your getting real world performance for half the cost. Means you could have used 4 socket boards to get 48 real cores of power for the same price as a dual socket intel. Intel doesn't have quad socket boards on newegg, amd does, for 12 real core cpus. Thats more then 50% the power then a intel system. Intel is a joke. AMD beats intel in every way in the server area its physical fact. If anybody thinks 16 cores with 16 fake cores can beat even 24 real cores they get a for being derp, and 48 real cores impossible intel can even compete with, if you think so, you get a double... Also price per performance ratio amd wins by 50%. Thats madness. AMD is the real smart choice for real processing power. Wow intresting intel's 8 core cpu's run at 130w and amd's 12 core cpu's run at 115w. Not only does amd have more power, they use less power consumption with more cores.
Pneumon Blaster Quondam Souls of the Universe corporation R.A.G.E	Posted - 2011.03.25 05:48:00 - [91] Lag did not change. Fix the standings bug finally?
Monkey M3n GK inc. Pandemic Legion	Posted - 2011.03.25 05:57:00 - [92] Edited by: Monkey M3n on 25/03/2011 05:57:14 Why didn't you get the Xeon w3690?
Dek'athor	Posted - 2011.03.25 07:01:00 - [93] Edited by: Dek''athor on 25/03/2011 07:04:22 Hmmm, this may explain why the time from right-clik-> sell until THE TIME the dialog box with the average region item price INCREASED. This is in a ~100 players mission hub, no modules lag, nothing wrong with anything, just market data interrogation takes about 5-15 seconds PER ITEM. I noticed this this in the last few days... Did I understand well that the DB has new SSD technology? Solid State Drive? as in diskless HDD's? I seem to recall some products had issues with the initial read time, i.e. the time it takes from the initial read request until the data gets out there. Once the initial read is out of the way, the data would be read at a awesome speed à making nice love with large file read performance, but absolutely murdering small file read performance. Edit: inserted obligatory emoticon:
Super Whopper I can Has Cheeseburger	Posted - 2011.03.25 07:12:00 - [94] Originally by: Sith LordXXX Edited by: Sith LordXXX on 25/03/2011 05:35:33 Edited by: Sith LordXXX on 25/03/2011 05:30:49 Opteron 6174's would have been a much better upgrade as apose to 8 fake cores in the intel and intel costing twice as much going for $2400, the AMD's have 12 real cores giving far more processing power then a intel chip with only 8 real cores with a marketing gimmic of 8 fake cores. 16 threads don't equal twice the processing power. The performance boost 2 threads give is fake. Like smoke and mirrors you can't get power from something that isn't there. As apose to 12 real physical cores with 12 threads. Your getting real world performance for half the cost. Means you could have used 4 socket boards to get 48 real cores of power for the same price as a dual socket intel. Intel doesn't have quad socket boards on newegg, amd does, for 12 real core cpus. Thats more then 50% the power then a intel system. Intel is a joke. AMD beats intel in every way in the server area its physical fact. If anybody thinks 16 cores with 16 fake cores can beat even 24 real cores they get a for being derp, and 48 real cores impossible intel can even compete with, if you think so, you get a double... Also price per performance ratio amd wins by 50%. Thats madness. AMD is the real smart choice for real processing power. Wow intresting intel's 8 core cpu's run at 130w and amd's 12 core cpu's run at 115w. Not only does amd have more power, they use less power consumption with more cores. The moment you find out why people keep swallowing the Intel propaganda, rather than buy quality you let me know. CCP, you're pretty daft for buying Intel. It's been known for a very long time that AMD are superior but you keep buying Xeon's. You getting kickbacks from Intel for not buying AMD or you just love wasting money?
Erichk Knaar Caldari Noir. Noir. Mercenary Group	Posted - 2011.03.25 08:24:00 - [95] ITT: People who don't get why Intel is leading AMD. http://images.anandtech.com/graphs/amd12core_032610044429/22157.png There's your precious AMD chip in a database test. Other wise, as someone who understands what all of that is: Nice toys :) Seconding the guy above who mentioned Greenplum. Stuff is sick (And cost effective).
Aldariandra Gallente MunsterMunch The 0rphanage	Posted - 2011.03.25 08:44:00 - [96] By the way. Those CPU graphs look rather familiar to me. Quest Foglight Performance Analysis for SQL by any chance? :D
Xenofarion Gallente Swords Horses and Heavy Metal	Posted - 2011.03.25 09:03:00 - [97] Quote: 9 x 300 GB SSD's drives RAID 5 attached via IBM V7000 36 x 600 GB SAS drives RAID 10 attached via IBM V7000 And all connected with an OnBoard-RAID-controller? No, seriously, keep it coming CCP! -- those who can, do those who can't, complain

CCP Yokai	Posted - 2011.03.25 09:15:00 - [98] Originally by: Hawk TT CCP, now you have to look at Intel Xeon X5698 Dual-Core 4.4GHz and X569X @ 4.66GHz... I suppose those would be nice match for the SOL nodes... Spoiler Alert!!! Watch my very short 2 min section of the Keynote today at 15:00... All I can say right now... but there is way! more to come.

Charles Javeroux Gallente INTERSTELLAR CREDIT	Posted - 2011.03.25 09:19:00 - [99] Wohoo \o/...after reading this d-blog, it feels like second X-mas

CCP Yokai	Posted - 2011.03.25 09:20:00 - [100] Edited by: CCP Yokai on 25/03/2011 09:21:06 General Response to "I thought the DB was not under load". It wasn't back then... but it is now. Changes I can think of off the top of my head that affect the DB. Inventory 64 API Evegate Enhanced Backups Cleanup jobs Simple DB size growth We are doing ALOT behind the scenes that needs more power from the DB and we were obviously starting to break it.


CCP Yokai	Posted - 2011.03.25 09:27:00 - [101] Originally by: Korerin Mayul you have four 8g ports per server, rolled into 2 trunks of 16g? - sweet!! what is managing the host level trunking? I havent seen this done in anger before THIS Additionally, be really careful with thin provisioning, I speek form bitter, bitter experience that using that stuff in prod will cause many tears and late night clickeyfests. Only being used for non-production warehousing and such. Just more cool stuff we didn't have before. Daddy likes options you know?

Super Whopper I can Has Cheeseburger	Posted - 2011.03.25 09:28:00 - [102] Originally by: Erichk Knaar ITT: People who don't get why Intel is leading AMD. http://images.anandtech.com/graphs/amd12core_032610044429/22157.png There's your precious AMD chip in a database test. Other wise, as someone who understands what all of that is: Nice toys :) Seconding the guy above who mentioned Greenplum. Stuff is sick (And cost effective). A company that is buying thousands of CPU's only looks at one graph? I'm surprised you didn't post some gaming benchmark.

CCP Yokai	Posted - 2011.03.25 09:29:00 - [103] Originally by: Svizac Marmotov Is Tranquility going to become self-aware? My best guess... 04-15-2011 (I'm having it do my taxes)


CCP Yokai	Posted - 2011.03.25 09:30:00 - [104] Edited by: CCP Yokai on 25/03/2011 09:32:07 Originally by: ModeratedToSilence Originally by: CCP Yokai Originally by: Marconus Orion I'm not very tech savy so for us laymen non-super geeks, please using a car reference, what is the comparison? We going from a Fiesta to a Mustang? We had a Ferrari 1985 288 GTO We have a Ferrari 2011 599 GTO Both great... both fast... the new one is just magic. make sense? Attempting to align computer technology development with automotive development is stupidity. Computer hardware tends to double in its basic attributes every 18-24 months. If we had the same development in motor vehicles we would not be playing internet spaceships - we would be in real life spaceships. Note: I agree with you and this is why I used a 1985 to 2011 comparison for a 2008 to 2011 server change. Seems about right to me. But I get what you mean... hard to compare really... I just like the 599 GTO enough to make up any excuse to use it.

The Snowman Gallente The Ascension	Posted - 2011.03.25 09:32:00 - [105] What happens to all the old equipment? Do you sell it to try recover some of the cost?
Super Whopper I can Has Cheeseburger	Posted - 2011.03.25 09:36:00 - [106] Originally by: CCP Yokai Originally by: Hawk TT CCP, now you have to look at Intel Xeon X5698 Dual-Core 4.4GHz and X569X @ 4.66GHz... I suppose those would be nice match for the SOL nodes... Spoiler Alert!!! Watch my very short 2 min section of the Keynote today at 15:00... All I can say right now... but there is way! more to come. You could also do your bit by not supporting a rogue apartheid regime.
Muul Udonii Minmatar Sick Tight Controlled Chaos	Posted - 2011.03.25 09:47:00 - [107] Yes it's all very nice, but does it have a camera?
Rahnim	Posted - 2011.03.25 09:51:00 - [108] And all this stuff is live now? when did you change this stuff? did it fix the large scale fleet lag?

CCP Yokai	Posted - 2011.03.25 09:53:00 - [109] Originally by: Rahnim And all this stuff is live now? when did you change this stuff? did it fix the large scale fleet lag? Yes, 03-17-2011, NO

Cresalle	Posted - 2011.03.25 09:53:00 - [110] Nice. Now don't light it on fire.
Dr Sheepbringer Gallente Halinallen veroparatiisi Inglorious Carebears	Posted - 2011.03.25 10:24:00 - [111] So the hamsters went from Rambo...probably the third one already to the bloodiest movie ever Rambo 4 then? Just as a ballpart...how much does that stuff cost? I mean I'm not talking about just one part etc. In a whole, how much does a Rambo 4 degree hamster mayhem cost? Originally by: CCP Shadow Dr. Sheepbringer -- It's not that kind of horn.
Fredd Mosibfu	Posted - 2011.03.25 10:30:00 - [112] woah im learning to be a network manager and i think i took the right education.. these stats make me horny if only i could do my internship at ccp but its so far away
SwissChris1	Posted - 2011.03.25 10:32:00 - [113] Wow I just sent those specs around the office and everyone is SUPER jelous amazing stuff...
Viktor Rasmussen Gallente Eagle's Club	Posted - 2011.03.25 12:40:00 - [114] simply awesome!
drinking12many Minmatar Nine Inch Ninja Corp	Posted - 2011.03.25 12:50:00 - [115] Edited by: drinking12many on 25/03/2011 12:51:55 Originally by: Sith LordXXX Edited by: Sith LordXXX on 25/03/2011 05:35:33 Edited by: Sith LordXXX on 25/03/2011 05:30:49 Opteron 6174's would have been a much better upgrade as apose to 8 fake cores in the intel and intel costing twice as much going for $2400, the AMD's have 12 real cores giving far more processing power then a intel chip with only 8 real cores with a marketing gimmic of 8 fake cores. 16 threads don't equal twice the processing power. The performance boost 2 threads give is fake. Like smoke and mirrors you can't get power from something that isn't there. As apose to 12 real physical cores with 12 threads. Your getting real world performance for half the cost. Means you could have used 4 socket boards to get 48 real cores of power for the same price as a dual socket intel. Intel doesn't have quad socket boards on newegg, amd does, for 12 real core cpus. Thats more then 50% the power then a intel system. Intel is a joke. AMD beats intel in every way in the server area its physical fact. If anybody thinks 16 cores with 16 fake cores can beat even 24 real cores they get a for being derp, and 48 real cores impossible intel can even compete with, if you think so, you get a double... Also price per performance ratio amd wins by 50%. Thats madness. AMD is the real smart choice for real processing power. Wow intresting intel's 8 core cpu's run at 130w and amd's 12 core cpu's run at 115w. Not only does amd have more power, they use less power consumption with more cores. Your right real cores > hyperthreading but thats not to say performance gains wont be seen. Also its can be very transaction database load dependent. We had done benchmarking on our esx servers with hyperthreading off and in the past you were usually better off leaving it off. Now days there is a definite improvement in the performance I think we saw somewhere around 30% or so. Now is that to say they should have thought about amd yes they probably did but since it appears they go with IBM hardware which I believe does offer AMD they may have been constrained or helped by some other cpu feature/ kind of database transactions they do that they went with the Intel which right now for most loads tends to be better. Even in my company we tend to go with Intel even though my preference is AMD from a cost performance standpoint. Though my piddly database server only has 8 cores no HT and 16GB of ram cries and just standard raid 10 SAN storage we only get about 5000-7000 IOPS with the caching and all and during inventory updates my DB cries. Maybe next time I can talk them into more RAM and some flash san storage if they give me the money :(
Illwill Bill Reign of Anarchy	Posted - 2011.03.25 12:52:00 - [116] I came. Originally by: CCP Zymurgist Revenge is a dish best served with auto-cannons.
EliteSlave Minmatar Macabre Votum Morsus Mihi	Posted - 2011.03.25 13:07:00 - [117] Originally by: Sith LordXXX Edited by: Sith LordXXX on 25/03/2011 05:35:33 Edited by: Sith LordXXX on 25/03/2011 05:30:49 Opteron 6174's would have been a much better upgrade as apose to 8 fake cores in the intel and intel costing twice as much going for $2400, the AMD's have 12 real cores giving far more processing power then a intel chip with only 8 real cores with a marketing gimmic of 8 fake cores. 16 threads don't equal twice the processing power. The performance boost 2 threads give is fake. Like smoke and mirrors you can't get power from something that isn't there. As apose to 12 real physical cores with 12 threads. Your getting real world performance for half the cost. Means you could have used 4 socket boards to get 48 real cores of power for the same price as a dual socket intel. Intel doesn't have quad socket boards on newegg, amd does, for 12 real core cpus. Thats more then 50% the power then a intel system. Intel is a joke. AMD beats intel in every way in the server area its physical fact. If anybody thinks 16 cores with 16 fake cores can beat even 24 real cores they get a for being derp, and 48 real cores impossible intel can even compete with, if you think so, you get a double... Also price per performance ratio amd wins by 50%. Thats madness. AMD is the real smart choice for real processing power. Wow intresting intel's 8 core cpu's run at 130w and amd's 12 core cpu's run at 115w. Not only does amd have more power, they use less power consumption with more cores. Actually having a HT is very beneficial, as you state that it is a "Fake core" yes, but its only a "FAKE core" as the CPU actually has an extra register per core thus giving a second lane of data to be held for processing, The CPU itself is not to blame for this trick as it is merely the way current x86-64 operating systems are written. Also AMD does not have the horsepower that INTEL has when it comes to server grade chipsets. Nothing to do with kickbacks. the job i work for isnt as big as CCP when it comes to enterprise level but considering we have 100 physical virtual host's which are currently all AMD, while AMD was nice when it was not number crunching we slowly started to grow in the past year which really showed AMD's weakness when it comes to doing the crunching it just doesnt have the horsepower that is needed. CPU clock cycles have been pegged at near 70% the past year and have not come down at all. While we brought in 10 INTEL boxes of the same clock rates migrated a few servers onto the boxes, the cycles have dropped down to mere 25-30% which means I can load more VM's on it.
Ban Doga	Posted - 2011.03.25 13:53:00 - [118] Originally by: Tinkanium Originally by: Ban Doga It is also true that there is always a limiting factor and addressing one will usually make something else a limiting factor. Yeah man. In fact based on what you yourself said you're already wrong. There's always a limiting factor. If you fix that, and it is no longer the limiting factor, then by the logic that there's always a limiting factor something else must become the limiting factor. 1) there can be several limiting factors at the same time ("a" does not imply "exactly one") 2) addressing something does not imply fixing it ("better" does not imply "best") Also, "usually" is a superset of "always" so what I said was not wrong at all. You might call it inaccurate but it isn't even that: Just assume you have 2 limiting factors at the same time (eg you can't run from New York to London in 2 hours because humans can't run that fast AND there's a huge ocean between those two), if you eliminate 1 of those limits (let's say you find a way to run over water) you'd still have one limiting factor (you just can't run that fast). But that's not a new limiting factor. You just "fixed" the other one. Originally by: Tinkanium Even in the dev blogs there's always someone. Seriously. Incredibly long post containing nothing but quotes and seemingly intelligent logic that basically amounts to nothing. Please do save us all some time and just read along k? It's not my fault you decide to read something and think "that is wrong" just because you don't understand the simple difference between "a" and "the" while trying to argue about logic. And if you think that's an "incredibly long post" I'm actually surprised you managed to read the devblog. Just stick to reading the shorter posting if you the long ones bother you k?
Callea Melin Amarr MevilD	Posted - 2011.03.25 14:22:00 - [119] Edited by: Callea Melin on 25/03/2011 14:25:17 Hi there, well I've been following all of CCP's blogs so far and sometimes got some ideas even for our business. We've been running for almost 2 years now on 8 Quad Core AMD Opteron 8359 2.3 GHZ processors for our main database, and when we went live with it, had about 20% CPU peak. Well, we underestimated IO and User growth quite a bit at that time, so now we are struggling for at least 6 month with a usual 90% cpu and constant IO bottlenecks. New storage is on its way, but even with this the CPU's are well overloaded nowadays. The rest is a similar system, Windows 2008 R2 Enterprise 64, Enterprise SQL Server 2008 R2 CU2 64, 256 GB Ram and so on. Using this for a good 2 years now. Our main Database is about 2 TB large now with a more or less constant growth of ~1.1 GB/day. Im also really keen on getting this new HP P9500 storage with a rather small load of 19x8 15k rpm 2.5" SAS disks up and running for prod here. Can't post graphs here unfortunatly, just sharing some technical stuff since I like to :) Cheers ps: For all that want to have such stuff, yes it is indeed very nice to work with it... but getting used to such stuff tends to be expensive for private stuff, since you want bigger and better stuff for yourself too :P
Valeroth Kyarmentari	Posted - 2011.03.25 15:45:00 - [120] Very sexy graphs.
adriaans Amarr Ankaa. Nair Al-Zaurak	Posted - 2011.03.25 17:28:00 - [121] Sweet.... --signature-- F.CS boost: Here Vid: Link
Tinkanium	Posted - 2011.03.25 18:10:00 - [122] Originally by: The Snowman What happens to all the old equipment? Do you sell it to try recover some of the cost? Note the close temporal proximity of this deployment to the addition of Duality to the server repertoire. Opportunists.
3nLiGteN3d On3 The Minutemen Gentlemen's Agreement	Posted - 2011.03.25 22:31:00 - [123] Thanks for the update, I always wondered how you all were handling the traffic on your SQL cluster. I'm a Microsoft Consultant for partner organizations in the US, so I'm all too familiar with the SQL layout and lack of compute clustering with MSSQL. I was a bit surprised to hear you were literally running all of TQ on one beefy (that's an understatement) SQL box with a failover SQL cluster. I love what you did with the SSD SAN, and I'm sure that given the amount of traffic you guys pull, that is really the only way to go. My question for the SQL part is what version of Microsoft SQL server are you running in the node? SQL 2008 R2 Enterprise? I saw the bit about HyperThreading and thought you might still be using a 32-bit Windows 2008 OS. Just curious, thanks for the geek blog
Serret Nevets Fusion Enterprises Ltd Morsus Mihi	Posted - 2011.03.26 01:08:00 - [124] Its too bad this doesn't make a difference for lag.
Herr Nerdstrom Caldari Havoc Violence and Chaos BricK sQuAD.	Posted - 2011.03.26 01:55:00 - [125] With the improved database failover, does this mean that nodes will soon be able to get reinforced in realtime, perhaps even automatically via load sensing?
Mister Rocknrolla	Posted - 2011.03.26 02:20:00 - [126] So, to a layman, is this considered "cutting-edge" or just "really, REALLY good" stuff? Just wondering if development of this sort of computing capacity lends itself to other industries? Does CCP interact with other industries to help them in developing this kind of computing power? TL;DR: Could TQ find a cure for cancer, an alternative to fossil fuels and an efficient way to travel across the galaxy if we weren't using it to play games?
Erichk Knaar Caldari Noir. Noir. Mercenary Group	Posted - 2011.03.26 02:28:00 - [127] Originally by: Mister Rocknrolla So, to a layman, is this considered "cutting-edge" or just "really, REALLY good" stuff? Just wondering if development of this sort of computing capacity lends itself to other industries? Does CCP interact with other industries to help them in developing this kind of computing power? TL;DR: Could TQ find a cure for cancer, an alternative to fossil fuels and an efficient way to travel across the galaxy if we weren't using it to play games? For it's purpose, it's bleeding edge. There have been other things done in different spaces, but for gaming, CCP is way ahead.

CCP Yokai	Posted - 2011.03.26 12:08:00 - [128] Originally by: Erichk Knaar Originally by: Mister Rocknrolla So, to a layman, is this considered "cutting-edge" or just "really, REALLY good" stuff? Just wondering if development of this sort of computing capacity lends itself to other industries? Does CCP interact with other industries to help them in developing this kind of computing power? TL;DR: Could TQ find a cure for cancer, an alternative to fossil fuels and an efficient way to travel across the galaxy if we weren't using it to play games? For it's purpose, it's bleeding edge. There have been other things done in different spaces, but for gaming, CCP is way ahead. What that guy said... It isn't made of stuff only found in Area 51 Silos... but for a DB... it is way at the top and edge of what can be done. The V7000 is very very new and we have been testing since it released.

Gabriel Karade Gallente Noir. Noir. Mercenary Group	Posted - 2011.03.26 14:49:00 - [129] ...Winsauce with a cherry of awesome on top -------------- Video - 'War-Machine'
Serret Nevets Fusion Enterprises Ltd Morsus Mihi	Posted - 2011.03.27 14:02:00 - [130] Yeah... Well unfortunately we/I, that have played eve for a while now (~5 years), keep hearing through the years of all these great things CCP is doing to reduce lag. Will it really make a difference? Probably not.
Norxil	Posted - 2011.03.27 21:03:00 - [131] Quote: TL;DR: Could TQ find a cure for cancer, an alternative to fossil fuels and an efficient way to travel across the galaxy if we weren't using it to play games? The "cure for cancer" and "SETI" programs love other kind of power: GPU power.. Thats because CPU's arent very good at floating "stuff" Oh and a bit offtopic: fossil fuels arent from fossils: http://www.wnd.com/?pageId=45838
Malcanis Caldari Alcohlics Anonymous Scum Alliance	Posted - 2011.03.28 12:42:00 - [132] Originally by: Serret Nevets Yeah... Well unfortunately we/I, that have played eve for a while now (~5 years), keep hearing through the years of all these great things CCP is doing to reduce lag. Will it really make a difference? Probably not. Yeah I remember those awesome lag-free 700-man battles we used to get in 2006 as well. Seriously, if you can't stand the lag in massive fleet battles, why be a member of an alliance who specifically and deliberately use node crashing as a "tactic"? Peak TQ performance is pretty good right now. Peak fleet size has increased far faster, in part because of people like your own fleet commanders. Blame them as well if you're going to blame CCP. Malcanis' Law: Whenever a mechanics change is proposed on behalf of "new players", that change is always to the overwhelming advantage of richer, older players.
Darteis Elosia Gallente PHOENIX 2ND C.A.G. DEM0N HUNTERS	Posted - 2011.04.06 15:02:00 - [133] Awesome new servers! Must admit, i had to fap after reading all that tech ****. So hmm, one can only wonder what the next iteration will be like.. SOL blades beeing nVidia tesla based? Because that would sort of kick ass. But it would also require you rewrite the game from ground up hrmm? My only logic here being "if it can simulate an atomic bomb detonating in the ground, it must be good enough for fleetfights!"
Voith	Posted - 2011.04.07 02:54:00 - [134] I spotted a few errors that caused my Database nerd rage to... well rage. 1) Buffer Cache hit ratio is mostly irrelevant to database performance. 2) RAM isn't the biggest performance factor for Databases, knowing how to use a database is. See the Bounty change a few years ago. 3) Anyone named "Denormalized" shouldn't be allowed with in 500 meters of a database. Number 3 is the most important one.

Pages: 1 2 3 4 5 :: [one page]

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.