EVE Search - CCP Valar Why MSSQL vs MySQL

All Channels

EVE General Discussion

CCP Valar Why MSSQL vs MySQL

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Author	Thread Statistics \| Show CCP posts - 1 post(s)
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 02:26:00 - [1] Edited by: Eventy One on 11/06/2009 02:31:35 As far as I've seen MySQL 5.x's performance is far better than MySQL 4.x's which would have been around when EVE started. Although MSSQL outperformed MySQL 4, it didn't so by much, but as far as I can tell MySQL 5.x's performance is generally better than MSSQL's especially on 64-bit machines. But with MySQL 5.x ability to change out table formats (MyISAM, BerkeleyDB) gives MySQL 5.x the ability to use MyISAM rather than the default BerkeleyDB. MyISAM is far more efficient, being more compact on disk and less demand on CPU cycle / memory why stick with MSSQL? Is the idea of converting tables from MSSQL over to MySQL too frightening or is there some other reason? (I would think stackless python would work faster on a *nix machine also) It would seem to be in-line with your policy of "need for speed", so its reasonable to ask why you're not moving in this direction. PS CCP Valar, your old Jove character was better than your current one.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 03:11:00 - [2] Originally by: Durzel MySQL has come a long way but at the end of the day MSSQL is still a tried-and-tested enterprise-level solution. I could have swore - it appeared as though you were suggesting MySQL wasn't enterprise grade. I must be mistaken.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 15:23:00 - [3] Anyone who says "MySQL simply isn't fast enough to justify the risks" either hasn't looked at MySQL (recently) or hasn't bench-marked it against other DBs. MySQL is more versatile, and I personally believe more extensible, than MSSQL all while being more efficient. I'm still curious to know what internal discussions the CCP DB folks have on the matter.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 15:51:00 - [4] Originally by: SpaceSquirrels I don't mean to sound to much like an A-hole, but that's just your opinion. Well - not quite. There are a ton of objective third-party metrics on the internet that show the assertion to be true. Then again there are the large multi-nationals using MySQL (and postgres). Originally by: SpaceSquirrels You're some random guy on the ol interwebs. What are your qualifications, experience etc in the "the biz"? If you are in the IT field you should know platforms deployed on a scale required such as eve there's more involved than just speed, and cost. Maybe you are in fact right about this however as this an MMO game forum you have 0 credibility like everyone else here. You also sound like a salesman at this point as well. Logically, to attack the person making a point rather than the point itself is a fallacy. Even if I'm some random guy in the ol interwebs, and a janitor at boot, doesn't speak to the assertion that MySQL is as valid a DB as MSSQL. At the heart of the question, wasn't a peeing contest with another internet nobody, but was instead an interest in ways CCP might better make use of infiniband. This internet nobody has first hand experience seeing the improvement a memory laden machine with Infiniband's Sockets Direct Protocol (SDP) can have on a DB (all at 20 Gbps). Not bad for a janitor eh?
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 16:09:00 - [5] Edited by: Eventy One on 11/06/2009 16:12:29 Originally by: Durzel No offence but a PDF that doesn't even go into specific performance advantages relating to a database architecture (or even mention MySQL, MSSQL, etc) together with an OP littered with ambiguous assertions like "as far as I can tell", "generally better", "I would think", etc - it doesn't really reflect well. Hmm - lets see. Assuming the average DB app's performance is primarily dependent upon memory accessibility, and secondarily limited by IO - performance can be improved how? Maxing available memory Reducing I/O latency table/index optimization Ok so maxing performance comes down to buying memory, performing table/index optimization on a daily basis, and that leaves what? That's right - the issue of I/O latency. It doesn't seem you've given this issue a whole lot of thought given your off-the-mark dismissal. BTW I'm fairly confident MySQL gives you the option of directly accessing / specifying read/write methods be it shared memory or optimized sockets. How much access does MSSQL give you over these things?
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 16:15:00 - [6] Originally by: Professor Dumbledore Originally by: Durzel MySQL has come a long way but at the end of the day MSSQL is still a tried-and-tested enterprise-level solution. You do realize google uses mysql for lots of stuff right? QFT. Thanks for mentioning that. I wasn't going to mention Google or some of the largest stock market trading companies that use MySQL enterprise, b/c more often than not the Microsoft fan club is in denial about LAMP. I'm glad someone did however.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 16:25:00 - [7] Edited by: Eventy One on 11/06/2009 16:26:08 Originally by: Durzel Yup. I never said MySQL was useless did I? They use it for (among other things) their Adwords system. If memory serves they use a database of their own design for their key technologies. Which brings up another point - that DB technology aside, table methods and hardware, the actual application itself often, has as much to do with performance as everything else. (Which is why, when obtaining metrics, it makes sense to use one's actual DB for comparison purposes) From what I understand, EVE has many (perhaps many, many) small tables rather than some monolithic implementation.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 17:40:00 - [8] Edited by: Eventy One on 11/06/2009 17:40:51 Originally by: Alcoholic Bird Oh and MySQL simply does not scale as much as MSSQL. Simply put Mysql has a processor limit and MSSQL i think does not, but its more technical than that. - AB Oh dear - perhaps you need to get your facts straight. The original post was a request To CCP, On their reasons for going with the DB they did. If you're going to accuse me of not having my facts straight - mind showing me where they aren't? I think there's a bit of that MS FUD being sewn here (fewer facts than emotions)
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 18:03:00 - [9] Edited by: Eventy One on 11/06/2009 18:03:06 Originally by: Alcoholic Bird No MS fud about. I don't hate MySQL and personally won't use it but everything is task specific. Would you tow an oil tanker with a speed boat? OH an one little fact for you, loads on the net. MySQL can only scale up to 16 threads, so 16 cores or 4way 4cores. MSSQL can easily scale over more than 64 physical processors and terabytes of ram, it's how it was designed. ... except that CCP's cluster is not a monolithic shared memory machine. Its a cluster.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 18:32:00 - [10] Originally by: Alcoholic Bird Maybe CCP is not running a huge monolithic server but according to their blogs the database server is pretty beefy and the new one by the sounds of it has some nice upgrade space too. Could be - but I'd seriously like to hear from CCP why/how they've architected the things the why they've had. So far, the most substantial thing we've seen in this thread, is the suggestion, that CCP's thinking could have been influenced primarily by support contract considerations.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 20:34:00 - [11] Edited by: Eventy One on 11/06/2009 20:36:25 Originally by: RC Denton Based on the fact that they are buying the biggest RAMSAN they can find their main problem is IO performance on the SAN. The DB engine may have some effect on that, but once you have things tuned as much as possible you're stuck with the raw output that the hardware can provide you and that's going to be your limiting factor Good points. This last one, about DB engine performance ultimately being tied to raw output of hardware, is why I was thinking making use of the sockets direct protocol provided by Infiniband would be a good idea. What is the maximum throughput of a normal disk array - something less than 3 Gbps or something, 2.4 Gbps perhaps? Regardless, with the exception of having absolutely everything supported by RAMSAN, the throughput to disk could happen at 20 Gbps. (What is the throughput of RAMSAN BTW - is it akin to memory calls?) So, if CCP is thinking about having all DB supported by RAMSAN, fine. If they are doing some type of load-balanced staged/cached approach it could be something like RAMSAN cache in front of infiniband to disk using SDP. Someone suggested Postgres above, and I agree Postgres is generally better than MySQL, but evenso, Postgres or MySQL are better for Infiniband SDP apps than MSSQL. I believe, when CCP started this business, their decision was the right one. I'm not so sure it still is given where technology has gone.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 21:07:00 - [12] Edited by: Eventy One on 11/06/2009 21:09:41 Originally by: Lost Hamster Now 5 year have passed. MySQL have developed a lot, but I wouldn't trust a 1.5 terabyte database on it. And why change a well known, good enterprise level architecture to something else? There is no gain on it. 1.5T If that's true - Wow! Nice DB, down boy - down! Agreed - changing DB's would be a big task and a conservative approach is is preferable. However, CCP's "need for speed" policy has looked at many other implementation decisions and a-hem corrected them. While looking at bottle necks, code based limitations have received scrutiny, but it seems (to me at least), the most obvious one, has simply had better hardware thrown at it. If they are going to ultimately reduce the bottle necks - perhaps the most obvious one, the Database, should be re-examined as well, with respect to its actual implementation, and not just the hardware running it - to see if the risk of a DB migration might actually pay larger dividends still. A conservative approach can be ignored if there are bigger gains to be made, given the risk involved.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.12 16:36:00 - [13] Edited by: Eventy One on 12/06/2009 16:37:20 Originally by: Ptarmigent Also, the mssql isn't broken so it doesn't need fixing! The point wasn't that mssql is broke. The point was, what is the most appropriate tool for the job given other considerations? It could be mssql is. I suggest giving MySQL a relook. Others have disagreed instead suggesting Oracle, and Postgres. Fair points all. Fact is, CCP is very quiet on their thinking when it comes to DB. Someone suggested it is because their DB implementation is proprietary, a competitive advantage. Given all of the effort to obtain speed - why not engage the community in the thinking about DB or clearly identify it as something they're not willing to discuss publicly. As it stands there have been a few threads on the issue; this one's up to 3 pages, and still its a bit of a black box.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.12 18:50:00 - [14] Originally by: Dr Slaughter Oh, and EleventyOne, the database really isn't their problem at the moment. It may become so again when they finally get Infiniband working as player state IO to the database is bound to come much quicker. One would guess they're going to try to move player state from node to node over the IB connections and only update the database with critical changes. Good point. I assume stackless python's real value is making player state (or any state really) really flat and more accessible to changes. I could see how I/O client / server interactions could also benefit from I/B. You suggest they may actually be able to migrate player state from node to node and minimize db writes; that's a very interesting suggestion! Originally by: Dr Slaughter It would be really interesting to get a comment from the HPC cell how their work is progressing on the SOL service re-factoring and if they've managed to get two IB connected nodes passing data (other than IP over IB which would be cheating) using shared memory etc. Agreed. One of the things that interests me about how CCP handles the back end is the HPC implementation (having an interest in HPC). Now that you mention it, I can see that using IB (sockets direct) will have larger gains on things other than DB access; Such as piping IB sockets directly to stackless python channels, giving persistent in-game items more responsive interactions with each other. It would be nice to hear more from the HPC cell (if there is such a cell).
Eventy One Magellan Exploration and Survey	Posted - 2009.06.12 21:27:00 - [15] Edited by: Eventy One on 12/06/2009 21:27:41 Originally by: Nisd I really think every one should read some of this: http://www.microsoft.com/sqlserver/2008/en/us/compare.aspx Oh! Its at microsoft's site - gotta be neutral and objective therefore. Sorry, I prefer 3rd party opinions not associated with MySQL, MSSQL, Oracle etc.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.14 00:15:00 - [16] Originally by: Maglorre Anyone suggesting that replacing the DB backend is a suitable option at this stage has no idea what they are talking about. This is an exaggeration. First, a DB conversion from one format to another is not difficult. Secondly, since most DB's offer somewhat similar functionality, any code changes that need to be done are also, doable. No one was suggesting such a change HAD to be done. What was suggested was a DB change shouldn't be excluded if CCP was seriously committed to making speed improvements all around. CCP hasn't weighed in on the matter, so it is merely academic at this point.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.14 05:50:00 - [17] Originally by: Tara'Quoya Rax This entire thread is irrelevant. Can't be. It managed to get you to post here.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.14 06:20:00 - [18] Originally by: Phantom Slave Since there's already a thread open talking about CCP's network architecture (to a point), I guess I'll ask my questions here. Has CCP implemented Infiniband? If so, has it been fully integrated? As far as I'm aware they are still working on it. A number of people have asked CCP for updates however. Few have been given - so its still in the works it appears.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.14 23:34:00 - [19] Edited by: Eventy One on 14/06/2009 23:34:08 Originally by: Mioelnir Oracle and IBM DB2 might have been a contender (probably hardware issues), PostgreSQL may have been a contender (mirroring was edgy back then), but MySQL certainly was not. In the world of databases, you do a lot of black magic and voodoo, putting your firstborn son into a really undiserable position, but you do not sacrifice correctness for speed if you want to stay around. From the responses I'm getting here, I'm kind of getting the idea Oracle, and PostgreSQL might have been good contenders. I did consider Postgres, knowing it scales well and had features MySQL didn't see until recently. I'm also aware that alot of Medical dbs that place reliability high on the list of desired features, use Oracle. I'm not sure if CCP has considered reviewing their DB choice in terms of their need for speed. Some people just seem offended at the suggestion CCP should abandon MSSQL (without considering if there are good reasons to do so).
Eventy One Magellan Exploration and Survey	Posted - 2009.06.15 14:20:00 - [20] Originally by: ceaon i am not a expert in this but dint oracle buy SUN Something like that.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.15 19:28:00 - [21] Originally by: Kel Nissa But to be honest, trying to switch from mssql to mysql just because you a) hate microsoft or b) have no clue about real databases is simply a joke. Except that, no one suggested switching just because Microsoft was detestable. Nor was cluelessness about databases the reason. This thread is about asking CCP to tell us if the need for speed policy has extended as far as DB's or if they've looked at other ones.
Eventy One Magellan Exploration and Survey THAT Alliance	Posted - 2009.06.17 23:48:00 - [22] Originally by: Sidrat Flush I believe what we all need is a new tech post from CCP direct. List as many variables and tech-statements as possible, and if you feel like being really evil throw in some management speak goblede**** too. Well, yes, a new tech post from CCP would be nice .. but I'd be happy to forego the management speak Give the dirt on EVE's current and future state of DB zen.
Eventy One Magellan Exploration and Survey THAT Alliance	Posted - 2009.06.19 14:12:00 - [23] Thanks for that CCP Valar. There you have it.

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.