EVE Search - CCP Valar Why MSSQL vs MySQL

All Channels

EVE General Discussion

CCP Valar Why MSSQL vs MySQL

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 [2] 3 4 5 :: one page
Author	Thread Statistics \| Show CCP posts - 1 post(s)
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 16:09:00 - [31] Edited by: Eventy One on 11/06/2009 16:12:29 Originally by: Durzel No offence but a PDF that doesn't even go into specific performance advantages relating to a database architecture (or even mention MySQL, MSSQL, etc) together with an OP littered with ambiguous assertions like "as far as I can tell", "generally better", "I would think", etc - it doesn't really reflect well. Hmm - lets see. Assuming the average DB app's performance is primarily dependent upon memory accessibility, and secondarily limited by IO - performance can be improved how? Maxing available memory Reducing I/O latency table/index optimization Ok so maxing performance comes down to buying memory, performing table/index optimization on a daily basis, and that leaves what? That's right - the issue of I/O latency. It doesn't seem you've given this issue a whole lot of thought given your off-the-mark dismissal. BTW I'm fairly confident MySQL gives you the option of directly accessing / specifying read/write methods be it shared memory or optimized sockets. How much access does MSSQL give you over these things?
Ciylana Nemamiah Gallente Aliastra	Posted - 2009.06.11 16:10:00 - [32] Originally by: Eventy One Edited by: Eventy One on 11/06/2009 02:31:35 As far as I've seen MySQL 5.x's performance is far better than MySQL 4.x's which would have been around when EVE started. Although MSSQL outperformed MySQL 4, it didn't so by much, but as far as I can tell MySQL 5.x's performance is generally better than MSSQL's especially on 64-bit machines. But with MySQL 5.x ability to change out table formats (MyISAM, BerkeleyDB) gives MySQL 5.x the ability to use MyISAM rather than the default BerkeleyDB. MyISAM is far more efficient, being more compact on disk and less demand on CPU cycle / memory why stick with MSSQL? Is the idea of converting tables from MSSQL over to MySQL too frightening or is there some other reason? (I would think stackless python would work faster on a nix machine also) MySQL just isn't that* good, only reason it is used so much is because it is cross platform and free. For the kind of heavy lifting CCP requires you are going to need a real database server like MSSQL or Oracle. I work the IT for a business-home delivery company and we have recently done a major software overhaul. MySQL simply fell over when trying to handle our workload of around 150 queries a second, I bet eve is far more demanding than that. .... Our database isnt a trillion gigaterraquadabytes either! It would seem to be in-line with your policy of "need for speed", so its reasonable to ask why you're not moving in this direction. PS CCP Valar, your old Jove character was better than your current one.
Professor Dumbledore Amarr GoonFleet GoonSwarm	Posted - 2009.06.11 16:11:00 - [33] Originally by: Durzel MySQL has come a long way but at the end of the day MSSQL is still a tried-and-tested enterprise-level solution. You do realize google uses mysql for lots of stuff right?
Mashie Saldana BFG Tech	Posted - 2009.06.11 16:15:00 - [34] It's quite simple really, single vendor for development pltaform, server OS and database. Any risk of having multiple vendors bickering and pointing fingers at each others when things break isn't worth it.
Durzel The Xenodus Initiative.	Posted - 2009.06.11 16:15:00 - [35] ScarySquirrel: You're not wrong about there being people who won't use non-MS products, or spreading FUD. I've come across plenty of people who even when faced with "better" open-source alternatives will pay over the odds for licences to remain tied to Microsoft. That being said I actually think the performance aspect is a bit of a smokescreen. I know from first hand experience that Oracle and MSSQL, with the right hardware and tailored installation (which is very important) are very performant. There may be things that MySQL or another DB technology does better, or faster. The big thing from the point of view of a DBA is support. Paying Oracle or Microsoft a licence means that when the s*t hits the fan I (or the customer) can contact them 24/7/365 and have someone who can tell us exactly* what needs to be logged & what needs to be done to get the customer back up and running in the event of a crisis. For any enterprise that lives and dies by the availability of their database you can't put a price on that service. In all of the projects I've been involved with where there has been an expectation/requirement from the customer that they have uninterrupted & guaranteed service, even in the event of hardware failure, there has never been any doubt that the correct solution that every party involved can be happy with is either MSSQL or Oracle. That's not a diss on MySQL in any way, it's just what I've personally experienced.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 16:15:00 - [36] Originally by: Professor Dumbledore Originally by: Durzel MySQL has come a long way but at the end of the day MSSQL is still a tried-and-tested enterprise-level solution. You do realize google uses mysql for lots of stuff right? QFT. Thanks for mentioning that. I wasn't going to mention Google or some of the largest stock market trading companies that use MySQL enterprise, b/c more often than not the Microsoft fan club is in denial about LAMP. I'm glad someone did however.
Durzel The Xenodus Initiative.	Posted - 2009.06.11 16:18:00 - [37] Originally by: Professor Dumbledore Originally by: Durzel MySQL has come a long way but at the end of the day MSSQL is still a tried-and-tested enterprise-level solution. You do realize google uses mysql for lots of stuff right? Yup. I never said MySQL was useless did I? They use it for (among other things) their Adwords system. If memory serves they use a database of their own design for their key technologies.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 16:25:00 - [38] Edited by: Eventy One on 11/06/2009 16:26:08 Originally by: Durzel Yup. I never said MySQL was useless did I? They use it for (among other things) their Adwords system. If memory serves they use a database of their own design for their key technologies. Which brings up another point - that DB technology aside, table methods and hardware, the actual application itself often, has as much to do with performance as everything else. (Which is why, when obtaining metrics, it makes sense to use one's actual DB for comparison purposes) From what I understand, EVE has many (perhaps many, many) small tables rather than some monolithic implementation.
Playing Eve	Posted - 2009.06.11 16:28:00 - [39] Originally by: Durzel The big thing from the point of view of a DBA is support. Paying Oracle or Microsoft a licence means that when the s*t hits the fan I (or the customer) can contact them 24/7/365 and have someone who can tell us exactly* what needs to be logged & what needs to be done to get the customer back up and running in the event of a crisis. For any enterprise that lives and dies by the availability of their database you can't put a price on that service. Sun Microsystems offers 24x7 support for MySQL as well as consulting support. Sure, if you're downloading MySQL for free and running it that way, you're not going to get that kind of support. But I wouldn't expect CCP not to pay for support for their systems.
Alcoholic Bird	Posted - 2009.06.11 17:25:00 - [40] Edited by: Alcoholic Bird on 11/06/2009 17:27:41 Pointed to the OP. Please get your facts straight before doing a DB vs DB. The moment someone says DB vs DB it is pretty much garbage from the start. I assume CCP chose MSSQL not just because of performance but many many other factors that MySQL are still implementing/fixing. Another thing that leads me to think about you not having your head screwed on is EVE goes through literally millions and millions and millions of queries/updates in a day. With TABLE level locking on MyISAM you are implementing a fail system since MyISAM will never keep up. InnoDB maybe but there are still performance issues with it too. Not saying MySQL is terrible and maybe they can be used for large systems but the maybe not large and extreme speed demanding systems. Synthetic benchmarks should never be taken at face value. Speed aside maybe MSSQL has features that CCP use that mySQL simply does not have, ever thought of that. IMO although good for read speed on small things MyISAM is the biggest ever fail in MySQL for anything bigger than 100 rows <--- /sarcasm on Oh and MySQL simply does not scale as much as MSSQL. Simply put Mysql has a processor limit and MSSQL i think does not, but its more technical than that. - AB
Illectroculus Defined	Posted - 2009.06.11 17:39:00 - [41] Edited by: Illectroculus Defined on 11/06/2009 17:41:05 What I've generall seen is that MySql owns everything out there now as long as you stay away from transactions (if you really need transactions then go for Oracle). There are plenty of giant sites which show that mysql scales - I work at a site which has 40million users and over 100million uniques per month and mysql has never let us down (we laso have an oracle system from a company we acquired, but it doesn't get hit nearly so hard). MSSQL is a pretty poor contender these days, but, most crucially of all for CCP it's what they're using. Unless it's going to give giant improvements it's unlikely to be worth the developer time to shift architectures. Bigger and better hardware is always cheaper than a team of developers salaries.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 17:40:00 - [42] Edited by: Eventy One on 11/06/2009 17:40:51 Originally by: Alcoholic Bird Oh and MySQL simply does not scale as much as MSSQL. Simply put Mysql has a processor limit and MSSQL i think does not, but its more technical than that. - AB Oh dear - perhaps you need to get your facts straight. The original post was a request To CCP, On their reasons for going with the DB they did. If you're going to accuse me of not having my facts straight - mind showing me where they aren't? I think there's a bit of that MS FUD being sewn here (fewer facts than emotions)
Alcoholic Bird	Posted - 2009.06.11 17:56:00 - [43] Originally by: Eventy One Edited by: Eventy One on 11/06/2009 17:40:51 Originally by: Alcoholic Bird Oh and MySQL simply does not scale as much as MSSQL. Simply put Mysql has a processor limit and MSSQL i think does not, but its more technical than that. - AB Oh dear - perhaps you need to get your facts straight. The original post was a request To CCP, On their reasons for going with the DB they did. If you're going to accuse me of not having my facts straight - mind showing me where they aren't? I think there's a bit of that MS FUD being sewn here (fewer facts than emotions) No MS fud about. I don't hate MySQL and personally won't use it but everything is task specific. Would you tow an oil tanker with a speed boat? OH an one little fact for you, loads on the net. MySQL can only scale up to 16 threads, so 16 cores or 4way 4cores. MSSQL can easily scale over more than 64 physical processors and terabytes of ram, it's how it was designed.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 18:03:00 - [44] Edited by: Eventy One on 11/06/2009 18:03:06 Originally by: Alcoholic Bird No MS fud about. I don't hate MySQL and personally won't use it but everything is task specific. Would you tow an oil tanker with a speed boat? OH an one little fact for you, loads on the net. MySQL can only scale up to 16 threads, so 16 cores or 4way 4cores. MSSQL can easily scale over more than 64 physical processors and terabytes of ram, it's how it was designed. ... except that CCP's cluster is not a monolithic shared memory machine. Its a cluster.
Alcoholic Bird	Posted - 2009.06.11 18:14:00 - [45] TQ is a cluster yes but the database server is not afaik. (just two replicated servers) Also if you look at the new server specs it's a 4 way server so if it suddenly did not have enough oomph then CCP could easily chuck in two more processors if I read the blog right. This would take thread count to 24 which MySQL for example does not scale onto. Maybe CCP is not running a huge monolithic server but according to their blogs the database server is pretty beefy and the new one by the sounds of it has some nice upgrade space too.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 18:32:00 - [46] Originally by: Alcoholic Bird Maybe CCP is not running a huge monolithic server but according to their blogs the database server is pretty beefy and the new one by the sounds of it has some nice upgrade space too. Could be - but I'd seriously like to hear from CCP why/how they've architected the things the why they've had. So far, the most substantial thing we've seen in this thread, is the suggestion, that CCP's thinking could have been influenced primarily by support contract considerations.
Washell Olivaw	Posted - 2009.06.11 18:39:00 - [47] Originally by: Eventy One Could be - but I'd seriously like to hear from CCP why/how they've architected the things the why they've had. This covers a lot of TQ design stuff. Not sure it also covers DB design. Do realize that part of their reasoning and decisions are competition sensitive information. CCP is the only company with the experience of building and running a server capable of 50k+ ccu and that information and expertise has significant value. Originally by: Signature Everybody has a photographic memory, some people just don't have film.
Nareg Maxence Gallente	Posted - 2009.06.11 19:05:00 - [48] This topic has been beaten to death with torn up network cables several times in the past.
Korerin Mayul Amarr The R.I.T.U.A.L Corp	Posted - 2009.06.11 19:08:00 - [49] I vote they should get a few Z10's in, nd re-write the whole ting with CICS and DB2.
Nareg Maxence Gallente	Posted - 2009.06.11 19:10:00 - [50] Edited by: Nareg Maxence on 11/06/2009 19:10:30 Actually nothing beats the performance of flat file databases. Paradox is the way to go.
Admiral IceBlock Caldari Northern Intelligence	Posted - 2009.06.11 20:01:00 - [51] Oracle.
Allen Ramses Caldari Typo Corp	Posted - 2009.06.11 20:13:00 - [52] Originally by: Admiral IceBlock Oracle. They didn't have enough money when they started out to get an Oracle license. Hell, they don't have enough money now to get an Oracle license... Unless you want your subscription costs to rise to $150 per character...
RC Denton	Posted - 2009.06.11 20:22:00 - [53] Edited by: RC Denton on 11/06/2009 20:23:37 Well a few points about the differences: MyISAM doesn't provide the ACID properties so you'd better not put data you care about in a table using the MyISAM engine. The storage engines that do support ACID aren't any faster As of MySQL 5.0 there weren't any good HA options built into the product(ya replication but you had to deal with Deterministic vs not) MS has extremely good support for their enterprise products Based on the fact that they are buying the biggest RAMSAN they can find their main problem is IO performance on the SAN. The DB engine may have some effect on that, but once you have things tuned as much as possible you're stuck with the raw output that the hardware can provide you and that's going to be your limiting factor
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 20:34:00 - [54] Edited by: Eventy One on 11/06/2009 20:36:25 Originally by: RC Denton Based on the fact that they are buying the biggest RAMSAN they can find their main problem is IO performance on the SAN. The DB engine may have some effect on that, but once you have things tuned as much as possible you're stuck with the raw output that the hardware can provide you and that's going to be your limiting factor Good points. This last one, about DB engine performance ultimately being tied to raw output of hardware, is why I was thinking making use of the sockets direct protocol provided by Infiniband would be a good idea. What is the maximum throughput of a normal disk array - something less than 3 Gbps or something, 2.4 Gbps perhaps? Regardless, with the exception of having absolutely everything supported by RAMSAN, the throughput to disk could happen at 20 Gbps. (What is the throughput of RAMSAN BTW - is it akin to memory calls?) So, if CCP is thinking about having all DB supported by RAMSAN, fine. If they are doing some type of load-balanced staged/cached approach it could be something like RAMSAN cache in front of infiniband to disk using SDP. Someone suggested Postgres above, and I agree Postgres is generally better than MySQL, but evenso, Postgres or MySQL are better for Infiniband SDP apps than MSSQL. I believe, when CCP started this business, their decision was the right one. I'm not so sure it still is given where technology has gone.
Alcoholic Bird	Posted - 2009.06.11 20:53:00 - [55] The RamSan 440 supplies 45Gbps (Forty-Five) thoughput. I do believe though the issues is latency and not entirely throughput. Oracle is more along the lines of not per licence but how much is your life worth
Lost Hamster Serenity and Unicum Hungarian Corp	Posted - 2009.06.11 20:55:00 - [56] I think it's really simple: At the time when they developed the game, they had two choice: Oracle, or Microsoft. My-SQL was existing at that time, but it doesn't had any kind of support, not mentioning the availability issues, like clustering, transaction based operation, etc. So they began to develop on Microsoft system, then the next logical step was to use MS-SQL. Now 5 year have passed. MySQL have developed a lot, but I wouldn't trust a 1.5 terabyte database on it. And why change a well known, good enterprise level architecture to something else? There is no gain on it.
Eventy One Magellan Exploration and Survey	Posted - 2009.06.11 21:07:00 - [57] Edited by: Eventy One on 11/06/2009 21:09:41 Originally by: Lost Hamster Now 5 year have passed. MySQL have developed a lot, but I wouldn't trust a 1.5 terabyte database on it. And why change a well known, good enterprise level architecture to something else? There is no gain on it. 1.5T If that's true - Wow! Nice DB, down boy - down! Agreed - changing DB's would be a big task and a conservative approach is is preferable. However, CCP's "need for speed" policy has looked at many other implementation decisions and a-hem corrected them. While looking at bottle necks, code based limitations have received scrutiny, but it seems (to me at least), the most obvious one, has simply had better hardware thrown at it. If they are going to ultimately reduce the bottle necks - perhaps the most obvious one, the Database, should be re-examined as well, with respect to its actual implementation, and not just the hardware running it - to see if the risk of a DB migration might actually pay larger dividends still. A conservative approach can be ignored if there are bigger gains to be made, given the risk involved.
Mashie Saldana BFG Tech	Posted - 2009.06.11 21:20:00 - [58] Originally by: Eventy One 1.5T If that's true - Wow! Nice DB, down boy - down! Agreed - changing DB's would be a big task and a conservative approach is is preferable. However, CCP's "need for speed" policy has looked at many other implementation decisions and a-hem corrected them. While looking at bottle necks, code based limitations have received scrutiny, but it seems (to me at least), the most obvious one, has simply had better hardware thrown at it. If they are going to ultimately reduce the bottle necks - perhaps the most obvious one, the Database, should be re-examined as well, with respect to its actual implementation, and not just the hardware running it - to see if the risk of a DB migration might actually pay larger dividends still. A conservative approach can be ignored if there are bigger gains to be made, given the risk involved. It doesn't matter if you think a different database is faster, CCP won't change it. If you want to know why, just look at my previous reply.
Lost Hamster Serenity and Unicum Hungarian Corp	Posted - 2009.06.11 21:37:00 - [59] Originally by: Eventy One 1.5T If that's true - Wow! Nice DB, down boy - down! I saw a presentation document from CCP, from last year Sept. At that time the database size was 1.3T, And I can assume that the database have grown at least 200M in the last 9 month. Some facts from 2008 Sept: - Database transactions are usually around 1500 per second, reaching 2000 per second at peak hours - æinvItemsÆ table receives over 9 million insertions in a day (not counting update and select statements)
Alcoholic Bird	Posted - 2009.06.11 21:47:00 - [60] Originally by: Lost Hamster Originally by: Eventy One 1.5T If that's true - Wow! Nice DB, down boy - down! I saw a presentation document from CCP, from last year Sept. At that time the database size was 1.3T, And I can assume that the database have grown at least 200M in the last 9 month. Some facts from 2008 Sept: - Database transactions are usually around 1500 per second, reaching 2000 per second at peak hours - æinvItemsÆ table receives over 9 million insertions in a day (not counting update and select statements) Actually a dev blog from last year said along the lines of 3000 queries per second. That estimates to ~250 million queries per day. TBH I think the database server will run into hardware limits before it runs into its own first.

Pages: 1 [2] 3 4 5 :: one page
First page \| Previous page \| Next page \| Last page

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.