EVE Search - CCP: More Hardware != Improved Performance

Check My IP Information

All Channels

EVE General Discussion

CCP: More Hardware != Improved Performance

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 [2] 3 4 5 :: one page

Author	Thread Statistics \| Show CCP posts - 1 post(s)
Lana Bird	Posted - 2007.01.11 22:16:00 - [31] Originally by: Dark Shikari Originally by: Lana Bird Wow. Now that was constructive. Yes, it was. I don't think people should be telling CCP how to code their game. Features? Sure. Behind-the-scenes coding? They know how to do it themselves. Unless you have access to the code, you can't fairly tell them how to write their own game. I realy dont see the OP telling CCP how to code their game, imo he just asked the question if they are looking into the SQL department or not.
AlleyKat Gallente The Avalon Foundation	Posted - 2007.01.11 22:24:00 - [32] Originally by: MysticNZ When the new client comes out that takes advantage of the gpu, this might all change. I agree. Recruiting!
Dark Shikari Caldari Imperium Technologies Firmus Ixion	Posted - 2007.01.11 22:28:00 - [33] Originally by: Duff Man Edited by: Duff Man on 11/01/2007 22:06:40 Originally by: Dark Shikari Originally by: Lana Bird Wow. Now that was constructive. Yes, it was. I don't think people should be telling CCP how to code their game. Features? Sure. Behind-the-scenes coding? They know how to do it themselves. Unless you have access to the code, you can't fairly tell them how to write their own game. No actually DS it wasn't. Normally what you have to say is worth listening to, however I must agree with others here that "CCP have obviously thought of this so move on" is not a constructive addition to this thread. The is not "telling" CCP anything. He has posted what is a well thought out and appropriately detailed / explained question. In my opinion, that deserves an equally well thought out and detailed response, even if that responce is "We've looked into it and here is why it is not a good idea to swap architectures". I see his complaint, but he doesn't actually make any suggestions, only "don't use SQL." Well that's great, but what should CCP use? -[23] Member- Listen to EVE-Trance Radio! (DSTrance channel ingame)
Barrier Solo Infinity Shipyard Masters Interstellar Starbase Syndicate	Posted - 2007.01.11 22:46:00 - [34] I work in what could be called, loosely, systems integration. CCP has tied their infrastruture to Windows technologies. SQL server is not widely recognized as an enterprise-level product. It fits into basically the same bracket as MySQL, PostgreSQL (sp?). It works great for midrange applications, like an online ordering system with maybe 100k application transactions a day. DB2 and Oracle are great high end enterprise DBs; with competent DBAs, these databases scream. You NEED a database admin for these things. CCP may have got suckered into getting SQL server so they could hire cheap database admin labor; I am sure they are good at their job, but given they haven't been admin'ing SQL server at a large mission critical site (since no one uses it for that) you can expect a database admin to tune the DB for 100k transactions a day instead of a second. I'm sure their hardware is fine, but some serious work needs to be done on indexes, optimizing application code to use the indexes properly, etc. Whip the donkey twice as hard and it's not going to move any faster. CCP - time to experiment with other databases on a test cluster. Also, think about creating market DB clusters, space clusters etc instead of 40+ tables all for the same thing, on one cluster. Barrier Solo, CEO, INSM, ISS Join us! http://oldforums.eveonline.com/?a=topic&threadid=402528
Herculite Hunters Agency Firmus Ixion	Posted - 2007.01.11 22:47:00 - [35] The fact that no devs have commented on this thread most likely shows the OP is on the right track. I'm not a techy, but even my biology major mind can figure out that something isn't being done right. Performance is worse, not better in almost all situations from when I started playing one year ago.
Barrier Solo Infinity Shipyard Masters Interstellar Starbase Syndicate	Posted - 2007.01.11 23:00:00 - [36] Originally by: Herculite The fact that no devs have commented on this thread most likely shows the OP is on the right track. I'm not a techy, but even my biology major mind can figure out that something isn't being done right. Performance is worse, not better in almost all situations from when I started playing one year ago. They keep adding content which is good, but to support content they need new database tables and database entries. Which just adds to the queries per second - basically database bloat. Bloat is combated by optimizing the DB and using the right kinds of tables, things like that. The fact that DB performance gets slightly worse over time instead of better, indicates they do not have either the right DB, or the right people working on the DB. The right people working on the DB would work internally within the DB to make it faster, externally to make sure they have the right hardware, and externally with the game coders to make sure the queries make sense. I coded an app once for myself, and a query took 23 seconds. I started using query caching, which speeded it up quite a bit... maybe 200%. Then I started using indexes and querying the indexes, and my results time are far below 1 second. You can also use LIMIT statements and ORDER statements to get only the data you need. If I am searching the Contract system, I would do a SELECT SQL_CACHE * FROM whatever_table ORDER BY name_field LIMIT 0,29 for the first page, then LIMIT 30,59 for the second, instead of fetching everything and sorting on the client. Stuff like that. Barrier Solo, CEO, INSM, ISS Join us! http://oldforums.eveonline.com/?a=topic&threadid=402528
Dotar Sojat Reality Check	Posted - 2007.01.12 00:43:00 - [37] Judging by some of the responses in this thread and the restraint of the OP amid the flames along with the typical lack of dev response, the bulls-eye was probably hit dead center on this one. I know they haven't responded but I'm betting this thread has started some serious brainstorming around the shop at CCP. All any of us wants is for this game to be able to function at the level it deserves. good thread.
Krxon Blade Caldari Apogee Group	Posted - 2007.01.12 01:01:00 - [38] It seems that their blades has stacking penality. More they add lesser performace (per unit) they gain :) --
VeniVici	Posted - 2007.01.12 01:25:00 - [39] Originally by: Krxon Blade It seems that their blades has stacking penality. More they add lesser performace (per unit) they gain :) got a good chuckle out of that one Barrier brings up several very important points that I can only agree with. In the end though, for all we know, CCP has already pursued these avenues. If this is the case, then that would also mean that they have reached the limit of what their current architecture can do. This is an even more troublesome point as it means that optimization isn't enough anymore, which is even more concerning. Yes I have assumed SQL Server as their DBMS based on the datatypes used from the data dumps. But CCP could also be using Sybase instead. Both have very similar data types and syntax (depending on the version). If CCP is in fact on the SQL Server bandwagon (no blame there...MSFT used to hand it out for free way back when, which is a boon to an ambitious new project), then CCP needs to investigate serious optimizations as Barrier detailed. If they already have optimized everything as far as it will go, then frankly you need an enterprise level DBMS. Sybase serves as a solid and convenient alternative, as database migration will be a lot easier than shifting to something like Oracle (not to mention Sybase is easier on the wallet). And do some aggressive optimization there, because there are plenty of ways to squeeze some serious milage. Indices, caches, and proper constraint management are your friends.
Arlenik Emmanouelik Republic of Red Army	Posted - 2007.01.12 01:30:00 - [40] I'm not an expert in any computing field, and I won't try to sound like one. CCP claims that TQ is on the Top 500 Supercomputers list, and from my understanding TQ runs windows server 2003. Well, about 75% of the systems on the Top 500 list run Linux, about 20% Unix, and less than 1% windows. Now unless that statistic is wrong, that says alot about TQ. Top 500

Frug SYOID Fimbulwinter	Posted - 2007.01.12 01:32:00 - [41] I don't have the expertise Barrier does, but what he's saying makes a lot of sense. The content is top notch, but the implementation is so unprofessional in so many ways. - - - - - - - - - - Do not use dotted lines - - - - - - - - - - - - - - - - - - or automatic signatures - - - - - - - -
VeniVici	Posted - 2007.01.12 01:41:00 - [42] Edited by: VeniVici on 12/01/2007 01:37:53 Just reading over the RamSan press release (CCP Release) and noticed they did say specifically MSFT SQL Server is their DBMS. So optimize if you can, and if you can't, Sybase may be a good option. Also, please, oh please, tell me you're using those RAM-disks for index storage (freeing your caches for data caching). Some DBA's would kill for a couple of those.
Tarron Sarek Gallente Solid Industries Inc.	Posted - 2007.01.12 01:43:00 - [43] Originally by: Krxon Blade It seems that their blades has stacking penality. More they add lesser performace (per unit) they gain :) That one made me laugh ;) I'm no database expert, but from what I gathered by talking to friends, most of them fellow computer science students, MSSQL isn't that popular among them. It's not a bad product, easy to use and an ok choice for normal applications. But the oracle database seems to be the more professional and high-performance solution. Again, this is just hearsay, but it fits the picture and my limited experience. ________________________ - Posting on forums can be more arduous than mowing your lawn with nail scissors -
VeniVici	Posted - 2007.01.12 01:51:00 - [44] Originally by: Tarron Sarek But the oracle database seems to be the more professional and high-performance solution. Oracle is a big player in rock solid enterprise databases. Oracle enterprise licenses do cost a pretty penny but are worth it if you can use the product fully, or have the spare change. Another big concern I'm sure would be floated is that migrating MSSQL to Oracle would take a lot of time and effort to do properly as both their syntax and data types are significantly different where it counts (especially with the number of proprietary data types I see in the data dumps). I bring up Sybase because its a nice in between. Costs less, still offers more enterprise optimizations and features than MSSQL, and has a very similar, if not identical, syntax, and data types (including all those smallints and smalldatetimes).
Sorela Gallente	Posted - 2007.01.12 02:01:00 - [45] Originally by: Dark Shikari They know how to do it themselves. Hehe you clearly do not work in the games industry. Databases were very well established when UO came out but they still made the mistake of using flat files on the backend to store massive amounts of data for instance. This is an industry notorious for circular hiring and a "not invented here" mentality. Obviously things have been improving but it's a very logical assumption to suspect that CCP might not even have a proper DBMS support staff or the DBMS might be ignored too much.
Dark Shikari Caldari Imperium Technologies Firmus Ixion	Posted - 2007.01.12 02:04:00 - [46] Originally by: Sorela Originally by: Dark Shikari They know how to do it themselves. Hehe you clearly do not work in the games industry. Databases were very well established when UO came out but they still made the mistake of using flat files on the backend to store massive amounts of data for instance. This is an industry notorious for circular hiring and a "not invented here" mentality. Obviously things have been improving but it's a very logical assumption to suspect that CCP might not even have a proper DBMS support staff or the DBMS might be ignored too much. Hmm. Somehow that doesn't surprise me. -[23] Member- Listen to EVE-Trance Radio! (DSTrance channel ingame)
El Marchetto	Posted - 2007.01.12 02:07:00 - [47] Edited by: El Marchetto on 12/01/2007 02:08:09 If the DB code has been written to ANSI standards, there would be minimal work in porting the data model itself. There are migration tools for import/export supplied with all these database engines so thats no show stopper either. The hardware they've chosen 'may' have been tuned specifically for MS SQL, though I'd hope CCP hav'ent tied their hands too much either. The most critical dependencies may well be business related, the contracts that have been signed, service levels etc, who knows The thing is, if it needs fixing, please CCP do the contrastive analysis, and 'if' either Oracle or Sybase delivers the raw throughput we're asking for, just tell us the plan, we will wait, honest But, if performance continues to degrade, and EVE gets more and more popular, well we're on a slippery slope, and thats the last thing we all want... Originally by: Sorela Hehe you clearly do not work in the games industry. Databases were very well established when UO came out but they still made the mistake of using flat files on the backend to store massive amounts of data for instance. This is an industry notorious for circular hiring and a "not invented here" mentality. Obviously things have been improving but it's a very logical assumption to suspect that CCP might not even have a proper DBMS support staff or the DBMS might be ignored too much. get down on bended knee, and prays to the heavens Please please don't let this be so For if it is so, the plague of lagness shall be with us always
Sorela Gallente	Posted - 2007.01.12 02:15:00 - [48] Originally by: VeniVici If CCP is in fact on the SQL Server bandwagon (no blame there...MSFT used to hand it out for free way back when, which is a boon to an ambitious new project), then CCP needs to investigate serious optimizations as Barrier detailed. If they already have optimized everything as far as it will go, then frankly you need an enterprise level DBMS. It's far worse than you imagine. They had something else and they actually changed to SQL Server on purpose. I'll bite the bullet and actually say what the problem is for those of you who missed some interviews that happened like a year or so ago. Certain key people who work in CCP are very pro-microsoft. I won't name names but T-shirts with penguins being harmed are involved :p. I would not be suprised if these key people are reaching outside their area's of expertise and influencing decisions in bad ways (aka like picking Microsoft SQL server which is clearly a poor choice). They are very smart obviously and sometimes smart game coders/designers have a problem with trying to do to much because let's face it they usually have what it takes to tackle all sorts of fields. But everyone can overextend. This is all just vague surmising on my part of course.
Kldraina	Posted - 2007.01.12 02:23:00 - [49] Poor Valar.
Aphotic Raven Gallente E X O D U S Imperial Republic Of the North	Posted - 2007.01.12 02:23:00 - [50] Originally by: Dark Shikari So clearly you know more about coding and database management than everyone at CCP? They probably "figured this out" 3 years before you started playing, and have already started to work on it. Yep... 3 years later... still not solved. Out of interest... where are you Dark Shikari? since war deccing my alliance i havent seen you burning up the killboards. In fact your last kill was last year. I want to see the infamous forum poster and T2 hound in action

Minsc Gallente A.W.M Ka-Tet	Posted - 2007.01.12 02:45:00 - [51] Edited by: Minsc on 12/01/2007 02:42:08 Originally by: Sorela Originally by: VeniVici If CCP is in fact on the SQL Server bandwagon (no blame there...MSFT used to hand it out for free way back when, which is a boon to an ambitious new project), then CCP needs to investigate serious optimizations as Barrier detailed. If they already have optimized everything as far as it will go, then frankly you need an enterprise level DBMS. It's far worse than you imagine. They had something else and they actually changed to SQL Server on purpose. I'll bite the bullet and actually say what the problem is for those of you who missed some interviews that happened like a year or so ago. Certain key people who work in CCP are very pro-microsoft. I won't name names but T-shirts with penguins being harmed are involved :p. I would not be suprised if these key people are reaching outside their area's of expertise and influencing decisions in bad ways (aka like picking Microsoft SQL server which is clearly a poor choice). They are very smart obviously and sometimes smart game coders/designers have a problem with trying to do to much because let's face it they usually have what it takes to tackle all sorts of fields. But everyone can overextend. This is all just vague surmising on my part of course. Erm...I'm pretty sure they've been using MSSQL from day one, oh and tux was brought in well after the new server was brought in. Freaking Linux Zealots Originally by: Sharkbait please for the love of god read the dam stickies
VeniVici	Posted - 2007.01.12 02:51:00 - [52] Originally by: Sorela It's far worse than you imagine. They had something else and they actually changed to SQL Server on purpose. ... Certain key people who work in CCP are very pro-microsoft. I won't name names but T-shirts with penguins being harmed are involved :p. I would not be suprised if these key people are reaching outside their area's of expertise and influencing decisions in bad ways (aka like picking Microsoft SQL server which is clearly a poor choice). It's great to hear a bit of history here. (Though I guess this is in dispute as fact?). I desperately hope though that CCP doesn't let their bottom line be driven by simple bias. This is their bottom line we're talking about, because frankly if the system can't handle more users then they have, they won't be drawning new users in. Choices that are good for software vendors and their supporters, are often far from being good for your company. You need to investigate all of the possible solutions and pick the one that's appropriate. In the end even if there's a subjective choice made to stick with SQL Server, in the end it comes down to using the DBMS, whatever that may be, properly. Also, a dedicated DBA team/member is looking like a necessity for EVE, especially with the richness and mass of data accumulated, and the features looking to be implemented. When someone, usually the poor system administrator, gets to have "database administration" tagged on to their roles and responsibilities, either the system, the database, or both end up going up in flames. As an aside, what were they using before SQL Server?
Dark Shikari Caldari Imperium Technologies Firmus Ixion	Posted - 2007.01.12 02:52:00 - [53] Originally by: Aphotic Raven Out of interest... where are you Dark Shikari? since war deccing my alliance i havent seen you burning up the killboards. In fact your last kill was last year. I want to see the infamous forum poster and T2 hound in action I've actually been out of PvP for a while lately . Its been a combination of RL stuff keeping me off of EVE (so that I can't spend long blocks of time in gangs), the fact that I generally hang around Jita to manage my production, and the massive amount of time I spend on STUFF (the T2 sales corporation I run). Maybe I can toss my production for a day, run down to Agil, kit up something, and start shooty shooty again -[23] Member- Listen to EVE-Trance Radio! (DSTrance channel ingame)
Dark Shikari Caldari Imperium Technologies Firmus Ixion	Posted - 2007.01.12 02:53:00 - [54] Edited by: Dark Shikari on 12/01/2007 02:50:33 Originally by: Minsc Originally by: Sorela Originally by: VeniVici If CCP is in fact on the SQL Server bandwagon (no blame there...MSFT used to hand it out for free way back when, which is a boon to an ambitious new project), then CCP needs to investigate serious optimizations as Barrier detailed. If they already have optimized everything as far as it will go, then frankly you need an enterprise level DBMS. It's far worse than you imagine. They had something else and they actually changed to SQL Server on purpose. I'll bite the bullet and actually say what the problem is for those of you who missed some interviews that happened like a year or so ago. Certain key people who work in CCP are very pro-microsoft. I won't name names but T-shirts with penguins being harmed are involved :p. I would not be suprised if these key people are reaching outside their area's of expertise and influencing decisions in bad ways (aka like picking Microsoft SQL server which is clearly a poor choice). They are very smart obviously and sometimes smart game coders/designers have a problem with trying to do to much because let's face it they usually have what it takes to tackle all sorts of fields. But everyone can overextend. This is all just vague surmising on my part of course. Erm...I'm pretty sure they've been using MSSQL from day one, oh and tux was brought in well after the new server was brought in. Freaking Linux Zealots He never even used the word "Linux". What in god's name are you talking about? Microsoft SQL sucks for the purpose CCP is using it. Linux is an operating system, not a database server, and has nothing to do with Microsoft SQL. -[23] Member- Listen to EVE-Trance Radio! (DSTrance channel ingame)
Mysticaa Gallente Fringe Roamers of Goa	Posted - 2007.01.12 03:23:00 - [55] I always love reading threads like this! Look at all the wannabe programmers! But more to the point. I believe the OP has a point. You have upgraded everything else and we are still having issues. Maybe it is time to start thinking about upgrading the database archetechture(sp?). I am sure many of us would be happy if you announced that the next expansion after Kia 2 would be DBupgrade 1 ----------------------------------------------- Why do I post here? Originally by: Tao Han "TANK CEO!!!" Quick Wrangler, to the Banmobile!!! Sig snatched by Xorus
Eian	Posted - 2007.01.12 04:03:00 - [56] Originally by: Anatolius Originally by: VeniVici From what I am led to believe by talking with fellow users and browsing the data dumps provided, it seems EVE runs on a Windows server cluster supported by a SQL Server database cluster (it had better be a cluster). Having talked with some of the DBA's I know, they have literally laughed when I have described EVE, the performance experienced, and the fact it is likely on a SQL Server cluster. SQL Server does great in many cases, but OLTP is not one of them. Could be worse; at least they're not running it on MySQL. As it were, I'm not sure how you're getting database performance issues out of frame repeats. That makes little sense. Unless they're continually pumping ship locations into the database; which makes even less sense. Being one place, then suddenly being where you were before is a symptom of good, old-fashioned lag, nothing more. The market, however, is a good sign of the DB needing some love. Failing that, the client needing some love. My client shouldn't pause for a very noticible second when opening the market window, whether it's waiting for a response to a query or not. Good enough for NASA. Good enough for EVE?
Kldraina	Posted - 2007.01.12 05:57:00 - [57] Originally by: VeniVici Also, a dedicated DBA team/member is looking like a necessity for EVE, especially with the richness and mass of data accumulated, and the features looking to be implemented. That would be Valar, Database Admin. Poor guy gets kept pretty busy. I remember him promising once to never take more than 3 days off at a time, 'cause whenever he does, bad things happen to TQ.
Lea Re Caldari CAPITAL TRUST FREGE Alliance	Posted - 2007.01.12 07:22:00 - [58] But the improvement and performance work isn't only done in Content, it's a three-pronged approach. We have two full software cells now working on our core and client technology. The need for speed, improvements and performance is essential there. copied from dev blog
Galimiy Portret R.u.S.H. Red Alliance	Posted - 2007.01.12 09:28:00 - [59] Edited by: Galimiy Portret on 12/01/2007 09:28:52 Originally by: VeniVici Originally by: Dark Shikari So clearly you know more about coding and database management than everyone at CCP? They probably "figured this out" 3 years before you started playing, and have already started to work on it. Funny how just asking a question of CCP implies that I think I know more than them. All I'm asking is if CCP is even looking into other database architectures? Sure at the time the choice may have been clear, but over the last 5 years database technologies have drastically changed, and the choice of architecture made at the start might be far from an effective choice now. The option is there to switch architectures, it does cost time and money, but so do massive hardware upgrades. CCP knows better the cost of changing architectures, the feasibility of doing so, and the viability of this on performance issues. It's for them to decide but I also know companies tend to take very visible steps to help with performance in order to gain publicity more so than performance. Changing the database architecture is a very non-visible change. It may help performance tremendously but is often not something that can be used to get more publicity. I just hope CCP isn't limiting its solutions in this way. You will never get a definite answer to this question. It's just the way life is. If you are lucky, you will get: "We are considering all possibilities and working hard to implement the best solution". Here, I even saved some dev time to log in and post it. Originally by: dark shikari Yes, it was. I don't think people should be telling CCP how to code their game. Features? Sure. Behind-the-scenes coding? They know how to do it themselves. Unless you have access to the code, you can't fairly tell them how to write their own game. Make EvE open source... Or parts of it. :) Fx. the GUI. ...now in RED
Ather Ialeas Amarr Karjala Inc. Imperial Republic Of the North	Posted - 2007.01.12 10:03:00 - [60] Originally by: Galimiy Portret Make EvE open source... Or parts of it. :) Fx. the GUI. It already is in a sense, cue stackless Python. Oh wait, it's used for exploiting. Darn! --- About the super computers using Linux, you all should remember that Linux is merely a kernel of an OS, not an OS itself. What this means is that most likely those super computers are running a heavily tailored non-public version of Linux with their own proprietary software to do the needed db stuff. It's not the same Linux you can download from the internet and it most certainly isn't one of those "my first Linux LiveCD with crayons" things.



Pages: 1 [2] 3 4 5 :: one page
First page \| Previous page \| Next page \| Last page

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.