| Pages: 1 2 3 :: [one page] |
| Author |
Thread Statistics | Show CCP posts - 1 post(s) |
XiticiX
   |
Posted - 2006.09.11 21:48:00 -
[1]
 Quote:
Massively multiplayer online game Eve Online has set a new record for the largest supercomputer cluster in the games industry.
The Eve Online server cluster manages over 150 million database transactions per day on a 64-bit hardware architecture from IBM.
The database servers use solid state disks instead of traditional hard drives to handle more than 400,000 random I/Os per second.
CCP Games, which created Eve Online, recently set a world record by hosting 30,000 concurrent users on a single server shard. The company now hopes to increase that to at least 50,000 users.
"The sharp growth rate of Eve Online was pushing the limits of the technology we replaced," said CCP Games chief executive Hilmar Veigar PTtursson.
"Our goal was to implement a scalable solution that could accommodate the influx of new subscribers and gracefully manage the steadily increasing demand put on our infrastructure."
The upgraded server cluster features dual-processor 64-bit AMD Opteron-based IBM BladeCenter LS20 blade servers, as well additional enhancements to the cluster's internet backbone.
Mighty impressive indeed! Way to go CCP!
|
Galk
Gallente
Autumn Tactics
|
Posted - 2006.09.11 21:54:00 -
[2]
"Our goal was to implement a scalable solution that could accommodate the influx of new subscribers and gracefully manage the steadily increasing demand put on our infrastructure."
They do this with great success
Nahh seriously ccp press releases and interviews are always very nice read
______
|
Admiral IceBlock
Caldari
Northern Intelligence
SMASH Alliance
|
Posted - 2006.09.11 21:57:00 -
[3]
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
|
LUGAL MOP'N'GLO
|
Posted - 2006.09.11 21:57:00 -
[4]
Impressive no doubt. It still can't handle uber bookmark copying though... It has been pretty bad the last couple of days. 
~~~~~~~~~
I wish my lawn was EMO so it would cut itself. I approve of this message. |
Reiisha
Frontier Technologies
|
Posted - 2006.09.11 21:59:00 -
[5]
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
Done.
|
Karl Shade
FinFleet
Lotka Volterra
|
Posted - 2006.09.11 22:00:00 -
[6]
Quote:
CCP Games, which created Eve Online, recently set a world record by hosting 30,000 concurrent users on a single server shard. The company now hopes to increase that to at least 50,000 users.
Baby jesus and all his toddler apostles go
-
|
evistin
Multiverse Corporation
|
Posted - 2006.09.11 22:05:00 -
[7]
Yes it can handle 50k users, but it never said with what level of Quality of Service. 
But look, they are talking hardware, no amount of hardware can resuce poor coding. Thats why they are busy rewriting code.
-------------------
Management and Leadership û The Eve-online Guide |
Magunus
The Forsakened Few
The ARR0W Project
|
Posted - 2006.09.11 22:39:00 -
[8]
Originally by: evistin
Yes it can handle 50k users, but it never said with what level of Quality of Service.
But look, they are talking hardware, no amount of hardware can resuce poor coding. Thats why they are busy rewriting code.
Your arrogance astounds me. Name another game that handles 30k concurrent users. If not handling 30k users flawlessly means the code is 'poor coding', show me a game which has nothing but 'good coding'. You have no idea what you're talking about.
---
In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move.
-- Douglas Adams, 'The Restaurant at the End of the Universe' |
evistin
Multiverse Corporation
|
Posted - 2006.09.11 22:41:00 -
[9]
I apologize, I meant it as a joke.
I know the task is not easy at all. I am from a IT background, I know quite realsitcly how much strain the servers are under.
I shall withdraw my comment
-------------------
Management and Leadership û The Eve-online Guide |
Magunus
The Forsakened Few
The ARR0W Project
|
Posted - 2006.09.11 22:42:00 -
[10]
Originally by: evistin
I apologize, I meant it as a joke.
I know the task is not easy at all. I am from a IT background, I know quite realsitcly how much strain the servers are under.
I shall withdraw my comment
Ok, sorry, I guess I've just seen too many of those types of comments recently and just assume people mean it. Call me touchy. :)
---
In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move.
-- Douglas Adams, 'The Restaurant at the End of the Universe' |
Acwron
Minmatar
Cataclysm Enterprises
|
Posted - 2006.09.11 22:49:00 -
[11]
Edited by: Acwron on 11/09/2006 22:49:44
Originally by: Magunus
Originally by: evistin
Yes it can handle 50k users, but it never said with what level of Quality of Service.
But look, they are talking hardware, no amount of hardware can resuce poor coding. Thats why they are busy rewriting code.
Your arrogance astounds me. Name another game that handles 30k concurrent users. If not handling 30k users flawlessly means the code is 'poor coding', show me a game which has nothing but 'good coding'. You have no idea what you're talking about.
On a more serious note CCP hosted a Python event were many high profile Python devs got together in island in order to improve the performance of stackless Python. Which afaik they achieved.
|
Shiraz Merlot
Octavian Vanguard
RAZOR Alliance
|
Posted - 2006.09.11 23:08:00 -
[12]
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
Your proposal is acceptable.
|
DropZone 187
|
Posted - 2006.09.11 23:29:00 -
[13]
Originally by: Magunus
Originally by: evistin
Yes it can handle 50k users, but it never said with what level of Quality of Service.
But look, they are talking hardware, no amount of hardware can resuce poor coding. Thats why they are busy rewriting code.
Your arrogance astounds me. Name another game that handles 30k concurrent users. If not handling 30k users flawlessly means the code is 'poor coding', show me a game which has nothing but 'good coding'. You have no idea what you're talking about.
Neither do you from your post.
Second Life is an example of a game that supports over 30k simultaneous users on what is defined as 1 shard. In fact, they surpassed the 200k mark. Nay-sayers would say that they don't count because it isn't a pvp style game, but genre shouldn't have anything to do with it.
Not only that, if you believe the propaganda of this being the world's largest gaming cluster, you believe too much in what people tell you. A 70 machine, 140 processor system isn't that big - in fact, since it is based on common hardware you can buy anywhere (AMD 64 bit procs) one could resonably build it better and at a far less cost than their current systems integrator.
And as it goes, the proof is in the pudding - if the cluster was so great, why is it getting bogged down after 25k users?
Anyhow, just a reality check. I hate false propaganda more than poor performance. At least with poor performance they can blaim it on someone else.
|
Par'Gellen
Gallente
Low Grade Ore
|
Posted - 2006.09.12 00:06:00 -
[14]
Originally by: DropZone 187
Originally by: Magunus
Originally by: evistin
Yes it can handle 50k users, but it never said with what level of Quality of Service.
But look, they are talking hardware, no amount of hardware can resuce poor coding. Thats why they are busy rewriting code.
Your arrogance astounds me. Name another game that handles 30k concurrent users. If not handling 30k users flawlessly means the code is 'poor coding', show me a game which has nothing but 'good coding'. You have no idea what you're talking about.
Neither do you from your post.
Second Life is an example of a game that supports over 30k simultaneous users on what is defined as 1 shard. In fact, they surpassed the 200k mark. Nay-sayers would say that they don't count because it isn't a pvp style game, but genre shouldn't have anything to do with it.
Not only that, if you believe the propaganda of this being the world's largest gaming cluster, you believe too much in what people tell you. A 70 machine, 140 processor system isn't that big - in fact, since it is based on common hardware you can buy anywhere (AMD 64 bit procs) one could resonably build it better and at a far less cost than their current systems integrator.
And as it goes, the proof is in the pudding - if the cluster was so great, why is it getting bogged down after 25k users?
Anyhow, just a reality check. I hate false propaganda more than poor performance. At least with poor performance they can blaim it on someone else.
I was going to say something similar but you said it perfectly.
It boils down to: 30k users is nice but if the game is lagging and crashing daily then it's not such a big accomplishment. It's actually worse due to more people seeing it lag and crash 
Starmaps - An Insta Solution |
Blind Man
Caldari
0utbreak
|
Posted - 2006.09.12 00:12:00 -
[15]
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
Agreed.
SignatureTemporarily Out Of Order |
Mithrantir Ob'lontra
Gallente
Ixion Defence Systems
The Cyrene Initiative
|
Posted - 2006.09.12 00:13:00 -
[16]
Originally by: DropZone 187
Originally by: Magunus
Originally by: evistin
Yes it can handle 50k users, but it never said with what level of Quality of Service.
But look, they are talking hardware, no amount of hardware can resuce poor coding. Thats why they are busy rewriting code.
Your arrogance astounds me. Name another game that handles 30k concurrent users. If not handling 30k users flawlessly means the code is 'poor coding', show me a game which has nothing but 'good coding'. You have no idea what you're talking about.
Neither do you from your post.
Second Life is an example of a game that supports over 30k simultaneous users on what is defined as 1 shard. In fact, they surpassed the 200k mark. Nay-sayers would say that they don't count because it isn't a pvp style game, but genre shouldn't have anything to do with it.
Not only that, if you believe the propaganda of this being the world's largest gaming cluster, you believe too much in what people tell you. A 70 machine, 140 processor system isn't that big - in fact, since it is based on common hardware you can buy anywhere (AMD 64 bit procs) one could resonably build it better and at a far less cost than their current systems integrator.
And as it goes, the proof is in the pudding - if the cluster was so great, why is it getting bogged down after 25k users?
Anyhow, just a reality check. I hate false propaganda more than poor performance. At least with poor performance they can blaim it on someone else.
I don't want to start a fight but i guess that you know that any hardware can be found on the market so it's there is nothing that is specialized only to servers. Especially in the IT industry. One only needs the cash to buy whatever he/she desires.
In fact you could also read a little more between the lines on the OP and see that there is a special version of the CPUs being used (the powered down version) which has been tested by IBM to works seamlessly with their servers. A 70 server cluster is a big deal also IMHO since this is nothing one can easily find anywhere in the world.
I haven't experienced the lag at the extend people say they do, so i won't get in that since i don't know the circumstances (never been in Jita, i have been in some battles with many participants).
Also your calculations are a little wrong since these CPUs are dual cored which means 2 in the packet of one so it is a 70 server 280 CPUs in reality.
And to add something to the pudding when it was just over below 30k users i logged in and i haven't experienced lag, which goes to prove that everything is relative to the perspective and circumstances of the observer.
As i said to the beginning of my post this is not to start a fight or flame or anything. Issues do exist and they need to be solved, but the effect they have differs from perspective to perspective. I don't want to degrade their significance but CCPs effort is something that needs to be recognized. Allthough they still have a lot of work ahead of them
-------
Nobody can be exactly like me. Even I have trouble doing it. |
Hllaxiu
Shiva
Morsus Mihi
|
Posted - 2006.09.12 00:14:00 -
[17]
Originally by: DropZone 187
Second Life is an example of a game that supports over 30k simultaneous users on what is defined as 1 shard. In fact, they surpassed the 200k mark. Nay-sayers would say that they don't count because it isn't a pvp style game, but genre shouldn't have anything to do with it.
Second life has surpassed 10k concurrent users. They have over 200k accounts (they actually have 600k). EVE has surpassed over 30k concurrent users and 150k accounts.
---
Our greatest glory is not in never failing, but in rising up every time we fail. - Emerson |
Sentient Void
QunSegh
Forces of Freedom
|
Posted - 2006.09.12 00:34:00 -
[18]
A friend of mine plays second life.
/me points and laughs at friend.
Not only that... but if what the last poster said is true, that's only 5% of all users online at once... we had like 20% of all users on at once... shows the higher level of dedication to the game on EVE's standpoint, if you ask me.
Regarding EVE Online... People really do seem to want something beautiful but unforgiving.
|
evistin
Multiverse Corporation
|
Posted - 2006.09.12 00:38:00 -
[19]
2nd life has problems. They let personal data slip into the public.
Linkage
For all the games draw, this is a set back for anyone.
-------------------
Management and Leadership û The Eve-online Guide |
Hllaxiu
Shiva
Morsus Mihi
|
Posted - 2006.09.12 00:41:00 -
[20]
Originally by: Sentient Void
A friend of mine plays second life.
/me points and laughs at friend.
Not only that... but if what the last poster said is true, that's only 5% of all users online at once... we had like 20% of all users on at once... shows the higher level of dedication to the game on EVE's standpoint, if you ask me.
I wouldn't read into the percentage of total accounts thing too much, since the recurring fee is optional, any and all accounts can be reactivated at any time. The EVE subscription counts obviously only include subscribers, I bet that every eve account in existance is much higher than 150k.
---
Our greatest glory is not in never failing, but in rising up every time we fail. - Emerson |
Malthros Zenobia
Caldari Independent Navy Reserve
Kimotoro Directive
|
Posted - 2006.09.12 00:46:00 -
[21]
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
I remember some people saying that about 30k.
From the looks of the replies, you're going to be getting podded alot within the next year.
Sorry you can't afford a dev so you get me instead ^^ - Xorus
I hear Xorus is only 50 isk an hour  - Immy
Oooh that could get Suvetar for the day! - Cathath |
Karl Shade
FinFleet
Lotka Volterra
|
Posted - 2006.09.12 01:35:00 -
[22]
Originally by: Malthros Zenobia
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
I remember some people saying that about 30k.
From the looks of the replies, you're going to be getting podded alot within the next year.
I don¦t think the good admiral doubts CCP¦s ability to hit the 50k concurrent mark, neither do I. I just can¦t for the life of me feel very excited about the 30k we have now considering the abysmal performance in one of the games big draws, massive pvp-battles and while I hope they will, I don¦t see them pulling that off any better with 50k.
I am still hoping ccp will surprise us all with some recoding/tooling/whateverthingamajing that let¦s that big stuff actually happen again. Just a bit disillusioned lately. 
-
|
Dixon
Caldari
|
Posted - 2006.09.12 02:47:00 -
[23]
Originally by: Karl Shade
Originally by: Malthros Zenobia
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
I remember some people saying that about 30k.
From the looks of the replies, you're going to be getting podded alot within the next year.
I don¦t think the good admiral doubts CCP¦s ability to hit the 50k concurrent mark, neither do I. I just can¦t for the life of me feel very excited about the 30k we have now considering the abysmal performance in one of the games big draws, massive pvp-battles and while I hope they will, I don¦t see them pulling that off any better with 50k.
I am still hoping ccp will surprise us all with some recoding/tooling/whateverthingamajing that let¦s that big stuff actually happen again. Just a bit disillusioned lately.
The game lagged when we only had 5000 users online. The game stability will always swing back and forwards. This time it's sadly swung a little too far back, but that will change.
- - - - - -
I have no strong feelings one way or the other... |
XiticiX
|
Posted - 2006.09.12 05:16:00 -
[24]
With 30,000 concurrent users, it's amazing the farm isn't catching fire, or freezing completely. CCP truly does need to be recognized for their efforts. This is coming from a c#/sql database developer. I myself find it astounding the amount of data that must be transferred, digested, and spit back out to the player. A SINGLE player. Let alone 30,000 of them at once. Sure, there are problems. I haven't experienced lag myself, but if a lot of players are complaining - look at it this way - it just means more people to pod ;)
Seriously, the effort the development team put into planning, deployment, and scalability is a tribute to how great a game this really is. The team should be appluaded - not criticized. Excellent work guys - keep it up!
|
Galk
Gallente
Autumn Tactics
|
Posted - 2006.09.12 05:33:00 -
[25]
Originally by: Admiral IceBlock
Bwahahahahahaha! LOL @ 50k USERS! OVER MY DEAD BODY!
You are soooo wrong...
The power of 2 is due up again soon.
This year with added timecode goodness going on, it's high feasible that many new alt's will be created to fill that number
______
|
mazzilliu
Caldari
SniggWaffe
|
Posted - 2006.09.12 06:42:00 -
[26]
eve is ftw.
and let's copy more bookmarks to make a new database transaction record
will still draw sigs 4 isk click here |
Qutsemnie
Caldari
Deep Core Mining Inc.
|
Posted - 2006.09.12 07:13:00 -
[27]
I thought for sure a developer said they would never do power of 2 again as it simply was to good an offer.
All those solid state hard drives.... mmmmmm. That would be home computer luxury right there.
|
Noluck Ned
FATAL REVELATIONS
Lotka Volterra
|
Posted - 2006.09.12 07:35:00 -
[28]
I saw Jesus in N-RAEL last friday night.
HE was weeping.
Originally by: Posidrive
So technically being a pirate is't quite as easy as I thought after reading this guide.
|
Kalaan Oratay
The Imperial Commonwealth
The Phantom Conglomerate
|
Posted - 2006.09.12 07:38:00 -
[29]
/me points at the Chinese shard
/me scratches head 
--- Originally by: Archilies
Ignore what others say: Fit what you want, with what you have, whenever you want.
|
BurnHard
|
Posted - 2006.09.12 07:41:00 -
[30]
Originally by: Sentient Void
A friend of mine plays second life.
/me points and laughs at friend.
Not only that... but if what the last poster said is true, that's only 5% of all users online at once... we had like 20% of all users on at once... shows the higher level of dedication to the game on EVE's standpoint, if you ask me.
No, it isn't. Second Life probably doesn't require as many alts. Although it doesn't make any difference to the record, because IMHO a client instance is a client instance; a large percentage of those instances are alts.
|
Sable Schroedinger
Gallente
Jericho Fraction
The Star Fraction
|
Posted - 2006.09.12 08:02:00 -
[31]
I'm starting to suspect that the current performance issues are out of CCP's hands. I implemented SQL server 2005 at work about a month ago - where as we never really had a problem with blocks and deadlocks before, I'm seeing an increased occurrence of it. Its still early days, so I've got little to no information on it all yet, but its starting to look to me like 2005 has some flaws in that area. Its that or its so called "on line" operations are not quite as "on line" as they would have you believe - transaction log backups and index reorganisation to name 2. Could be that if theres a problem we're waiting on MS to release a patch.
p.s. please don't jump in with "implement a proper DB such as X" comments. They're redundant and puerile.
--------------------------------------------
Nothing is as cruel as the righteousness of innocents |
Matthew
Caldari
BloodStar Technologies
|
Posted - 2006.09.12 08:03:00 -
[32]
50K online really shouldn't be a problem, it's what'll happen if that 50K tries to keep the same proportion of people in Jita and similar systems as we currently see.
Take the reboot this weekend. Before it happened, I saw a queue to get into Jita was being reported as over 750. Think about that - even if everyone who was currently in Jita was magically removed, not everyone in the queue could get in before the system filled again. You have to ask why so many people insisted on waiting in the obviously ridiculous queue, rather than finding something else to do.
Combine the 750 in the queue with the likely 600+ actually in the system, and you've got 1350+ trying to use one system. So if we say there's 70 nodes and 30k online, that's 4.5% of the playerbase trying to cram onto 1.4% of the server. In a clustered system like TQ, that's never going to be pretty.
Expanding the capacity of TQ to handle a distributed load is easy, just add more nodes. It's increasing the single-node capacity that's the problem.
-------
There is no magic Wand of Fixing, and it is not powered by forum whines. |
SIlver Light
Minmatar
5punkorp
Interstellar Starbase Syndicate
|
Posted - 2006.09.12 08:13:00 -
[33]
Originally by: Kalaan Oratay
/me points at the Chinese shard
/me scratches head
Last word I saw on that was that Serenity wasn't officially CCPs. What they've done is rent out the server and game code to Optic to allow them to run eve-china. There's a few devs over there as advisors, but that's the relationship as I understand it.
------
Proud Member of 5punkorp |
BurnHard
|
Posted - 2006.09.12 08:38:00 -
[34]
Originally by: Sable Schroedinger
I'm starting to suspect that the current performance issues are out of CCP's hands. I implemented SQL server 2005 at work about a month ago - where as we never really had a problem with blocks and deadlocks before, I'm seeing an increased occurrence of it. Its still early days, so I've got little to no information on it all yet, but its starting to look to me like 2005 has some flaws in that area. Its that or its so called "on line" operations are not quite as "on line" as they would have you believe - transaction log backups and index reorganisation to name 2. Could be that if theres a problem we're waiting on MS to release a patch.
p.s. please don't jump in with "implement a proper DB such as X" comments. They're redundant and puerile.
Deadlocks are inevitable, it's how you handle them that makes a difference. You can always restructure your schema to make them less likely. But I wouldn't like to be the one doing that on a billion record table ;). On a related note, I've noticed deadlocks and timeouts with my test code on 2005 I didn't get with 2000. It wouldn't suprise me if it needs a patch or two before it's up to standard.
|
Lazuran
|
Posted - 2006.09.12 08:48:00 -
[35]
Edited by: Lazuran on 12/09/2006 08:48:14
Originally by: Sentient Void
we had like 20% of all users on at once... shows the higher level of dedication to the game on EVE's standpoint, if you ask me.
Shows more idling and using multiple accounts at the same time.
"The whole of NYC is not 1.0. Some back alley in the Bronx is deep 0.0, while right outside NYPD headquarters is 1.0." -- Slaaght Bana |
Beigehornet
Caldari
Digital Fury Corporation
|
Posted - 2006.09.12 09:12:00 -
[36]
hrmmm
Nice statement shame it still cant handle the current user base. 24k users pushes the server to the max already and when its over that its almost unusable.
Good to know CCP is flashing their IBM bling thats no more than Plastic with Gold plateing.
GG CCP
|
Sable Schroedinger
Gallente
Jericho Fraction
The Star Fraction
|
Posted - 2006.09.12 09:28:00 -
[37]
Originally by: BurnHard
Deadlocks are inevitable, it's how you handle them that makes a difference. You can always restructure your schema to make them less likely. But I wouldn't like to be the one doing that on a billion record table ;). On a related note, I've noticed deadlocks and timeouts with my test code on 2005 I didn't get with 2000. It wouldn't suprise me if it needs a patch or two before it's up to standard.
Whilst deadlocks are inevitable, I agree, the increase I'm seeing is from 1 per 3 months or so to 1 or 2 per day!
However, each time the blocked process is a system one - the aforementioned transaction log backup and index reorganisation (not rebuild). So the waters are a little muddy here.
--------------------------------------------
Nothing is as cruel as the righteousness of innocents |
Garia666
Amarr
adeptus gattacus
Lotka Volterra
|
Posted - 2006.09.12 09:31:00 -
[38]
The hardware is impressive however the problem lies in the code.
They dont code anymore like they used to.. Not as in the Old day`s
The Old day`s
After seeing this you will know what i mean..
|
Splagada
Minmatar
Tides of Silence
|
Posted - 2006.09.12 11:27:00 -
[39]
"400,000 random I/Os per second."
oh my .... doesnt the smoke from the hds cause problems in the offices? :p
-
Tides of Silence recruiting miners and overall fun people |
Matthew
Caldari
BloodStar Technologies
|
Posted - 2006.09.12 11:50:00 -
[40]
Originally by: Splagada
"400,000 random I/Os per second."
oh my .... doesnt the smoke from the hds cause problems in the offices? :p
That's why they use this instead. It's basically a big-ass box full of RAM, that pretends it's a hard drive. Though I gather that it isn't big enough to hold all the tables, and they had to move some tables off to a normal disk array to make room until they take delivery of another one (which Oveur confirmed they have ordered, but these things probably aren't available on next day delivery!). Might be the reason why the DB isn't quite as nippy as it used to be, but the fix is in transit (hopefully quite literally )
-------
There is no magic Wand of Fixing, and it is not powered by forum whines. |
Lazuran
|
Posted - 2006.09.12 12:03:00 -
[41]
Originally by: Matthew
That's why they use this instead.
It worries me that even this isn't enough anymore, when it is 1-2 orders of magnitude faster than what they had in late 2004 or so, when there was no lag and the number of concurrent users was roughly 20-30% of today's.
Looks like they need to fix more than just the hardware...
"The whole of NYC is not 1.0. Some back alley in the Bronx is deep 0.0, while right outside NYPD headquarters is 1.0." -- Slaaght Bana |
Matthew
Caldari
BloodStar Technologies
|
Posted - 2006.09.12 12:15:00 -
[42]
Originally by: Lazuran
Originally by: Matthew
That's why they use this instead.
It worries me that even this isn't enough anymore, when it is 1-2 orders of magnitude faster than what they had in late 2004 or so, when there was no lag and the number of concurrent users was roughly 20-30% of today's.
Looks like they need to fix more than just the hardware...
Afaik the problem they're having with the RAMSAN at the moment isn't one of speed, but of storage capacity. Which means that they have to put some tables on a "standard" fibre channel disk array, which is much slower. Once more capacity arrives and all the tables can go back on them, I would be very surprised if we managed to hit the max speed of the RAMSANs.
The trouble is that in a clustered system like TQ, a bottleneck at any level can cause problems, and you usually have different problems kicking in for different people at diffrent times. For example, the fastest DB imaginable isn't going to help Jita, because that bottleneck is on the SOL node actually running the system simulation.
Without the server diagnostics, we can't do more than make educated guesses about what may be the problem at any given point.
-------
There is no magic Wand of Fixing, and it is not powered by forum whines. |
Raquel Smith
Ferengi Commerce Authority
|
Posted - 2006.09.12 12:21:00 -
[43]
Originally by: Par'Gellen
I was going to say something similar but you said it perfectly.
It boils down to: 30k users is nice but if the game is lagging and crashing daily then it's not such a big accomplishment. It's actually worse due to more people seeing it lag and crash
Actually Second Life is as unstable as Eve. ;) My fiancTe plays and is constantly complaining about how regions of SL are crashing or having to go down for a reboot.
|
|
Valar
|
Posted - 2006.09.12 12:28:00 -
[44]
Originally by: Matthew
Afaik the problem they're having with the RAMSAN at the moment isn't one of speed, but of storage capacity. Which means that they have to put some tables on a "standard" fibre channel disk array, which is much slower. Once more capacity arrives and all the tables can go back on them, I would be very surprised if we managed to hit the max speed of the RAMSANs.
The trouble is that in a clustered system like TQ, a bottleneck at any level can cause problems, and you usually have different problems kicking in for different people at diffrent times. For example, the fastest DB imaginable isn't going to help Jita, because that bottleneck is on the SOL node actually running the system simulation.
Without the server diagnostics, we can't do more than make educated guesses about what may be the problem at any given point.
You are right, the problem with the RAMSAN is not the bandwidth, but the capacity.
Most of the database is on a 30 disk fiber optic array, a few indexes on a 10 disk fiber optic array and the heavily used stuff, like the items table, a few related tables and indexes that come with 'em, the transaction log and tempdb are on the RAMSAN.
Due to the lack of space on the RAMSAN recently, I've had to move objects from the RAMSAN to the disk arrays, and I've had to take measures so that in case the transaction log grows too much it starts expanding on to the disk array aswell.
The performance hit of moving the objects from the RAMSAN was not as much as we had feared, but when the transaction log has been on the disk array performance has suffered quite a bit. That however has only happened in exceptional circumstances and I always fix that as soon as I notice.
------
Valar
Database admin - Server operations team
CCP Games
How to write a good bugreport |
|
Par'Gellen
Gallente
Low Grade Ore
|
Posted - 2006.09.12 13:14:00 -
[45]
Originally by: Valar
You are right, the problem with the RAMSAN is not the bandwidth, but the capacity.
Most of the database is on a 30 disk fiber optic array, a few indexes on a 10 disk fiber optic array and the heavily used stuff, like the items table, a few related tables and indexes that come with 'em, the transaction log and tempdb are on the RAMSAN.
Due to the lack of space on the RAMSAN recently, I've had to move objects from the RAMSAN to the disk arrays, and I've had to take measures so that in case the transaction log grows too much it starts expanding on to the disk array aswell.
The performance hit of moving the objects from the RAMSAN was not as much as we had feared, but when the transaction log has been on the disk array performance has suffered quite a bit. That however has only happened in exceptional circumstances and I always fix that as soon as I notice.
Thanks for the info Valar. So when will the new RAMSAN arrive and be installed?
Starmaps - An Insta Solution |
Wild Rho
Amarr
Imperial Shipment
|
Posted - 2006.09.12 13:24:00 -
[46]
Originally by: Valar
stuff
I am so bloody glad I don't have your job tbh
WE ARE DYSLEXIC OF BORG. Refutance is systile. Your ass will be laminated. - Jennie Marlboro
|
Helison
Gallente
Times of Ancar
R i s e
|
Posted - 2006.09.12 13:29:00 -
[47]
Valar, the problems in the last days, were several regions lag extremly: Are these DB-related, or is it caused by a massive overload (or bug) on the SOL-Servers?
|
Gariuys
Evil Strangers Inc.
|
Posted - 2006.09.12 13:33:00 -
[48]
Originally by: Wild Rho
Originally by: Valar
stuff
I am so bloody glad I don't have your job tbh
100% agreed lol, fun but a nightmare most of the time.
|
Larshus Magrus
Caldari Provisions
|
Posted - 2006.09.12 13:33:00 -
[49]
Originally by: Valar
Originally by: Matthew
Afaik the problem they're having with the RAMSAN at the moment isn't one of speed, but of storage capacity. Which means that they have to put some tables on a "standard" fibre channel disk array, which is much slower. Once more capacity arrives and all the tables can go back on them, I would be very surprised if we managed to hit the max speed of the RAMSANs.
The trouble is that in a clustered system like TQ, a bottleneck at any level can cause problems, and you usually have different problems kicking in for different people at diffrent times. For example, the fastest DB imaginable isn't going to help Jita, because that bottleneck is on the SOL node actually running the system simulation.
Without the server diagnostics, we can't do more than make educated guesses about what may be the problem at any given point.
You are right, the problem with the RAMSAN is not the bandwidth, but the capacity.
Most of the database is on a 30 disk fiber optic array, a few indexes on a 10 disk fiber optic array and the heavily used stuff, like the items table, a few related tables and indexes that come with 'em, the transaction log and tempdb are on the RAMSAN.
Due to the lack of space on the RAMSAN recently, I've had to move objects from the RAMSAN to the disk arrays, and I've had to take measures so that in case the transaction log grows too much it starts expanding on to the disk array aswell.
The performance hit of moving the objects from the RAMSAN was not as much as we had feared, but when the transaction log has been on the disk array performance has suffered quite a bit. That however has only happened in exceptional circumstances and I always fix that as soon as I notice.
Although this makes logical sense, the steps taking to fix these large tables do not. The STANDARD (and yes I maintain DB's as large, if not larger that what eve runs, with millions if not hundreds of millions of transactiosn per day) is simply to add more memory to the machine. Get the tables into memory and the problem goes away.
Ok you say but:
1) How do you keep the most used tables in memory? You don't. The OS does. Thats its job. Any good OS worth its salt will have no problems (asuming the DB is tuned correctly) maintaining huge tables in memory. Memmory access is magnitudes faster than accessing tables stored on a hard drive... no matter if its solid state storage or a traditional fibre raid array.
2) How do you have 250+ gigs of memory? My intel xeon box only support16/32 gigs!?! Buy some real hardware. The transactions eve is pushing need massive amounts of ram to scale. Even the new 64 gig 51xx series xeon motherboards arent going to cut it. ite the bullet, buy a real piece of EXPANDABLE INM/Sun hardware with a real OS and jsut be done with it. You are spending silly money on the ramsans which dont FIX anything.. just extend the problem out further.
The real limiting factor here is NOT the RAMSAN storage. Its RAM storage that CCP is trying to gimp around by pushing RAM into a soild state array because the archetecture they are for some reason stubornly married to cannot support the amount of ram needed to run the current application effeciently. ITs not jsut me saying this,. Talk to any DB engineer who works with large data sets... he will tell you exactly the same thing.
|
Gariuys
Evil Strangers Inc.
|
Posted - 2006.09.12 13:41:00 -
[50]
Originally by: Larshus Magrus
Originally by: Valar
Originally by: Matthew
Afaik the problem they're having with the RAMSAN at the moment isn't one of speed, but of storage capacity. Which means that they have to put some tables on a "standard" fibre channel disk array, which is much slower. Once more capacity arrives and all the tables can go back on them, I would be very surprised if we managed to hit the max speed of the RAMSANs.
The trouble is that in a clustered system like TQ, a bottleneck at any level can cause problems, and you usually have different problems kicking in for different people at diffrent times. For example, the fastest DB imaginable isn't going to help Jita, because that bottleneck is on the SOL node actually running the system simulation.
Without the server diagnostics, we can't do more than make educated guesses about what may be the problem at any given point.
You are right, the problem with the RAMSAN is not the bandwidth, but the capacity.
Most of the database is on a 30 disk fiber optic array, a few indexes on a 10 disk fiber optic array and the heavily used stuff, like the items table, a few related tables and indexes that come with 'em, the transaction log and tempdb are on the RAMSAN.
Due to the lack of space on the RAMSAN recently, I've had to move objects from the RAMSAN to the disk arrays, and I've had to take measures so that in case the transaction log grows too much it starts expanding on to the disk array aswell.
The performance hit of moving the objects from the RAMSAN was not as much as we had feared, but when the transaction log has been on the disk array performance has suffered quite a bit. That however has only happened in exceptional circumstances and I always fix that as soon as I notice.
Although this makes logical sense, the steps taking to fix these large tables do not. The STANDARD (and yes I maintain DB's as large, if not larger that what eve runs, with millions if not hundreds of millions of transactiosn per day) is simply to add more memory to the machine. Get the tables into memory and the problem goes away.
Ok you say but:
1) How do you keep the most used tables in memory? You don't. The OS does. Thats its job. Any good OS worth its salt will have no problems (asuming the DB is tuned correctly) maintaining huge tables in memory. Memmory access is magnitudes faster than accessing tables stored on a hard drive... no matter if its solid state storage or a traditional fibre raid array.
2) How do you have 250+ gigs of memory? My intel xeon box only support16/32 gigs!?! Buy some real hardware. The transactions eve is pushing need massive amounts of ram to scale. Even the new 64 gig 51xx series xeon motherboards arent going to cut it. ite the bullet, buy a real piece of EXPANDABLE INM/Sun hardware with a real OS and jsut be done with it. You are spending silly money on the ramsans which dont FIX anything.. just extend the problem out further.
The real limiting factor here is NOT the RAMSAN storage. Its RAM storage that CCP is trying to gimp around by pushing RAM into a soild state array because the archetecture they are for some reason stubornly married to cannot support the amount of ram needed to run the current application effeciently. ITs not jsut me saying this,. Talk to any DB engineer who works with large data sets... he will tell you exactly the same thing.
Yes and obviously the guys at CCP never thought of this. And have no reason not to do this, cause they're huge noobs at runnning a DB.
|
Lazuran
|
Posted - 2006.09.12 14:05:00 -
[51]
Originally by: Larshus Magrus
1) How do you keep the most used tables in memory? You don't. The OS does. Thats its job. Any good OS worth its salt will have no problems (asuming the DB is tuned correctly) maintaining huge tables in memory.
An OS, which does this better than an experienced DBA, does not exist yet, as far as I know.
Quote:
2) How do you have 250+ gigs of memory? My intel xeon box only support16/32 gigs!?! Buy some real hardware. The transactions eve is pushing need massive amounts of ram to scale. Even the new 64 gig 51xx series xeon motherboards arent going to cut it. ite the bullet, buy a real piece of EXPANDABLE INM/Sun hardware with a real OS and jsut be done with it. You are spending silly money on the ramsans which dont FIX anything.. just extend the problem out further.
Meh, there are at least 2 "cheap" x86 configurations out there with support for 128GB RAM and 16 CPU cores (e.g. Iwill's Opteron boxes). Perhaps you can go even higher (Opterons can address much more physical RAM, I don't know which existing chipsets support more than 128GB though), but the x86 architecture is somewhat limiting and I'm not aware of an MS SQL port to other hardware. It does support clustering though (but I don't know whether this improves performance or provides only failover-capabilities). And no, I wouldn't want to wait for CCP to port the database to Oracle ;-).
"The whole of NYC is not 1.0. Some back alley in the Bronx is deep 0.0, while right outside NYPD headquarters is 1.0." -- Slaaght Bana |
Isyel
Minmatar
Masuat'aa Matari
Ushra'Khan
|
Posted - 2006.09.12 15:03:00 -
[52]
I still say i barely ever have any lag (even when there were 150 people in local and some rather big fights with lots of frig action lately going on ). Now, i'm referring to pure server - client relation lag.
I do have issues with lag, but it's mainly graphical, or so it appears. The game stutters like crazy in any moderate fight or POS, while the ship and modules react normally. You turn off the UI and it's better, but still stutters. Now perhaps there is something to do with the normal lag but since modules and commands respond normally, meh. (yes, i have plenty of power to run eve, Oblivion runs smoothly :P). THAT would be my main EvE problem. 
|
Death Kill
Caldari
direkte
|
Posted - 2006.09.12 15:22:00 -
[53]
Originally by: Larshus Magrus
Ok you say but:
2) How do you have 250+ gigs of memory?
I dont have a clue about all this tech stuff, but the server at work have over 280gig ram.
Recruitment |
Larshus Magrus
Caldari Provisions
|
Posted - 2006.09.12 15:27:00 -
[54]
Originally by: Gariuys
Yes and obviously the guys at CCP never thought of this. And have no reason not to do this, cause they're huge noobs at runnning a DB.
Apparently, yes. I know your tone was sarcastic, but they do seem to be noobs at running very large databases. If you look around at other very large databases no one does it the way CCP is doing it. That does tend to lend itself to question their approach.
|
Larshus Magrus
Caldari Provisions
|
Posted - 2006.09.12 15:32:00 -
[55]
Originally by: Lazuran
Edited by: Lazuran on 12/09/2006 14:20:03
Originally by: Larshus Magrus
1) How do you keep the most used tables in memory? You don't. The OS does. Thats its job. Any good OS worth its salt will have no problems (asuming the DB is tuned correctly) maintaining huge tables in memory.
An OS, which does this better than an experienced DBA, does not exist yet, as far as I know.
Quote:
2) How do you have 250+ gigs of memory? My intel xeon box only support16/32 gigs!?! Buy some real hardware. The transactions eve is pushing need massive amounts of ram to scale. Even the new 64 gig 51xx series xeon motherboards arent going to cut it. ite the bullet, buy a real piece of EXPANDABLE INM/Sun hardware with a real OS and jsut be done with it. You are spending silly money on the ramsans which dont FIX anything.. just extend the problem out further.
Meh, there are at least 2 "cheap" x86 configurations out there with support for 128GB RAM and 16 CPU cores (e.g. Iwill's Opteron boxes). Perhaps you can go even higher (Opterons can address much more physical RAM, I don't know which existing chipsets support more than 128GB though), but the x86 architecture is somewhat limiting and I'm not aware of an MS SQL port to other hardware. It does support clustering though (but I don't know whether this improves performance or provides only failover-capabilities). And no, I wouldn't want to wait for CCP to port the database to Oracle ;-).
Edit: how wrong I was... MS SQL apparently works on HP Superdomes with Itanium CPUs (they support 1TB RAM easily):
http://www.tpc.org/results/individual_results/HP/hp_orca1tb_win64_ex.pdf
I was refering to CCP's tendancy to only use Intel x86 hardware on thier DB boxes. If you look at thier current hardware and thier past hardware, they have stuck to xeon's. My guess is that benchmarking M$Sql it might perform better on xeon boxes then optrons.. thats just a guess though.
However you are correct. There are 8 way optron boxes that can handle large amounts of buffered ram. However I hear their are wierd performance issues due to the large parallel signaling lanes. Intel is getting around this by going with serial FBDIMMS. We should see much larger capacity xeon boxes in teh future.
As far as Itanium. A completely viable archetecture. Moving to it would be a good thing. However Miscrosft canceled windows for itanium so they would have to move to an alternate OS.
|
Larshus Magrus
Caldari Provisions
|
Posted - 2006.09.12 15:33:00 -
[56]
Originally by: Death Kill
Originally by: Larshus Magrus
Ok you say but:
2) How do you have 250+ gigs of memory?
I dont have a clue about all this tech stuff, but the server at work have over 280gig ram.
Uh. hats not possible unless its a big boy box. Are you sure you don't mean 280 Megs of ram?
|
Abye
|
Posted - 2006.09.12 15:36:00 -
[57]
Originally by: Larshus Magrus
Originally by: Death Kill
Originally by: Larshus Magrus
Ok you say but:
2) How do you have 250+ gigs of memory?
I dont have a clue about all this tech stuff, but the server at work have over 280gig ram.
Uh. hats not possible unless its a big boy box. Are you sure you don't mean 280 Megs of ram?
Once you stop using X86 toys and use hardware that has been designed for serverwork from the beginning it is not that hard to pull that off.
|
Splagada
Minmatar
Tides of Silence
|
Posted - 2006.09.12 15:49:00 -
[58]
Originally by: Valar
You are right, the problem with the RAMSAN is not the bandwidth, but the capacity.
Most of the database is on a 30 disk fiber optic array, a few indexes on a 10 disk fiber optic array and the heavily used stuff, like the items table, a few related tables and indexes that come with 'em, the transaction log and tempdb are on the RAMSAN.
Due to the lack of space on the RAMSAN recently, I've had to move objects from the RAMSAN to the disk arrays, and I've had to take measures so that in case the transaction log grows too much it starts expanding on to the disk array aswell.
The performance hit of moving the objects from the RAMSAN was not as much as we had feared, but when the transaction log has been on the disk array performance has suffered quite a bit. That however has only happened in exceptional circumstances and I always fix that as soon as I notice.
i just had a geekgasm
-
Tides of Silence recruiting miners and overall fun people |
Jobie Thickburger
Gallente
Intergalactic Mining
|
Posted - 2006.09.12 17:42:00 -
[59]
Originally by: Larshus Magrus
Originally by: Gariuys
Yes and obviously the guys at CCP never thought of this. And have no reason not to do this, cause they're huge noobs at runnning a DB.
Apparently, yes. I know your tone was sarcastic, but they do seem to be noobs at running very large databases. If you look around at other very large databases no one does it the way CCP is doing it. That does tend to lend itself to question their approach.
Sometimes it pays to think outside the Box though, Just because the "Tried and True" Method works, Dosen't mean you can't come up with a better way to do it...
Grats CCP on the record and all, but could someone, Espically the OP, Post a link to the source of this infomation? I hate it when people quote and don't tell where it came from. Easy way to make fabricated infomation seem true...
Retired CEO, MGTTG
|
Matthew
Caldari
BloodStar Technologies
|
Posted - 2006.09.12 17:51:00 -
[60]
Originally by: Larshus Magrus
Originally by: Gariuys
Yes and obviously the guys at CCP never thought of this. And have no reason not to do this, cause they're huge noobs at runnning a DB.
Apparently, yes. I know your tone was sarcastic, but they do seem to be noobs at running very large databases. If you look around at other very large databases no one does it the way CCP is doing it. That does tend to lend itself to question their approach.
But those databases aren't running at the heart of the TQ cluster, are they? Different uses call for different implementations.
Do you know the interface requirements of the rest of the cluster with the DB? Do you know the patterns of access, read/write ratios, the data structures used? Do you, in fact, know anything at all about the DB they are running that would allow you to make an even remotely informed evaluation of their implementation?
Unless you've somehow stolen Valar's DBA login, I seriously doubt it.
-------
There is no magic Wand of Fixing, and it is not powered by forum whines. |
Hllaxiu
Shiva
Morsus Mihi
|
Posted - 2006.09.12 18:40:00 -
[61]
To the monday morning DBAs, why don't you send CCP your resume? They're looking at hiring a DBA:
http://www.ccpgames.com/jobs/default.asp
---
Our greatest glory is not in never failing, but in rising up every time we fail. - Emerson |
Magunus
The Forsakened Few
The ARR0W Project
|
Posted - 2006.09.12 18:46:00 -
[62]
Originally by: Hllaxiu
To the monday morning DBAs, why don't you send CCP your resume? They're looking at hiring a DBA:
http://www.ccpgames.com/jobs/default.asp
Hrm... I wonder if they'd let me take both the DBA and assistant chef jobs...
---
In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move.
-- Douglas Adams, 'The Restaurant at the End of the Universe' |
Jobie Thickburger
Gallente
Intergalactic Mining
|
Posted - 2006.09.12 18:46:00 -
[63]
Originally by: XiticiX
Linkage
Shiney, Thanks.
And once again, WTG CCP!
Retired CEO, MGTTG
|
Sevarus James
Minmatar
Meridian Dynamics
|
Posted - 2006.09.12 18:56:00 -
[64]
Originally by: Matthew
Originally by: Larshus Magrus
Originally by: Gariuys
Yes and obviously the guys at CCP never thought of this. And have no reason not to do this, cause they're huge noobs at runnning a DB.
Apparently, yes. I know your tone was sarcastic, but they do seem to be noobs at running very large databases. If you look around at other very large databases no one does it the way CCP is doing it. That does tend to lend itself to question their approach.
But those databases aren't running at the heart of the TQ cluster, are they? Different uses call for different implementations.
Do you know the interface requirements of the rest of the cluster with the DB? Do you know the patterns of access, read/write ratios, the data structures used? Do you, in fact, know anything at all about the DB they are running that would allow you to make an even remotely informed evaluation of their implementation?
Unless you've somehow stolen Valar's DBA login, I seriously doubt it.
You don't have to have Valar's login to understand the fundamental problem. It AIN'T the database that's at the heart of their system...its the operating system that CONTROLS the database...and everything else in their system.
Working in a command center as an analyst supporting fortune 200 companies world wide, I don't have to be a DBA to see which platforms "just work" and which platform has interminable issues.
CCP can hotfix, patch, upgrade till their wallets turn inside out, but if the foundation of the system is made of clay, it doesn't matter WHAT the rest of it is built from, it will STILL have problems.
The best rant against microsoft ever written...and the longest...and the most detailed.
-----
------------
Updated Linux Desktop+EVE+EVE-TV |
Matthew
Caldari
BloodStar Technologies
|
Posted - 2006.09.12 19:38:00 -
[65]
Originally by: Sevarus James
You don't have to have Valar's login to understand the fundamental problem. It AIN'T the database that's at the heart of their system...its the operating system that CONTROLS the database...and everything else in their system.
Working in a command center as an analyst supporting fortune 200 companies world wide, I don't have to be a DBA to see which platforms "just work" and which platform has interminable issues.
No, you know what "just works" on the systems you work with. TQ is not one of those systems. It is not running the software those systems run.
Without understanding both the system you know, and TQ, you have no way of knowing that your "just works" will transfer accross in the way you expect. Basing any computer-related design on "it just works" assumptions is a recipe for disaster. It's the sort of attitude that means you launch a rocket where one part of the launch system is talking in 64-bits, but the other bit is reading it in as 16-bit actually happened. That "just worked"....right up to the point where it didn't.
Originally by: Sevarus James
The best rant against microsoft ever written...and the longest...and the most detailed.
Ahh, thank you for revealing your real agenda.
-------
There is no magic Wand of Fixing, and it is not powered by forum whines. |
Taedron
Minmatar
Republic University
|
Posted - 2006.09.12 20:12:00 -
[66]
Originally by: LUGAL MOP'N'GLO
Impressive no doubt. It still can't handle uber bookmark copying though... It has been pretty bad the last couple of days.
If it were really in a database, bookmark copying would be trivial. Just add a row to a table.
|
Hllaxiu
Shiva
Morsus Mihi
|
Posted - 2006.09.12 20:22:00 -
[67]
Originally by: Taedron
Originally by: LUGAL MOP'N'GLO
Impressive no doubt. It still can't handle uber bookmark copying though... It has been pretty bad the last couple of days.
If it were really in a database, bookmark copying would be trivial. Just add a row to a table.
Yeah, the problem from what I understand is that theres far to many rows in that table. 
---
Our greatest glory is not in never failing, but in rising up every time we fail. - Emerson |
Eilie
Minmatar
|
Posted - 2006.09.12 21:10:00 -
[68]
Originally by: Sevarus James
The best rant against microsoft ever written...and the longest...and the most detailed.
Well that was very interesting and I just learned alot (assuming it's all true) and it wasn't really a rant either...
Best quote from that site:
Quote:
"Windows [n.]
A thirty-two bit extension and GUI shell to a sixteen bit patch to an eight bit operating system originally coded for a four bit microprocessor and sold by a two-bit company that can't stand one bit of competition."
|
Sevarus James
Minmatar
Meridian Dynamics
|
Posted - 2006.09.13 03:15:00 -
[69]
Originally by: Matthew
Originally by: Sevarus James
You don't have to have Valar's login to understand the fundamental problem. It AIN'T the database that's at the heart of their system...its the operating system that CONTROLS the database...and everything else in their system.
Working in a command center as an analyst supporting fortune 200 companies world wide, I don't have to be a DBA to see which platforms "just work" and which platform has interminable issues.
No, you know what "just works" on the systems you work with. TQ is not one of those systems. It is not running the software those systems run.
Without understanding both the system you know, and TQ, you have no way of knowing that your "just works" will transfer accross in the way you expect. Basing any computer-related design on "it just works" assumptions is a recipe for disaster. It's the sort of attitude that means you launch a rocket where one part of the launch system is talking in 64-bits, but the other bit is reading it in as 16-bit actually happened. That "just worked"....right up to the point where it didn't.
Originally by: Sevarus James
The best rant against microsoft ever written...and the longest...and the most detailed.
Ahh, thank you for revealing your real agenda.
Actually Matt, there isn't an agenda. As for the comments I made about 'just works', it was more an overall 'look' at various systems. In my line of work I support multiple server OS's, deal with middleware, DBA support across various platforms for MULTIPLE customers with differing needs and contracts.
I don't particularly care if its windows xp/64, 2000, 2003, or linux, aix, what have you. I'm only telling you that in a bigger picture sense, the platform that is WEAKEST and most problematic is the underlying windows OS.
The link was something a co-worker gave me after beating heads against the wall on DB problems for a world wide client (think of a famous mouse), that turned out to be operating system related...guess which operating system?
There doesn't have to be an agenda to speak from experience. You of ALL people should know that. Of course if you are a 'defender of the faith' then its a different story. Me, I prefer to use what works and is inherently secure from both a client and server standpoint. In my 20 years in the biz, it ain't nor has it EVER BEEN an MS product.
-----
------------
Updated Linux Desktop+EVE+EVE-TV |
Miss Overlord
Gallente
Garoun Investment Bank
|
Posted - 2006.09.13 05:04:00 -
[70]
well 40k is next goal and that will happen pretty quickly now if china and tq where on same server we would already be hitting 70k concurrent
|
Tehyarec
|
Posted - 2006.09.13 06:33:00 -
[71]
If only they could (afford to?) double the cluster hardware. Maybe then it would lag less along with the code/database optimizations Not that it'd be too bad for me most of the time except in the very busy systems, though after Dragon I've been seeing a weird amount of random disconnects.
|
Matthew
Caldari
BloodStar Technologies
|
Posted - 2006.09.13 08:28:00 -
[72]
Originally by: Sevarus James
There doesn't have to be an agenda to speak from experience. You of ALL people should know that. Of course if you are a 'defender of the faith' then its a different story. Me, I prefer to use what works and is inherently secure from both a client and server standpoint. In my 20 years in the biz, it ain't nor has it EVER BEEN an MS product.
I'm not trying to say MS products are perfect, but a lot of people do use them very successfully. While there is certainly plenty to critique about them, there's a lot of anti-MS stuff around that's just there because it's fashionable to bash them, rather than it having a proper founding. I've seen many cases of strange behaviour that people have blamed on MS, which eventually turned out to be something they messed up - it's got too easy to blame all the computing world's woes on them.
That's why arguments that use the broad-brush "MS Sucks" approach really get on my nerves - if there are specific known issues, then specify them.
Similarly, the "I see more MS products go wrong than anything else" argument is fundamentally flawed without several caveats that most using that argument don't even bother thinking about. After all, I've seen more home PC's running windows break than anything else. But then again I've seen far more home PC's running windows than anything else. If I know 50 people using windows, 5 using linux and 1 using a mac, and I've seen 10 windows boxes go wrong, 1 linux box go wrong and no macs go wrong, then I can say in full truth that "I see more MS products go wrong than anything else" and "I've never seen a mac go wrong", but on that evidence, you cannot say that the MS product is less reliable, or that the mac is the most reliable.
Then there's consideration of the relative competencies of the users. A lot of MS-related problems I've seen have far more to do with the competency of the user than the robustness of the system. Systems like linux tend to apply their own "idiot-filter" through their very nature and reputation.
Of course, even if we did manage to show that something else is a better technical solution right now, there's still the issue of whether it is a practical solution. How much would these alternatives cost, both in initial investment and ongoing support costs (24/7 top-grade on-site support)? How much would it cost to expand these alternatives later? How much would it cost to port the alternatives, both for the DB itself and the rest of the eve cluster dependant on it. How much testing would be required, and what are the risks if something goes wrong? How long would the whole process take, and where do you expect the system to be by then (the best solution now may be obsolete by the time it gets implemented)? It's actually very rare to have the best technical solution implemented, not through incompetence, but because of these sort of practical concerns.
That's where the RAMSAN upgrades really win big - practicality. They're specifically designed to look like "just another drive" to whatever they're plugged into (just a really, really fast one). As such, there's no need to change code, port databases, or test whole new systems. You just unit test and integrate it like any other drive.
Also when looking back at their choices, you have to remember the development path Eve has been down. What may seem like practical solutions now would have been complete overkill, and probably cripplingly expensive when the original TQ architecture was being put together for a much lower number of users.
-------
There is no magic Wand of Fixing, and it is not powered by forum whines. |
BurnHard
|
Posted - 2006.09.13 09:01:00 -
[73]
I use SQL Server 2005 at work every day (not as a DBA, as a developer) and I can safely say it's a highly robust, high performance solution. I don't think the issue is the DBMS at all because it can only be as good as the model it's implementing. I'm pretty sure CCP would probably code it differently if they had to start over but it's too late now. You just have to throw more metal at the problem and tweak here and there where you can.
|
Sevarus James
Minmatar
Meridian Dynamics
|
Posted - 2006.09.13 11:13:00 -
[74]
finis'
Originally by: Matthew
Of course, even if we did manage to show that something else is a better technical solution right now, there's still the issue of whether it is a practical solution. How much would these alternatives cost, both in initial investment and ongoing support costs (24/7 top-grade on-site support)? How much would it cost to expand these alternatives later? How much would it cost to port the alternatives, both for the DB itself and the rest of the eve cluster dependant on it. How much testing would be required, and what are the risks if something goes wrong? How long would the whole process take, and where do you expect the system to be by then (the best solution now may be obsolete by the time it gets implemented)? It's actually very rare to have the best technical solution implemented, not through incompetence, but because of these sort of practical concerns.
You make it sound like I'm demanding immediate changes. If you read the above statements regarding WHAT I do, then you should know I don't advocate throwing babies out with bathwater. They (ccp) are doing what they have to do to keep things running. My point is that there are better ways to accomplish the same things more efficiently. Is this something done over night? hell no. But as they grow, and evolve, they as a company need to realize that just throwing more hardware at a problem is not going to work forever. Especially if the numbers of customers escalate as they are predicting. At some point the weakness of the system cannot be overcome, and will have to be dealt with.
Originally by: Matthew
That's where the RAMSAN upgrades really win big - practicality. They're specifically designed to look like "just another drive" to whatever they're plugged into (just a really, really fast one). As such, there's no need to change code, port databases, or test whole new systems. You just unit test and integrate it like any other drive.
I agree that the RAMSAN drives ARE a big boost and bonus...but my point is still there. The controlling piece of the puzzle is inherently NOT robust. MS -SQL is a decent product. It had better be, MS bought the thing from Sybase. (notice they didn't code it...they BOUGHT it.) The underlying OS is the weak link. That was the point I was making all along.
Originally by: Matthew
Also when looking back at their choices, you have to remember the development path Eve has been down. What may seem like practical solutions now would have been complete overkill, and probably cripplingly expensive when the original TQ architecture was being put together for a much lower number of users.
I wouldn't argue this one way or another. CCP used what they knew. What they have accomplished with the software/server/hardware that they HAVE used continually amazes me. (in a good way). That they DIDN'T have to go through these kinds of hoops is the sad part to me. Then again, microsoft has kept me in house and home as an engineer with rediculous issues over the years, so I shouldn't complain too much.
thank you for an articulate debate on this however. It is always refreshing to see intelligence and reasoning rather than 'you suxor'.
Sev
-----
------------
Updated Linux Desktop+EVE+EVE-TV |
Sevarus James
Minmatar
Meridian Dynamics
|
Posted - 2006.09.13 11:13:00 -
[75]
Edited by: Sevarus James on 13/09/2006 11:16:14
Originally by: Matthew
I'm not trying to say MS products are perfect, but a lot of people do use them very successfully. While there is certainly plenty to critique about them, there's a lot of anti-MS stuff around that's just there because it's fashionable to bash them, rather than it having a proper founding. I've seen many cases of strange behaviour that people have blamed on MS, which eventually turned out to be something they messed up - it's got too easy to blame all the computing world's woes on them.
You really should read the 'rant'. The author isn't bashing out of a 'fashionable' reason, but the history, tendendencies and proclivities of MS. The EU and DOJ didn't come up with the decisions AGAINST microsoft for nothing.
Originally by: Matthew
That's why arguments that use the broad-brush "MS Sucks" approach really get on my nerves - if there are specific known issues, then specify them.
-hence my pointer to an indepth examination of them. Why rewrite what is already well documented, entertainingly written as well as informative?
Originally by: Matthew
Similarly, the "I see more MS products go wrong than anything else" argument is fundamentally flawed without several caveats that most using that argument don't even bother thinking about. After all, I've seen more home PC's running windows break than anything else. But then again I've seen far more home PC's running windows than anything else. If I know 50 people using windows, 5 using linux and 1 using a mac, and I've seen 10 windows boxes go wrong, 1 linux box go wrong and no macs go wrong, then I can say in full truth that "I see more MS products go wrong than anything else" and "I've never seen a mac go wrong", but on that evidence, you cannot say that the MS product is less reliable, or that the mac is the most reliable.
I'm not talking about home users. Or the clients. I'm talking about robust server architecture based upon TIGHT coding principles and a firm and experienced grasp of networking, security and all that goes with it. Again, I refer to the article linked. (and oh by the way, the author does NOT get into deep comparisons between platforms. His take is to just show microsoft in harsh light by its own history and products.)
Originally by: Matthew
Then there's consideration of the relative competencies of the users. A lot of MS-related problems I've seen have far more to do with the competency of the user than the robustness of the system. Systems like linux tend to apply their own "idiot-filter" through their very nature and reputation.
As have I. (competencies.) However, my point of view is from a different plane of existence than 'end users'. I'm coming from servicing fortune 200 companies from a SERVER/networking perspective. I don't know if I made that clear enough, but I work in a global command center, and not only have access to, but deal with multiple environments across the planet with user bases that absolutely DWARF EVE's userbase and overall usage.
I SEE, on a daily basis, the issues and problems, and to my point, the platform with repeated issues, races, memory problems/leaks, security flaws (on an ONGOING daily basis) is far and away the micosoft offerings more than ANY other platform.
[cont.]
-----
------------
Updated Linux Desktop+EVE+EVE-TV |
Miss Overlord
Gallente
Garoun Investment Bank
|
Posted - 2006.09.13 11:14:00 -
[76]
some love microsoft some hate em either way cant live with em and cant really live without em
|
Lazuran
|
Posted - 2006.09.13 11:32:00 -
[77]
Edited by: Lazuran on 13/09/2006 11:34:11
Originally by: BurnHard
I'm pretty sure CCP would probably code it differently if they had to start over but it's too late now. You just have to throw more metal at the problem and tweak here and there where you can.
Everyone who has ever built a performance-critical DB that is actually being used in a production environment, would have to agree with you there.
The same probably applies to their client<->server protocol, data formats, the UI etc. etc.. It takes a disproportionate amount of manpower to fix issues in the underlying architecture and it's dangerous to attempt to do it while trying to provide seamless updates and an uninterrupted service.
So, let's hope for EVE2 ;-).
"The whole of NYC is not 1.0. Some back alley in the Bronx is deep 0.0, while right outside NYPD headquarters is 1.0." -- Slaaght Bana |
| |
|
| Pages: 1 2 3 :: [one page] |