Pages: 1 2 3 :: [one page] |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 02:18:00 -
[1]
I've know 'lag' since the beginning. I believe CCP totally understands the issue also but there is some limitation it can not repair in the immediate future.
* Is it a hardware issue? Lag tends to be ok when the fleet are in position but as more and more commands are pressed, it seems like a backlog of commands piles up giving this delay.
* Does the server sort command based on user latency? That would sort out more commands faster by bias certain regional players.
My questions,
WHAT EXACTLY IS CAUSING LAG? DOES CCP UNDERSTAND THE CAUSE EXACTLY? IS THERE SOME FUNDAMENTAL LIMITATION IN TECHNOLOGY THAT WE ARE REACHING IN THIS GAME PREVENTING A FIX?
I appreciate everyone's opinion on the matter. |

DroneCommander
Gallente
|
Posted - 2008.04.15 02:20:00 -
[2]
People. |

Havohej
The Defias Brotherhood ANTHRAX DEATH
|
Posted - 2008.04.15 02:21:00 -
[3]
/me would like to know more about what actually causes this as well, other than the whole 'Jita doesn't lag 'cause eve is broken, Jita lags because people keep going UNTIL it lags' routine. (same goes for fleets, btw) |

Liang Nuren
Black Sea Industries Insurgency
|
Posted - 2008.04.15 02:23:00 -
[4]
Sigh, yes, they know. You really don't know how complicated the answer to your question is, or you wouldn't have asked it.
-Liang |

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 02:27:00 -
[5]
I wonder if Jita has more hardware dedicated to it. The entire 'reinforcing' nodes...I wonder how well that works? One should remember that the totality of fleet battle commands (and the fact that all the players are on the same grid) sounds like its a lot more demanding computationally. Does that mean its a mere CPU issue? Or is it a network speed issue? Or what?
I know we have some pros on this stuff on these boards whose opinions would be insightful. |

Crackzilla
The Shadow Order SMASH Alliance
|
Posted - 2008.04.15 02:29:00 -
[6]
To quote a recently podded cliche:
Lag is people!
|

Lithalnas
Amarr Headcrabs
|
Posted - 2008.04.15 02:31:00 -
[7]
lag is people, now in systems like motsu lag is isk farmers, ban the isk farmers to make regular mission runners happy. |

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 02:32:00 -
[8]
I noticed the correlation between lag and numbers a very long time ago.
I'm after something more detailed (I'm a physicist in the real world). |

Havok Dryke
Golden Gavel Enterprises The Cooperative
|
Posted - 2008.04.15 02:41:00 -
[9]
Edited by: Havok Dryke on 15/04/2008 02:41:20 The primary cause of lag are threads about the causes of lag. --------------------------------------
EVE is a cold, harsh world, filled with people that would kill you without a second thought. The forums are even worse. |

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 02:52:00 -
[10]
Glad to see everyone is as clueless about the issue as I.
|
|

Animin Mannja
|
Posted - 2008.04.15 02:56:00 -
[11]
Probably scalability issues. All the neat things that make things interesting, for nonsensical exmple: shiny ships with role bonuses affecting fleet members overheating their turret tracking speed with specialised ammo against rigged ships piloted by pilots with boosters and implants and skills and all sorts of bling, all add to the complexity of situations and thus the computational power required to resolve them. Of course, Jita's probably also collapsing under the weight of Market orders being updated several hundred times a second, and making sure that you can actually buy what you can see and that it hasn't changed in the last 30th of a second.
All this sort of stuff taxes resources to the limit. Seems we've been hitting the limits more often recently though, new people trying out new features, etc. Basically, the system isn't scaling as well as CCP would hope.
The short term fix is "Throw more power at it!" which is what is frequently spouted around here. That just extends the limit a bit, and is really impractical to be done on a large scale - dedicated nodes for each system just means 2/3rds of the shiny expensive equipment sitting idle as they never get close to the limit. The long term fix is to improve scalability by re-working some of the base code and infrastructure, compartmentalising it so the workload can be distributed better across multiple servers. This of course takes a lot of time and effort, and may not be possible in some cases - some things can't be separated.
|

Hannobaal
Gallente Shadow Forces Inc.
|
Posted - 2008.04.15 02:57:00 -
[12]
Edited by: Hannobaal on 15/04/2008 02:57:07
Originally by: Sionn Klorgh
WHAT EXACTLY IS CAUSING LAG?
Too many people in the same place doing too many things. What else could it possibly be?
Quote: DOES CCP UNDERSTAND THE CAUSE EXACTLY?
I think it's a pretty safe assumption to say that they understand the causes better and in more detail than any of us possibly could.
Quote: IS THERE SOME FUNDAMENTAL LIMITATION IN TECHNOLOGY THAT WE ARE REACHING IN THIS GAME PREVENTING A FIX?
Yeah.
|

Jovian Strain
The Collective Against ALL Authorities
|
Posted - 2008.04.15 02:59:00 -
[13]
Quote: Too many people in the same place doing too many things. What else could it possibly be?
No doubt. Is that a CPU issue or a network issue or what?
|

Animin Mannja
|
Posted - 2008.04.15 03:18:00 -
[14]
Originally by: Jovian Strain
Quote: Too many people in the same place doing too many things. What else could it possibly be?
No doubt. Is that a CPU issue or a network issue or what?
Probably both. Maybe not at the same time. I'm sure there are instances where the server can't keep up with computational tasks, and others where it can't spit out info to clients fast enough. I'd expect there are many different events affecting many different aspects of the servers, all of which would be seen as just 'lag'.
|

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 03:21:00 -
[15]
Quote: Probably scalability issues. All the neat things that make things interesting, for nonsensical exmple: shiny ships with role bonuses affecting fleet members overheating their turret tracking speed with specialised ammo against rigged ships piloted by pilots with boosters and implants and skills and all sorts of bling, all add to the complexity of situations and thus the computational power required to resolve them. Of course, Jita's probably also collapsing under the weight of Market orders being updated several hundred times a second, and making sure that you can actually buy what you can see and that it hasn't changed in the last 30th of a second.
All this sort of stuff taxes resources to the limit. Seems we've been hitting the limits more often recently though, new people trying out new features, etc. Basically, the system isn't scaling as well as CCP would hope.
The short term fix is "Throw more power at it!" which is what is frequently spouted around here. That just extends the limit a bit, and is really impractical to be done on a large scale - dedicated nodes for each system just means 2/3rds of the shiny expensive equipment sitting idle as they never get close to the limit. The long term fix is to improve scalability by re-working some of the base code and infrastructure, compartmentalising it so the workload can be distributed better across multiple servers. This of course takes a lot of time and effort, and may not be possible in some cases - some things can't be separated.
So ultimately, your view is that its a CPU issue. A typical CPU gets so many node to handle. Through 500 people fleet battle on that node and it just runs out of steam.
- I have heard that putting more cpu onto a node helps (reinforcing a node). - Pity this can't be dynamic based on the number of people in a system. I believe 'reinforcing' can only be done at downtime.
|

Animin Mannja
|
Posted - 2008.04.15 03:33:00 -
[16]
Originally by: Sionn Klorgh
So ultimately, your view is that its a CPU issue. A typical CPU gets so many node to handle. Through 500 people fleet battle on that node and it just runs out of steam.
- I have heard that putting more cpu onto a node helps (reinforcing a node). - Pity this can't be dynamic based on the number of people in a system. I believe 'reinforcing' can only be done at downtime.
IMO, in a lot of instances it's probably a CPU type issue. Might not be CPU per se, might be RAM, bus/disk I/O, etc, etc. Basically it can't keep up. I suspect 'reinforcing a node' is basically putting the particular systems on their own heavy-duty server, which would only be able to be done at downtime without interrupting anything, so is unable to be done dynamically with the current infrastructure.
However I believe Jita is already on a permanent reinforced node, so it's obviously not a cure-all. I don't think there can be a cure-all with the current infrastructure, which just means lots of threads like this until they finally finish Infiniband or whatever they've got in the pipeline at the moment.
|

Oldin Kinrod
|
Posted - 2008.04.15 03:34:00 -
[17]
A couple of dev blogs about lag:
The EVE Cluster Optimizations, Optimizations, Optimizations (Has animation explaining it) People stealing your wrecks? Players intruding your missions? |

Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2008.04.15 04:04:00 -
[18]
Client-side lag = not enough computing power to render the game (doesn't affect much except your FPS) Network lag = not enough bandwidth to transfer all data, or a bad response time (hardly the case these days, but can happen) SERVER-SIDE lag = not enough CPU to compute all user actions (the main cause of "lag" in this game)
The answer is to simply just throw more computing power at it.
Ok, it's a SIMPLE answer, but it's not an EASY answer. The big problem with this answer is economic concerns... yeah, you COULD have each solar system on a machine that could handle a couple of thousands of people doing their thing, however that machine might cost more than the whole yearly INCOME of CCP. So, you have to prioritize.
Meanwhile, a project is underway to try and change the way server-side simulations are handled, from "several solar systems on a certain node" (like it is now, with a few nodes running single solar systems, like Jita, for instance) to a more fluid approach, where SEVERAL nodes could be used to handle a single solar system, and "on the fly" rerouting of server hardware where needed. Sadly, this is not a project you do overnight, and it might be YEARS before that finally happens, IF it happens at all.
1|2|3|4|5. |

Rawr Cristina
Caldari Naqam
|
Posted - 2008.04.15 04:12:00 -
[19]
Originally by: Hannobaal
Too many people in the same place doing too many things.
with no mechanics, other than that of lag, to persuade people to avoid such places. ...
|

Pithecanthropus
|
Posted - 2008.04.15 04:13:00 -
[20]
Bad programming Lack of server dedication Never made preparations for game growth Adding content before adding quality Too much money makes them lazier
All of the above leads to lag. Players don't cause lag, we're victims of it due to the problems CCP refuses to fix.
|
|

Crackzilla
The Shadow Order SMASH Alliance
|
Posted - 2008.04.15 04:21:00 -
[21]
Originally by: Pithecanthropus Bad programming Lack of server dedication Never made preparations for game growth Adding content before adding quality Too much money makes them lazier
Generally its a business issue. No sane business person would sign off on developing a massively scalable system to be deployed years from the planning stage without customers. Now they've customers and a business need.
Some groups are tasked on performance but not all can be. Its just a matter of time. On the other hand the roi case may say to keep things as is and do nothing. Who knows.
|

Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2008.04.15 04:21:00 -
[22]
Originally by: Pithecanthropus Bad programming Lack of server dedication Never made preparations for game growth Adding content before adding quality Too much money makes them lazier
All of the above leads to lag. Players don't cause lag, we're victims of it due to the problems CCP refuses to fix.
Would that be the reason that "once upon a time" 50vs50 were already not just lagfests, but freezefests, yet now you can even have 200vs200 and be "only" lagfests ? Or is that the reason why Yulai, then Jita used to croak at 200-300-ish in local, but now you can take a stroll at 600 in local and still do your thing ? Or that at 800, it's still barely playable ? And only near 1k it finally croaks ?
Yup, there was absolutely no progress, no sir...
1|2|3|4|5. |

Pithecanthropus
|
Posted - 2008.04.15 04:27:00 -
[23]
Edited by: Pithecanthropus on 15/04/2008 04:30:16
Originally by: Akita T
Yup, there was absolutely no progress, no sir...
If progress led to where they are at... progress can lead to better. Thus, they are still bad, lacking, not making adjustments, and turning their heads from the problems.
I refuse to be at fault for lag, when the game THEY created molds us to do what we do. If you make a narrow tunnel and tell 30,000 people to run thru it, you'll get stuck... same goes for this game. CCP made it, CCP is at fault for opening the doors and letting us get stuck. CCP is at fault for putting out more quantity and less quality. I play a game, I play with the mechanics they gave me... why should lag be MY fault?
CCP has the accountability... you fail.
EDIT: and to be honest... I'm the one that has to adjust the game to make it play less lag free. Cliking off effects, no sound, no high graphics content... so yea... I'm doing what I can do. Taint my fault if I fight 200 vs 200... CCP wants us to. They just aren't prepared for it.
|

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 05:05:00 -
[24]
Quote: Meanwhile, a project is underway to try and change the way server-side simulations are handled, from "several solar systems on a certain node" (like it is now, with a few nodes running single solar systems, like Jita, for instance) to a more fluid approach, where SEVERAL nodes could be used to handle a single solar system, and "on the fly" rerouting of server hardware where needed.
So a CPU issue. The nodes are spread out over CPUs and these CPUs are not currently coded to dynamically redistribute load, so we get lag. Extra hardware thus will not do as much as some fundamental re-writing of the source code.
|

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 05:12:00 -
[25]
We REALLY need a sticky about the lag. I have explained this in 6 threads the last week.
The issue is that a system contains certain services. The market, space, etc. etc., and these are forced to be on 1 core(not a cpu, just a core), because the cluster talks to nodes over ethernet connection.
Therefor, the cluster can't scale by redistributing the load between more nodes, due to the ethernet connection having a large latency.
CCP is implementing infiniband to the cluster this year, in order to make it possible, and thus killing the lag-monster.
|

Zeba
Minmatar Pator Tech School
|
Posted - 2008.04.15 05:46:00 -
[26]
There are three types of lag. One is market hub lag with hundreds of trade alts all endlessly refreshing the market interface to gain the .01 isk advantage. Two is empire mission hub lag with thousands of players spamming the same Caldari Command agents thinking they are the only ones to give out faction missiles and CNRs. Three is 0.0 lag due to having whole regions running on one cpu so when a fleet fight happens and some random nub accesses the market the node crashes.
One and Two are totally player driven and CCP will never be able to do anything about it and Three is simply because the people who do fleet fights only make up about 15% of the playerbase and so get the comparative equivilent of server resorces.
Thats about it.  Why not. |

Robdon
|
Posted - 2008.04.15 05:53:00 -
[27]
Bad Coding.
Jugging by the amount of bugs and CCPs inability to fix them in any kind of quick manner, I would say that the state of the code must be appalling.
I've worked with computing for many many years, and most coders donĘt have a clue about performance, or even consider it.
You have to have a couple of guys who dedicate themselves to this area and review / analyse ALL code that gets into the system.
ItĘs a completely different thought process, that CCP just doesnĘt seem to have.
Now obviously I havenĘt seen the code, but things like this....
Fleet battles, the first thing you do is lock a target and fire all your weapons.
This involves clicking on maybe 6-7 (13 or more for ECM boats) modules to activate all your weapons, and thatĘs 7 transactions to the server that need to be processed.
There should be an 'activate all' button, that as a user you can setup and decide what modules get activated with this button, then if this is pressed 1 transaction is sent to the server, instead of 7.
This is just one example that I can see 'should' help.
It might sound harsh, but since trinity and the 'Need for speed', this game has just got worse. So what ever it is their coders are doing itĘs not what is needed. They need to get fresh dedicated coders who know what they are doing in this area, because CLEARY the current ones donĘt know what to do.
|

Zeba
Minmatar Pator Tech School
|
Posted - 2008.04.15 05:55:00 -
[28]
Originally by: Robdon lulz ima idjut
Bad reasoning. Why not. |

CHAOS100
Black Plague. RAZOR Alliance
|
Posted - 2008.04.15 06:04:00 -
[29]
Despite what most people think i'm pretty sure it is not a hardware issue. They have a very powerful computer cluster, which they could play pong quite nicely on. You could have the whole earth as a computer and it will still only be as fast and as good as your programming lets you be. Let's be honest, they developed the primary engine of the game back in 2002 or so? Sure they made a new graphics engine, but that is good for the client not the server. New network coding would probably be very difficult seeing as how it is kind of like trying to remake the foundation of a house. But CCP is just putting more layers on a broken foundation.
I remember distinctly that around the end of Cold War client and the entrance into RMR that lag started to get bad. I am not talking about player increase. I remember that we could have good sized battles right before the patch to RMR, with minimal lag. Then RMR hit and a battle with say 200 people was nearly unplayable. Check out kirkaphoenix's video 'Where have all the good fix gone' which was frapsed right after RMR. The lag in that video in unbearable. The player count did not rise that significantly after the patch to make a major difference. This leads me to believe that whatever code and 'content' that patch introduced suddenly made the servers run much less efficient, which they have been continually trying to fix by improving the server cluster.
So it is probably a coding/software issue, which CCP is probably reluctant to fix due to the large amount of time and effort required to re-code it, and would add no new visible value to the game that say... walking in station would. --------------
|

Inertial
The Python Cartel
|
Posted - 2008.04.15 06:41:00 -
[30]
Lag is caused by the server/network/client needing to compute more info than it can. Just like a Ibis can't break a passive drakes tank, a Intel 4004 can't handle all the instructions EVE gives it in time, before it is given a new set of instructions.
The problem with lag in eve is: Whenever there are a person in the system, the server needs to know;
Where the person is What he is in The condition of his ship The speed of the ship Modules activated etc. etc. etc.
All of this adds up, and when you have 600 people in a system it needs to know this for every one of them. This is especially bad in case of a fleet battle, where information about damage taken and ships destroyed is constantly sent, or Jita, where market information is constantly updated.
|
|

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 07:58:00 -
[31]
Those of you who say its all due to software should really try and read a bit on the topic before you post..
|

Robdon
|
Posted - 2008.04.15 08:28:00 -
[32]
Originally by: LaVista Vista Those of you who say its all due to software should really try and read a bit on the topic before you post..
I absolutely assure you, I have read up about THIS topic ;)
I've coded on systems for over 15 years, and specialise in performance. |

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 08:31:00 -
[33]
Originally by: Robdon
Originally by: LaVista Vista Those of you who say its all due to software should really try and read a bit on the topic before you post..
I absolutely assure you, I have read up about THIS topic ;)
I've coded on systems for over 15 years, and specialise in performance.
So you think that remaking the codebase can completely fix all issues?
In that case, you should get a job at CCP. Because if you can make code that can run between nodes connected with a high-latency and low-bandwidth media like ethernet, you are good at this stuff, and you should be a millionare. |

Robdon
|
Posted - 2008.04.15 08:44:00 -
[34]
Edited by: Robdon on 15/04/2008 08:46:20 Edited by: Robdon on 15/04/2008 08:45:21
Originally by: LaVista Vista
Originally by: Robdon
Originally by: LaVista Vista Those of you who say its all due to software should really try and read a bit on the topic before you post..
I absolutely assure you, I have read up about THIS topic ;)
I've coded on systems for over 15 years, and specialise in performance.
So you think that remaking the codebase can completely fix all issues?
In that case, you should get a job at CCP. Because if you can make code that can run between nodes connected with a high-latency and low-bandwidth media like ethernet, you are good at this stuff, and you should be a millionare.
I'm not a millionaire, but I've made enough money from my skills :)
No it wont fix all the problems, but it will help alot. You don't always have to remake the code base to fix performance issues, yes some things might need changing at a very low level and cost alot of dev time, but there will also be other things that can be done, that over time that will help.
Now this is what the 'Need for Speed' initiative was supposed to do, but I've been playing the game for over 1 year, and so far its just gotten worse.
I've spent years at a large global telecoms company rewriting software to get the max out of performance (without making the code obfuscated), so I kinda know about performance and coding.
I agree that our system would be different to a MMO, but the same principles can be applied.
As for working for CCP, well... that would be a fun idea, but not too sure I would want to live in the freezing cold in Iceland ;)
Any hoo, enough on this, hopefully todays patch might help out....
|

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 08:47:00 -
[35]
Edited by: LaVista Vista on 15/04/2008 08:47:17
Originally by: Robdon
Originally by: LaVista Vista
Originally by: Robdon
Originally by: LaVista Vista Those of you who say its all due to software should really try and read a bit on the topic before you post..
I absolutely assure you, I have read up about THIS topic ;)
I've coded on systems for over 15 years, and specialise in performance.
So you think that remaking the codebase can completely fix all issues?
In that case, you should get a job at CCP. Because if you can make code that can run between nodes connected with a high-latency and low-bandwidth media like ethernet, you are good at this stuff, and you should be a millionare.
I agree with the content of this post.. or lack of really 
EDIT: Bah, it was edited 
|

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 08:50:00 -
[36]
Originally by: Robdon
No it wont fix all the problems, but it will help alot. You don't always have to remake the code base to fix performance issues, yes some things might need changing at a very low level and cost alot of dev time, but there will also be other things that can be done, that over time that will help.
Now this is what the 'Need for Speed' initiative was supposed to do, but I've been playing the game for over 1 year, and so far its just gotten worse.
Well, i think you are forgetting a few things:
The issue right now is that ALL services of a system have to run on the same core. Becuase we can't have quantum processors, it just doesn't scale with systems of 600 people. So while code performance COULD have a slight impact on the lag, it isn't the answer.
The answer is to split the systems of a service between nodes. This isn't possible at present due to the high cost of talking between nodes.
When CCP implements infiniband, its possible.
|

Maglorre
|
Posted - 2008.04.15 08:51:00 -
[37]
Originally by: CHAOS100
oh and ps. the fact that they use windows instead of linux and microsoft SQL instead of MYSQL probably doesnt help either.
I hope that bit was a joke. I'm not going to jump into the windows vs linux debate since I doubt it has a serious impact on performance.
I'm no fan of MS SQL server but MySQL would quite simply choke to death on the load that EVE would produce. Not to mention that it's feature set is nowhere near complete enough (certainly not at the time that EVE was written and migrating something like this would be a very costly exercise)
|

Wild Rho
Amarr GoonFleet
|
Posted - 2008.04.15 08:52:00 -
[38]
Originally by: Sionn Klorgh
So ultimately, your view is that its a CPU issue. A typical CPU gets so many node to handle. Through 500 people fleet battle on that node and it just runs out of steam.
- I have heard that putting more cpu onto a node helps (reinforcing a node). - Pity this can't be dynamic based on the number of people in a system. I believe 'reinforcing' can only be done at downtime.
Not quite right. One CPU can run many nodes but you can only have a maximum of one CPU per node due to how the code is designed at a low level (I believe the development work the devs are doing in Infiniband may actually allow them to overcome this obstacle).
Systems like Jita already have their own dedicated CPU and adding more hardware won't solve the problem until the above mentioned bottleneck can be resolved. In Jitas case I believe a dev said most of the lag comes from the market transactions and similar. Players docked and players just flying through space do not generate very much lag at all. On the combat side while Jita does get plenty of action the scale of the fights are relatively small.
Reinforcing a node is when a system is given its own dedicated node for whatever duration is required when it experiences sudden heavy activity. It does not mean a node gets more CPUs but rather that other systems that would have been running on that cpu are allocated elsewhere freeing it up to focus on a single system. This can only be done at down time due to technical limitations (again something Infiniband may be able to address). Eve pvp is actually more technically complex than it was in the early days with many more attributes and calculations to be performed (gang bonuses and so on) per player and this complexity starts to bog the system down sooner.
|

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 08:53:00 -
[39]
Edited by: LaVista Vista on 15/04/2008 08:54:25
Originally by: Wild Rho
Originally by: Sionn Klorgh
So ultimately, your view is that its a CPU issue. A typical CPU gets so many node to handle. Through 500 people fleet battle on that node and it just runs out of steam.
- I have heard that putting more cpu onto a node helps (reinforcing a node). - Pity this can't be dynamic based on the number of people in a system. I believe 'reinforcing' can only be done at downtime.
Not quite right. One CPU can run many nodes but you can only have a maximum of one CPU per node due to how the code is designed at a low level (I believe the development work the devs are doing in Infiniband may actually allow them to overcome this obstacle).
Systems like Jita already have their own dedicated CPU and adding more hardware won't solve the problem until the above mentioned bottleneck can be resolved. In Jitas case I believe a dev said most of the lag comes from the market transactions and similar. Players docked and players just flying through space do not generate very much lag at all. On the combat side while Jita does get plenty of action the scale of the fights are relatively small.
Reinforcing a node is when a system is given its own dedicated node for whatever duration is required when it experiences sudden heavy activity. It does not mean a node gets more CPUs but rather that other systems that would have been running on that cpu are allocated elsewhere freeing it up to focus on a single system. This can only be done at down time due to technical limitations (again something Infiniband may be able to address). Eve pvp is actually more technically complex than it was in the early days with many more attributes and calculations to be performed (gang bonuses and so on) per player and this complexity starts to bog the system down sooner.
This man is spot on.
EDIT: Im not sure about the CPU thing. Isn't a system limited to 1 core? Of course, 1 CPU can have more than 1 core.
|

Wild Rho
Amarr GoonFleet
|
Posted - 2008.04.15 08:56:00 -
[40]
Originally by: LaVista Vista EDIT: Im not sure about the CPU thing. Isn't a system limited to 1 core? Of course, 1 CPU can have more than 1 core.
You could be right, I just remember seeing devs talk about it in terms of CPU so that's what stuck, either way you get the basic idea.
|
|

Robdon
|
Posted - 2008.04.15 08:59:00 -
[41]
Edited by: Robdon on 15/04/2008 09:00:09
Originally by: LaVista Vista So while code performance COULD have a slight impact on the lag, it isn't the answer.
Yes, they absolutely need to get onto splitting up the services, BUT also from my experience, alot of performance can be gained from reviewing and rewriting code to perform better, and not all coders can do this.
|

T0desengel
|
Posted - 2008.04.15 11:19:00 -
[42]
I'm sure this has been mentioned on other threads of this nature but from doing some quick investigating and from converstaions people have had before, both in and out of game i'll put down what i know and have heard.
STACKLESS PYTHON: CCP Announcement of it use : http://www.tentonhammer.com/node/10044
After some trawling through the interweb i've found out that Stackless Python cannot easily use a mulit-processor/core enviroment, http://www.stackless.com/pipermail/stackless/2007-August/000326.html http://www.stackless.com/pipermail/stackless/2007-September/003069.html (click on NEXT MESSAGE to view the continuation of the thread)
NODE Balancing/Reinforcement: This thread is pretty much what i have heard. Dynamic Node (load) balancing is not easily implemented or possible. http://oldforums.eveonline.com/?a=topic&threadID=501320 I believe server load statistics are run from the previous day and this data is used to try to predict what the following days activities will be Humans being what we are are extremely random creatures so that prediction can often be wrong.
If we keep the responses constructive perhaps a DEV would like to comment.
|

Celeste Coeval
The Gosimer and Scarab
|
Posted - 2008.04.15 11:47:00 -
[43]
Originally by: DroneCommander People.
Originally by: Lance Fighter This is either a troll or a noob... Ill take the noob route.
|

Mordred Wolf
Minmatar Republic Military School
|
Posted - 2008.04.15 12:04:00 -
[44]
It is bad coding, bad structure. CCP probably use some ******** database that is not designed for high-speed gaming. You look at high fps games where there are 10x as many people involved than in EVE and it comes down to quality code and using the right tools for the right job.
An interesting discussion in the GDC was about databases and MMOs and how unsuitable they are, some developers started to develop their own high performance database that was specifically designed for MMOs and it had a significant impact in performance. I bet CCP uses Mysql or something *** like that.
|

Jenny Spitfire
Caldari LoneStar Industries Veritas Immortalis
|
Posted - 2008.04.15 12:06:00 -
[45]
I trie to make it easi for you to understood. Eve lag is cuz it is something like tis code down.
HAI I HAS A LOKALGANG LOL LOKALGANG R 400 I HAS A ME LOL ME R 0 I HAS A YUO LOL YUO R 0 IM IN YR LOOPOEN UPPIN YR ME TIL BOTHSAEM ME AN LOKALGANG IM IN YR LOOPTOO UPPIN YR YUO TIL BOTHSAEM YUO AN LOKALGANG DOESSUMTING ME AN YOU IM OUTTA YR LOOPTOO IM OUTTA YR LOOPOEN KTHXBYE
You knoe why is lag? --------- Technica impendi Caldari generis. Pax Caldaria!
Recruitment -KB- |

Benco97
Gallente Exchangable Properties
|
Posted - 2008.04.15 12:07:00 -
[46]
Me.
Dreadfully sorry about it Chaps. |

Armoured C
Gallente Globaltech Industries The ENTITY.
|
Posted - 2008.04.15 12:07:00 -
[47]
just wait for infinniband
this should speed up the processes of information sent between all items and this is going to be the most effective
im sure CCP are on top of this they know there own servers and make up more than you lot most probably do
yes it a pain but most of us live through it for many years now
just be patient |

Xue XinMao
West Wind Waylanders
|
Posted - 2008.04.15 12:10:00 -
[48]
Log is caused, by sigmarons, they are eutrons in the center of the xenon clusted within the CCP Xavier server, the Structural damage sustained By The Cloritons Discusscadas Is rather huge, They tried the Cheesada trapul, but never worked well, The Hamstaruses are Being Obtained as we speak, and soon will have a Lag free Xenophonic eve. |

Armoured C
Gallente Globaltech Industries The ENTITY.
|
Posted - 2008.04.15 12:17:00 -
[49]
it teh jovians teh rs stealing bandwidth for cure for alien faces
|

Silver Night
Caldari Naqam
|
Posted - 2008.04.15 12:17:00 -
[50]
Originally by: Mordred Wolf It is bad coding, bad structure. CCP probably use some ******** database that is not designed for high-speed gaming. You look at high fps games where there are 10x as many people involved than in EVE and it comes down to quality code and using the right tools for the right job.
(High FPS game with >5000 people on the screen at once? Interesting.)
An interesting discussion in the GDC was about databases and MMOs and how unsuitable they are, some developers started to develop their own high performance database that was specifically designed for MMOs and it had a significant impact in performance. I bet CCP uses Mysql or something *** like that.
(Yeah, I'm sure it must be MySQL. It isn't like what they actually use has been repeatedly mentioned in this thread and linked in dev blogs or anything.)
A reasonable post by an alt who has obviously done all of his homework.
CCP has pinned their hopes/hypes on Infiniband and a supercomputer-type architecture. I'm personally gonna wait until that kicks in (and the ensuing month of bugs are handled) before I start wearing my 'The End is Near' poster board on the street corner.
Fact of the matter is, there is probably a bit of optimization to be done in the code. Which is of course why every third phrase you hear out of a Dev mouth is 'Need for Speed'. You know, that initiative where they are, get this, optimizing their code. It is ongoing, because the server-side code for the game is ridiculously large. And it needs to be. they do in fact have people only doing this.
So, they are upgrading both their code and their hardware. What else would you like? Voodoo blessing of the server? Tech 2 hamsters? Sacrifice of a lamb to the database gods? More beer for the coding monkeys? --------------
GLS Mr. State Caldari Patriot. Sansha's Nation Supporter Murderer of (his own) Frigates.
|
|

Rawr Cristina
Caldari Naqam
|
Posted - 2008.04.15 12:21:00 -
[51]
Originally by: Jenny Spitfire I trie to make it easi for you to understood. Eve lag is cuz it is something like tis code down.
HAI I HAS A LOKALGANG LOL LOKALGANG R 400 I HAS A ME LOL ME R 0 I HAS A YUO LOL YUO R 0 IM IN YR LOOPOEN UPPIN YR ME TIL BOTHSAEM ME AN LOKALGANG IM IN YR LOOPTOO UPPIN YR YUO TIL BOTHSAEM YUO AN LOKALGANG DOESSUMTING ME AN YOU IM OUTTA YR LOOPTOO IM OUTTA YR LOOPOEN KTHXBYE
You knoe why is lag?
  
<3 Jenny  ...
|

LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.04.15 12:35:00 -
[52]
Originally by: Mordred Wolf It is bad coding, bad structure. CCP probably use some ******** database that is not designed for high-speed gaming. You look at high fps games where there are 10x as many people involved than in EVE and it comes down to quality code and using the right tools for the right job.
An interesting discussion in the GDC was about databases and MMOs and how unsuitable they are, some developers started to develop their own high performance database that was specifically designed for MMOs and it had a significant impact in performance. I bet CCP uses Mysql or something *** like that.
Here is the presentation.
Also, CCP uses MSSQL, NOT mysql. The game design is about a decade old. These high-performance databases weren't available.
But the majority of the lag, doesn't, as far as i understand, spawn from the database, since most of the data is stored on a RAMDisk.
|

Robdon
|
Posted - 2008.04.15 13:12:00 -
[53]
Have to agree, that I don't think its a DB problem.
The reason I think this is because if you put a char in Jita, and do a market inquiry, it can take 30 seconds, before a result is shown.
If I'm in another quiet system in Forge, I can do the same market inquiry and it only takes a second or 2.
Therefor, its my conclusion that the DB must be able to cope with the requests, its the CPU that is processing the requests and communicating with the client that has the problem / bottle neck.
So, installing RamDisks, speeding up the DB etc, will probably have no/little effect on the overall lag on large systems.
Of course though, this is all speculation :) Without knowing we are all just guessing.
We just went through this kind of reasoning in RL, we've always tried to keep our record size down on our data files, cause lets face it, less data to read should = faster reads.
However, our file systems record locking method was actually the thing slowing down the system, and therefor, the disks had plenty of capacity to read the data + more. It was the locking mechanism that was the slowest part of the loop. So we could have put the whole system on a RAMdisk and it would have made no difference to the speed of our system.
Performance is a tricky area, and needs lots of experience and investigation. There is no 'Win' button with performance. You need to see what slows your system down, not just say, oh lets add more CPU or oh lets use RAM Disks etc.
|

Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2008.04.15 13:14:00 -
[54]
Originally by: Jenny Spitfire I trie to make it easi for you to understood. Eve lag is cuz it is something like tis code down.
HAI I HAS A LOKALGANG LOL LOKALGANG R 400 I HAS A ME LOL ME R 0 I HAS A YUO LOL YUO R 0 IM IN YR LOOPOEN UPPIN YR ME TIL BOTHSAEM ME AN LOKALGANG IM IN YR LOOPTOO UPPIN YR YUO TIL BOTHSAEM YUO AN LOKALGANG DOESSUMTING ME AN YOU IM OUTTA YR LOOPTOO IM OUTTA YR LOOPOEN KTHXBYE
You knoe why is lag?
Wait... wait... is that a code RAGNAROK ?

1|2|3|4|5. |

Sionn Klorgh
Minmatar The Collective Against ALL Authorities
|
Posted - 2008.04.15 13:45:00 -
[55]
Thanks for all the replies!
Quote: CCP is implementing infiniband to the cluster this year, in order to make it possible, and thus killing the lag-monster.
Can someone explain to me how this works? Also are there any blog posts by the DEVs about this? Finally, do you think this will work?
|

Armoured C
Gallente Globaltech Industries The ENTITY.
|
Posted - 2008.04.15 13:46:00 -
[56]
just like minnie player you stuck random stuff to teh servers and walla a ship err i mean a server
|

Lakut
Arcane Alliance
|
Posted - 2008.04.15 14:01:00 -
[57]
Edited by: Lakut on 15/04/2008 14:02:24 Skipped most of the thread, because smartasses know crap about computing but here's my 0.02 ISK as a computer science student.
- Time jump back to the early 21st century We are in a time where processors are still single core and 1GHz processors are the new ****. Some unknown company in a country nobody cares about starts the coding on a shiny new MMO. They don't write the code massively multithreaded, because that's a pandora's box to do properly (seriously, it is since you have to keep data consistent among all threads) and the benefit is still small and processors are getting faster and faster anyway, so we can just throw bigger server hardware at it anyway as the playerbase grows and optimize the code a bit. And multithreaded software performs worse on single CPU machines anyway, because of overhead.
- Time jump back to the present The GHz race in the processors department came to an abrupt halt after the Netburst debacle. The new hype now is to throw more cores on the die. The MMO by the formerly small company in a country still nobody cares about has grown a lot and has problems handling the load. Now it seems that the design decision from a few years ago bites them in the bumhole. People decide that they want to fight in fleets of at least 9000 pilots with everyone and their mom having drones, tons of fighters and everyone in a different ship with different stats and fittings making everyone unique doing stuff like shooting their weapons and activating modules.
And now they can't just blindly throw more computing power at any one node because a single thread still runs crap on 16 individually slow CPUs and the node management software from back then can't dynamically assign processing power to the node that needs it and the node server can't utilize more than a fixed number of processing units (be it cores or physical CPUs) anyway, see Jita. What is needed now is tons of code optimization and a massive rewrite for the node management layer as well as the node software itself. And this takes time, manpower and lots of smart people doing smart programming.
This is also where infiniband comes in, as a connection between the individual servers in the cluster. When more than one box has to handle one node together, ethernet isn't a bottleneck in the bandwidth department, but the latency is terrible, which is actually no problem for some traditional HPC applications where you're not in a time critical environment. Infiniband fixes the latency problem but won't help independently of the software redesign (that's for those whiners crying "INFINIBAND NOW" every now and then).
edit: Typo ----------
You get a wonderful view from the point of no return. |

Tammarr
|
Posted - 2008.04.15 14:15:00 -
[58]
Jenny Marry me 
Yes, the system should benefit from once been built to work in a better server oriented enviroment, now we can't have that, the overhaul would cost to much and not tackle the real issue which will be done with the infiniband structure. Hopefully.
Now back to Jenny, where do you live doll?
|

Lrrp
Minmatar Gallente Mercantile Exchange Tenth Legion
|
Posted - 2008.04.15 14:21:00 -
[59]
So we are saying if a 300 pilot fleet fight where everyone was in a Rifter with the exact same fittings and pilots had no implants, would be less laggy than same size battle where the typical fleet compositions are used?
|

Lakut
Arcane Alliance
|
Posted - 2008.04.15 14:25:00 -
[60]
Edited by: Lakut on 15/04/2008 14:25:01
Originally by: Lrrp So we are saying if a 300 pilot fleet fight where everyone was in a Rifter with the exact same fittings and pilots had no implants, would be less laggy than same size battle where the typical fleet compositions are used?
We should try someday.
I'm pretty confident that drone/fighters and their "AI" ( ) cause at least a reasonable part of the lag in big fleet engagements. After all, you suddenly have roughly 3-5 times the entitys on grid each with his own and individual stats. ----------
You get a wonderful view from the point of no return. |
|

Commander Criton
Gallente K.I.S.A.
|
Posted - 2008.04.15 14:33:00 -
[61]
In an ideal world everything would work fine, there would be no killing, no diseases, no poor and world peace. The fact is eve's code is not perfect I mean come on last i heard the UI was all produced using direct 3d which is a system hog right there But im sure CCP know the problems but are trying to fix them with the resources they have without having to shut the game down for an over haul, they have been applying patch's here and there to improve performance lets hope they keep that up and maybe one day the lag will be in the past.
|

Ralara
Caldari D00M. Triumvirate.
|
Posted - 2008.04.15 14:59:00 -
[62]
Originally by: Robdon Bad Coding.
Jugging by the amount of bugs and CCPs inability to fix them in any kind of quick manner, I would say that the state of the code must be appalling.
I've worked with computing for many many years, and most coders donĘt have a clue about performance, or even consider it.
You have to have a couple of guys who dedicate themselves to this area and review / analyse ALL code that gets into the system.
ItĘs a completely different thought process, that CCP just doesnĘt seem to have.
Now obviously I havenĘt seen the code, but things like this....
Fleet battles, the first thing you do is lock a target and fire all your weapons.
This involves clicking on maybe 6-7 (13 or more for ECM boats) modules to activate all your weapons, and thatĘs 7 transactions to the server that need to be processed.
There should be an 'activate all' button, that as a user you can setup and decide what modules get activated with this button, then if this is pressed 1 transaction is sent to the server, instead of 7.
This is just one example that I can see 'should' help.
It might sound harsh, but since trinity and the 'Need for speed', this game has just got worse. So what ever it is their coders are doing itĘs not what is needed. They need to get fresh dedicated coders who know what they are doing in this area, because CLEARY the current ones donĘt know what to do.
that wouldn't work - it's not the bandwidth of sending commands, it's the processing of each one. When you click on your 425mm Railgun II, what do you think happens?
the CPU needs to calculate:
Your distance to your target The Ammo used (distance / damage / damage type multipliers) Transversal Velocity Angular Velocity Speed of ships Your skills - trajectory analysis level, sharp shooter level etc your weapon mods your skills that affect your weapon mods Your fall off ..
I could go on - then it needs to work out how much that single railgun does to the enemy target.
And you have 8 of them active.
And there's 20 or 30 of you in gang. Or 200.
And then the same for the enemy.
And there's 20 or 30 of them.
Or 200.
Once it's done that, it needs to look at your target, work out their shields, their armour, their resistances, their skills, their mods that affect the shields and armour, the skills that affect the mods that affect the shields and the armour...
And the drones
Ohhh the drones.
and the fighters.
:-)
So you can see, it's not just a matter of clicking a button...
And it needs to update the nodes and spread this info across the entire grid for everyone to get the information all at once, to keep everyone in line.
And it has to do it for, say, 2000 populated solar systems.
And manage all the NPCs in missions.
And the mining.
And the manufacturing.
And the market in that region/system.
Mmmm... when you look at it like that, it's impressive how the sever doesn't just fall over out of its rack. -- Ralara / Ralarina
VOTE FOR RALARA AS YOUR CSM REPRESENTATIVE, AND RECEIVE A COUPON FOR A 10% DISCOUNT AT WORLD OF WARCRAFT.
http://oldforums.eveonline.com/?a=topic&threadID=74811 |

Freya Selene
Delucian Defence Initiative The Church.
|
Posted - 2008.04.15 15:02:00 -
[63]
As a computer science student you should know that changes or constandly implemented and is an on going process of daily operations.
Comparing the Eve clients ability from 2003 vs hardware of 2008 and arround is more something for an historical point of view.
Have you compared the premium client also with 2003 hardware?
|

Alz Shado
Ever Flow
|
Posted - 2008.04.15 15:15:00 -
[64]
Why does the topic remind me of an Onion "Statshot?"
24% - AFK Miners 1% - Mission Runners 41% - Gate campers 7% - Jenny Spitfire 18% - Some war 42 jumps away 9% - .01 ISK Leapfroggers
//// ---------=== []= ---------=== \\\\ Rifter(RedBad)
"Kill a man one is a murderer; kill a million, a conqueror; kill them all, a God." -- Jean Rostand |

Savant Hayt
Minmatar The Reich
|
Posted - 2008.04.15 15:28:00 -
[65]
Critters knawning on teh pipe.
CCP game wardens are attending to the situation.
Thank you for your understanding
|

Lakut
Arcane Alliance
|
Posted - 2008.04.15 15:36:00 -
[66]
Edited by: Lakut on 15/04/2008 15:36:38 Edited by: Lakut on 15/04/2008 15:36:11 I assume you meant me.
Originally by: Freya Selene As a computer science student you should know that changes or constandly implemented and is an on going process of daily operations.
Comparing the Eve clients ability from 2003 vs hardware of 2008 and arround is more something for an historical point of view.
Have you compared the premium client also with 2003 hardware?
Who the hell talked about the client? And anyway, adding more shinyness to a graphics engine (your client frankly isn't much more with all the logic being serverside) isn't a major rewrite of a complete and complex server software.
And sure, the server software changed over time, implementing new features, but the major design decisions (following the programming paradigms of the time) that still burden the server software weren't overcome and adding more entropy to the code as well is not helping. That it was slightly changed over years doesn't add maintainability. On the contrary, software approaching Entropy Death is something most software engineers fear. ----------
You get a wonderful view from the point of no return. |

Commander 598
GoonFleet GoonSwarm
|
Posted - 2008.04.15 16:10:00 -
[67]
I heard EVE sends packets in the 8MB range.
|

Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2008.04.15 16:11:00 -
[68]
Originally by: Commander 598 I heard EVE sends packets in the 8MB range.
Your hand slipped from the K key a bit down to the M key a little bit there, buddy...

1|2|3|4|5. |

Dr Slaughter
Rabies Inc.
|
Posted - 2008.04.15 16:29:00 -
[69]
Originally by: Sionn Klorgh
WHAT EXACTLY IS CAUSING LAG?
The underpants.
Originally by: Sionn Klorgh
DOES CCP UNDERSTAND THE CAUSE EXACTLY?
Yes.
Originally by: Sionn Klorgh
IS THERE SOME FUNDAMENTAL LIMITATION IN TECHNOLOGY THAT WE ARE REACHING IN THIS GAME PREVENTING A FIX?
No. We have the technology although the implementation is hard and potentially very expensive. There are many threads on this, such as:
Linkage Linkage
Typically search for the words 'infiniband' etc.
Originally by: Sionn Klorgh
I appreciate everyone's opinion on the matter.
 dealing with the UNDERPANTS of eve since 2004 |

TRYPTIC
Aliastra
|
Posted - 2008.04.15 17:15:00 -
[70]
I tried more than 2 years ago to explain what I thought created the lag, and all I got was a bunch of "you don't know what you're talking about" type responses.
This is the way it was explained to me by some network wonks:
The lag is due to 3 things - hardware, number of clients accessing the server and their distance(in jumps), and efficiency of the code being run.
Two of those three things can be controlled by those who run and maintain the server - get the best hardware money can buy and optimize the code.
The third, jump distance and number of clients accessing the server cannot be controlled. Hence there will ALWAYS be lag...no matter how many people will tell you otherwise. A guy playing in Hong Kong is not gonna have the same game experience as a guy playing in London. Mr Hong Kong is going to get server data quite a bit later than Mr London. It's the physics.
|
|

Aprudena Gist
Caldari GoonFleet GoonSwarm
|
Posted - 2008.04.15 17:20:00 -
[71]
horrible netcode is causing the lag.
|

Dr Slaughter
Rabies Inc.
|
Posted - 2008.04.21 23:18:00 -
[72]
Originally by: Aprudena Gist horrible netcode is causing the lag.
Evidently some people have made big strides in understanding the client recently. Perhaps you could back-up your statement with a hypothetical example of how not to implement a space mmo client... dealing with the UNDERPANTS of eve since 2004 |

Chainsaw Plankton
IDLE GUNS
|
Posted - 2008.04.21 23:54:00 -
[73]
You see the internet is a series of tubes, oveur decided to use these tubes and install a distillery as the price of alcohol is ridiculous in Iceland. they planed ahead and saw that they could use a "fanfest" to smuggle in booze from the UK/US. The game hasnt quite reached a critical point at which the fanfest smugglers can supply ccp with a years worth of booze. hopefully at the next fanfest there will be enough imported booze that CCP can disassemble their moonshine factory and dump the massive load that is eve through the tubes.
|

Forge Lag
|
Posted - 2008.04.22 01:12:00 -
[74]
Originally by: Jenny Spitfire I trie to make it easi for you to understood. Eve lag is cuz it is something like tis code down.
HAI I HAS A LOKALGANG LOL LOKALGANG R 400 I HAS A ME LOL ME R 0 I HAS A YUO LOL YUO R 0 IM IN YR LOOPOEN UPPIN YR ME TIL BOTHSAEM ME AN LOKALGANG IM IN YR LOOPTOO UPPIN YR YUO TIL BOTHSAEM YUO AN LOKALGANG DOESSUMTING ME AN YOU IM OUTTA YR LOOPTOO IM OUTTA YR LOOPOEN KTHXBYE
You knoe why is lag?
This.
No amount of good coding or the shiniest hardware will solve this. Jita is bad. Fleet battles are bad.
|

Belidonna
|
Posted - 2008.04.22 01:38:00 -
[75]
Lag has been a problem ever since the server broke about 10k users online at the same time. In 2005 there was a large server upgrade to solid state disks.
http://www.superssd.com/success/ccpgames.htm
"Performance analysis revealed that disk queues were 40 requests deep. Microsoft recommends upgrading hardware if disk queues exceed 3. Better hardware is needed if more users are accessing a server. "
"Typical disk access times are 2-5 milliseconds and what was needed was a faster disk access time of 20-50 microseconds. "
"EVE Online's underlying storage bottleneck is a classic problem with Online Transaction Processing. 10,000+ users accessing account information, warping across the galaxy, buying goods from black-market free-lance smugglers and upgrading their mining frigates to assault cruisers all at the same time puts immense stress on disk-based storage. The game makes 60 million process calls per day with around 1,250 transactions per second at peak hours."
This was years ago with peaks less than 20K users. We are double that now. The only way it will be fixed is a rewrite and hardware upgrade or sharding the universe so that limits the total number of users allowed to subscribe to a server implementation is finite. This will allow operation and fits within the performance benchmarks of the hardware. |

Tortun Nahme
Minmatar Umbra Synergy Final Retribution Alliance
|
Posted - 2008.04.22 01:58:00 -
[76]
lag is caused when dense people gather, the extra gravity causes time dialation (sp?) and everything slows down  |
|
|
|
Pages: 1 2 3 :: [one page] |