Pages: 1 2 3 4 5 6 7 8 [9] 10 11 12 13 14 15 16 17 .. 17 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 20 post(s) |
Siddy
Art of War Cruel Intentions
|
Posted - 2008.05.26 17:04:00 -
[241]
It will be futile.
If server can take 1000 in one grid, people will bring 1001!
|
Zeba
Pator Tech School
|
Posted - 2008.05.26 17:32:00 -
[242]
Originally by: Siddy It will be futile.
If server can take 1000 in one grid, people will bring 1001!
Hopefully with the proposed changes to how sov works the *need* for those 1001 ship mega blobs will go away. So far I think that if we can get a 500 ship fight to go smooth the bulk of alliance warfare will benfit.
inappropriate signature. ~WeatherMan |
IonKnight
|
Posted - 2008.05.26 17:40:00 -
[243]
Btw can anyone if this infiniband is clustering as quite interested in that subject. Hehe if i go into to jita and i get no lag i will faint :D. Btw where is the circle of death??
|
Siddy
Art of War Cruel Intentions
|
Posted - 2008.05.26 17:42:00 -
[244]
Edited by: Siddy on 26/05/2008 17:42:18
Originally by: Zeba
Originally by: Siddy It will be futile.
If server can take 1000 in one grid, people will bring 1001!
Hopefully with the proposed changes to how sov works the *need* for those 1001 ship mega blobs will go away. So far I think that if we can get a 500 ship fight to go smooth the bulk of alliance warfare will benfit.
Some allainces have capacity to strike 3 difirent systems with 400+ ship at once, yet they still chose to stuff everyone and they dog in same system. Not only that, but bring people in noobships and shuttles if one lacks combat ship.
Do you honestly think that a mere sov change can turn a ugly-******et-dukling of MMO world into an Sun Zu strategist?
You think to higly abaut the human race.
|
Termino Fenix
|
Posted - 2008.05.26 19:50:00 -
[245]
Originally by: Dr Slaughter
... Step 1. break up the services so they can all run on separate CPU cores, Step 2. re-write the services that sit at 99% CPU so that they can be distributed across multiple CPU's, Step 3. Allow services to span hosts using IB to provide shared memory/or similar HPC service between the distributed processes. ...
Step 1 is commonly referred to CPU affiliation and as complex as EVE is, it's no doubt a daunting task but it will pay off in spades.
Moving to InfiniBand will be fantastical, the throughput is breathtaking. I'm kind of surprised it wasn't done earlier but I guess it all comes down to priorities.
/HPC admin
|
Dr Slaughter
Rabies Inc.
|
Posted - 2008.05.26 22:27:00 -
[246]
Edited by: Dr Slaughter on 26/05/2008 22:32:27
Originally by: Termino Fenix
Originally by: Dr Slaughter
... Step 1. break up the services so they can all run on separate CPU cores, Step 2. re-write the services that sit at 99% CPU so that they can be distributed across multiple CPU's, Step 3. Allow services to span hosts using IB to provide shared memory/or similar HPC service between the distributed processes. ...
Step 1 is commonly referred to CPU affiliation and as complex as EVE is, it's no doubt a daunting task but it will pay off in spades.
Moving to InfiniBand will be fantastical, the throughput is breathtaking. I'm kind of surprised it wasn't done earlier but I guess it all comes down to priorities.
/HPC admin
From reading the CCP posts I had come to the conclusion that the SOL 'services' all sat on one CPU because of issues with the Python Interpreter's Global Interpreter Lock (GIL) not allowing multi-threaded design. This forced a monolithic design for the majority of the code. Despite Stackless allowing for Tasklets to squeeze every ounce (apologies to all you metric people) of power out of the CPU. Previously CCP had hoped the constant doubling of speed in CPU's would save them... then Intel and AMD went multi-core and I guess Python must have felt like not such a good choice.....
The first step is to split out SOL services so they can be run on different interpreters (I guess? Rather than re-write stackless's interpreter!) on the other cores and deal with passing state between them when needed. That shouldn't be as complex as step 2 and 3 in my naive list of what CCP should be doing, as the service code would largely remain the same except for the interprocess messaging needed for state exchanges (if any).
IonStorm, infiniband can be used in clustering but what it's really doing is giving you a 'local area system bus' across all the hosts that are IB connected. Essentially it's like everything is local to each CPU. Depending on the HPC implementation you can have very low level access (think shared memory) or high level access (think TCP/IP) at mind bendingly fast speeds. One downside is it's not cheap. I had to settle for FibreChannel to play with in my lab in the cellar.
CCP, any comments on HPCing the Stackless Python Interpreter? Is that what you're doing? Or are you going to have a Interpreter on each CPU core and extend it to support passing state via RDMA?
Rabies is unexpected ~~~~~~~~~~~~~~~~~~~~ dealing with the UNDERPANTS of eve since 2004 |
Fog Render
Mirage Industries
|
Posted - 2008.05.26 23:35:00 -
[247]
Can't they just use thicker wire?
|
|
CCP Lingorm
C C P
|
Posted - 2008.05.27 09:45:00 -
[248]
Originally by: Dr Slaughter
SOL services are currently sharing the same CPU that everything else is running on.
Step 1. break up the services so they can all run on separate CPU cores, Step 2. re-write the services that sit at 99% CPU so that they can be distributed across multiple CPU's, Step 3. Allow services to span hosts using IB to provide shared memory/or similar HPC service between the distributed processes.
CCP have been working on Step 1 for a while. Would be good to get an update. The complexity involved in (2)/(3) has been detailed by many posters and CCP themselves. Would also be great to hear how they're doing in the lab but I'm not holding my breath.
Forgot this thread even existed...
Update : Step 1 : Market and Chat have been separated for a while. We are working on pulling Station Services from the Solar System Service to give them their own node. Step 2 : The issue is that the Combat Engine will have to be single threaded (at this time), I outlined why way early in this thread, it has to do with getting killed by someone that is already dead, the idea of a combat processor and then a message sender process in lock step has been suggested to give the combat engine more cpu, I believe this was misquoted on the previous page. Step 3 : Planned for AFTER we have the infiniband and cluster stuff in and working ... so no change on this.
Not sure what else to update you on really. but I will try to remember this thread and stick more info in it.
As for the quote on page 9 from me in another thread. We would like to get python scaling over more than 1 cpu, and have some ideas on how to do it, but MPI and Infiniband are higher priority and are getting all the attention right now.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|
Tobias Solem
Quack Mongers
|
Posted - 2008.05.27 10:34:00 -
[249]
Do you have any kind of concrete timeframe on this? As in: months, years?
|
ElfeGER
Black Eclipse Corp Band of Brothers
|
Posted - 2008.05.27 17:06:00 -
[250]
how good is windows at running multiple nodes per cpu core? as in distribute like 50 nodes on 16 cores and let them balance in real time
I wonder if wine is able to run a solnode process as linux has several good task schedulers built in (ages ago the beta farcry dedicated windows server was running fine on wine)
|
|
|
CCP Lingorm
C C P
|
Posted - 2008.05.27 17:18:00 -
[251]
Originally by: El***ER how good is windows at running multiple nodes per cpu core? as in distribute like 50 nodes on 16 cores and let them balance in real time
I wonder if wine is able to run a solnode process as linux has several good task schedulers built in (ages ago the beta farcry dedicated windows server was running fine on wine)
Or Node process does that.
We run 1 process per CPU Core and that process can be dedicated to running a single Solar system, multiple systems, Chats, markets, gangs etc ... We do not run multiple instances of our node software on a single cpu on TQ ...
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|
Dr Slaughter
Rabies Inc.
|
Posted - 2008.05.27 22:45:00 -
[252]
Originally by: CCP Lingorm
Not sure what else to update you on really. but I will try to remember this thread and stick more info in it.
To be honest.. that was a great update. So in summary:
It's good to know you're actively working on pulling the station services out, that combat will still be single threaded (for now), and that the focus is on supporting basic MPI/Infiniband functionality before doing anything hideously clever with Pythons internals.
Also thanks for offering to keep an eye on this thread.
Assuming none of the changes are in testing yet on SiSi and the Infiniband hardware isn't ordered, are my expectations likely to be dashed if I'm thinking...
Progressively less monolithic sol system service from the fall, HPI/Infiniband deployment in the new year, GIL wrestling and cleverness sometime in the Spring?
Rabies is unexpected ~~~~~~~~~~~~~~~~~~~~ dealing with the UNDERPANTS of eve since 2004 |
|
CCP Lingorm
C C P
|
Posted - 2008.05.28 09:09:00 -
[253]
We actually have a test Infiniband cluster up and running for internal use.
Less monolithic Sol Service before the end of the year is a possibility, but no promises.
As to the rest I will talk to the Core guys adn see what they say.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|
LaVista Vista
Conservative Shenanigans Party
|
Posted - 2008.05.28 09:14:00 -
[254]
You should post some pictures of that cluster Lingorm. It sure sounds sexy!
|
Ethidium Bromide
ZEALOT WARRIORS AGAINST TERRORISTS Curatores Veritatis Alliance
|
Posted - 2008.05.28 09:48:00 -
[255]
Edited by: Ethidium Bromide on 28/05/2008 09:48:46 woot! i got no idea what all this means does this mean i need some sort of special connection to the interweb or is it ok if i use my old standard 4x DVD ROM to read both internet DVDs? cause last time i tried to watch pron faster i broke one of the interweb DVDs
Originally by: George Petsch Nochricht: Dei schwarer StroinlSser trifftn Karli[Baatzis] und ruiniert erm so richtig de Dosn, 1343.7 schhodn, oida.
|
RaTTuS
BIG
|
Posted - 2008.05.28 09:53:00 -
[256]
Originally by: Ethidium Bromide Edited by: Ethidium Bromide on 28/05/2008 09:48:46 woot! i got no idea what all this means does this mean i need some sort of special connection to the interweb or ...
no this is / will all happen in the server center - the infiniband stuff is for processing stuff on the servers - nothing to do with the clients -- BIG Lottery, BIG Deal, InEve
|
Ethidium Bromide
ZEALOT WARRIORS AGAINST TERRORISTS Curatores Veritatis Alliance
|
Posted - 2008.05.28 10:40:00 -
[257]
Originally by: RaTTuS
Originally by: Ethidium Bromide Edited by: Ethidium Bromide on 28/05/2008 09:48:46 woot! i got no idea what all this means does this mean i need some sort of special connection to the interweb or ...
no this is / will all happen in the server center - the infiniband stuff is for processing stuff on the servers - nothing to do with the clients
cool! thanks for the answer
Originally by: George Petsch Nochricht: Dei schwarer StroinlSser trifftn Karli[Baatzis] und ruiniert erm so richtig de Dosn, 1343.7 schhodn, oida.
|
Celeste Coeval
The Gosimer and Scarab
|
Posted - 2008.05.28 11:51:00 -
[258]
A mature thread on EvE-O nevah!
/me is thoroughly enjoying this. I would like to add, that this is the sort of thread CCP posts in, not the whining and generalizating that usually prowls these forums.
Sazumaan thankyou for a rational and well thought out topic.
Originally by: Lance Fighter This is either a troll or a noob... Ill take the noob route. |
Sazumaan Johnza
Southern Cross Incorporated Southern Cross Alliance
|
Posted - 2008.05.29 04:08:00 -
[259]
Originally by: Celeste Coeval
Sazumaan thankyou for a rational and well thought out topic.
I'm doing my best to bring brains together for CCP's benefit. Would love to be able to brag about the cluster being in the Top 500 utilising Infiniband. I guess it's the nerd in me ;) "Eve is more filling than roast steaks slowly grilling over a rotating fire whilst marinating in a combination of Australian fruity wines and the best imported herbs..." - SChimera [16.4.07] |
Ishina Fel
Synergy. Imperial Republic Of the North
|
Posted - 2008.05.29 10:02:00 -
[260]
Lingorm, I have a crazy idea.
I have no clue if it would even work in this format, but has CCP considered letting the community help? I mean, you have around 250,000 active subscribers with a very high geek level. At least a coule hundred of them must be able and willing to solve complex coding problems as a hobby.
What CCP could do:
- Specifically design small, stand-alone programming problems that mirror isolated issues that the EVE code architecture is facing. For example: "Write a load balancer that can, in real time, shift a process involving SQL queries to the CPU core with the lowest load. Use only Stackless Python version (blah)". Or, another example: "Here is an application that executes three processes in a stack. Unstack them for us." Disclaimer: I have only a very limited idea of what I'm talking about here
- Put these problems up in community competitions. Offer prizes for the first ten people who submit a working solution.
- Then, when your team does get around to tackling the project, they will already have access to a selection of existing solutions for different isolated parts of the big problem. Ideally, this will mean that they have a basis to start their work from, and will save more time than it took to draft up the problems for the competition. Maybe there will even be a submission from a mad hobby genius that is so fundamentally different that no normal software architect would come up with it, yet inspires the solution of multiple problems in one blow?
If this could be made to work without too much effort on CCP's side, it might more than just pay off in the end, and the community will have the feeling that they can actively help developing they game they love. It's win-win!
...but then again, it might only be a crazy, unworkable idea after all. Who knows?
|
|
Tobias Sjodin
Ore Mongers Black Hand.
|
Posted - 2008.05.29 10:06:00 -
[261]
I wonder if a peer-to-peer kind of architecture as torrents would work in a MASSIVE game. We have 40,000 clients connected and most likely not all of them use up the computing power. Having all connected clients help the server cluster with the load would be a pretty cool thing.
Black Hand.
|
Elphy
Noob Mercs
|
Posted - 2008.05.29 10:18:00 -
[262]
Originally by: Tobias Sjodin I wonder if a peer-to-peer kind of architecture as torrents would work in a MASSIVE game. We have 40,000 clients connected and most likely not all of them use up the computing power. Having all connected clients help the server cluster with the load would be a pretty cool thing.
P2P Processing is fine for those Seti@Home like things but for anything interactive like an MMO the latency would be measurable in ice ages.
|
|
CCP Lingorm
C C P
|
Posted - 2008.05.29 10:23:00 -
[263]
Originally by: Tobias Sjodin I wonder if a peer-to-peer kind of architecture as torrents would work in a MASSIVE game. We have 40,000 clients connected and most likely not all of them use up the computing power. Having all connected clients help the server cluster with the load would be a pretty cool thing.
And we could not 'trust' any results that where 'outsourced' we would have to pass the same problem to multiple computers and then compare results, this would mean tying up lots of extra time running checks ... not worth it. Sorry.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|
|
CCP Lingorm
C C P
|
Posted - 2008.05.29 10:24:00 -
[264]
Originally by: Ishina Fel Lingorm, I have a crazy idea.
I have no clue if it would even work in this format, but has CCP considered letting the community help? I mean, you have around 250,000 active subscribers with a very high geek level. At least a coule hundred of them must be able and willing to solve complex coding problems as a hobby.
What CCP could do:
- Specifically design small, stand-alone programming problems that mirror isolated issues that the EVE code architecture is facing. For example: "Write a load balancer that can, in real time, shift a process involving SQL queries to the CPU core with the lowest load. Use only Stackless Python version (blah)". Or, another example: "Here is an application that executes three processes in a stack. Unstack them for us." Disclaimer: I have only a very limited idea of what I'm talking about here
- Put these problems up in community competitions. Offer prizes for the first ten people who submit a working solution.
- Then, when your team does get around to tackling the project, they will already have access to a selection of existing solutions for different isolated parts of the big problem. Ideally, this will mean that they have a basis to start their work from, and will save more time than it took to draft up the problems for the competition. Maybe there will even be a submission from a mad hobby genius that is so fundamentally different that no normal software architect would come up with it, yet inspires the solution of multiple problems in one blow?
If this could be made to work without too much effort on CCP's side, it might more than just pay off in the end, and the community will have the feeling that they can actively help developing they game they love. It's win-win!
...but then again, it might only be a crazy, unworkable idea after all. Who knows?
Interesting .... very interesting.
I like the concept ... will chat with some people ... no promises but it could be fun to do.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
Originally by: Lord Fitz Eve is to WoW as Wow is to an 8 player game of Unreal Tournament.
|
|
masternerdguy
|
Posted - 2008.05.29 10:32:00 -
[265]
we need each system to run on a seperate sever.
|
Elphy
Noob Mercs
|
Posted - 2008.05.29 11:00:00 -
[266]
Originally by: masternerdguy we need each system to run on a seperate sever.
Not really as the more machines you introduce into a cluster the more points of failure there are.
Today you can get CPUs which have 4 cores with 8/16 core cpus coming in the near future you are able to run alot more on one machine that you used to with single core cpus.
Putting every system on a different physical machine would be wasteful. Considering that system services are being moved to run on different cores and for the sake of argument that eventually each system will have 8 services you could run 8 systems on a single quad cpus 16 core box (4 * 16 / 8). These would be in the same physical server and clustered with other servers providing the over all EVE universe. Now say rather than max out processor use on each box you only ran 6 systems per machine. This will give a spare CPU to load balance a system on that server if a large battle or indeed any form of large population are there.
Having a physical server for every system means more motherboards to go wrong, more memory to go wrong etc... etc....
Anyone will stop before I make even less sense.
|
ElfeGER
Black Eclipse Corp Band of Brothers
|
Posted - 2008.05.29 16:40:00 -
[267]
the issue is that there is running one process per cpu and probably bound to the cpu as well
by in creasing the processes to like 4x cores you could get have finer balancing as the os decides in realtime how the processes are distributed so you could have running 1 process on one core in a big battle/busy system while the other core runs the other 7 processes and is still not maxed out (that's where multi processor 4x cores come in handy)
on the other side that would increase the memory, db connection usage as well as inter node jumps
|
ElfeGER
Black Eclipse Corp Band of Brothers
|
Posted - 2008.05.29 16:43:00 -
[268]
Originally by: Ishina Fel For example: "Write a load balancer that can, in real time, shift a process involving SQL queries to the CPU core with the lowest load. Use only Stackless Python version (blah)"
hmmm why reinvent the wheel? the os handles that automaticly
|
Una D
Ex Coelis The Bantam Menace
|
Posted - 2008.05.29 17:00:00 -
[269]
What EVE needs is graceful deterioration of performance. So instead of huge fleet fight lagging out to hell and back just increase the time length (so one second in game takes 5 seconds in RL for example). The servers would be able to handle the load (and if you overload you can just stretch the time). Since the thread is fully serial it's not even necessary to make sure that client commands are scheduled correctly in time (first come first serve). This would make the game in to a slide show but that must be better than lag to death/node death.
Plus the lag affects everyone the same (including server controlled entities like drones).
|
Venchoba
The Arrow Project Morsus Mihi
|
Posted - 2008.05.29 17:33:00 -
[270]
What if you distribute computation on the same grid on several threads. One thread for area effects (movement, smartbomb, ECM burst, interdictor bubbles) and N threads with directed/local interaction (guns, missles and anything for which you need lock). Also, since this new cluster needs server code to be rewriten and one of big problems is I/O bandwidth, did CCP ever consider moving to mainframe? Mainframes are kings of I/O. Those things can process transactions of the credit cards in time of Christmas.
|
|
|
|
|
Pages: 1 2 3 4 5 6 7 8 [9] 10 11 12 13 14 15 16 17 .. 17 :: one page |
First page | Previous page | Next page | Last page |