| Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 .. 17 :: one page |
| Author |
Thread Statistics | Show CCP posts - 20 post(s) |
Sazumaan Johnza
Minmatar
Southern Cross Incorporated
Southern Cross Alliance
   |
Posted - 2008.01.14 03:19:00 -
[1]
Dear CCP,
Can you give us an update on the InfiniBand cluster upgrade project?
Is this still on track for Feb?
Many thanks,
Sazu
"Eve is more filling than roast steaks slowly grilling over a rotating fire whilst marinating in a combination of Australian fruity wines and the best imported herbs..." - SChimera [16.4.07] |
Brea Lafail
|
Posted - 2008.01.14 04:06:00 -
[2]
Care to explain this jargon for those of us who have to call tech support to plug in a toaster?
|
Siigari Kitawa
Gallente
The Aduro Protocol
The Fifth Race
|
Posted - 2008.01.14 04:10:00 -
[3]
Edited by: Siigari Kitawa on 14/01/2008 04:12:16
Bandwidth speeds in layman's terms:
Cable: o
InfiniBand: ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
_________________
Burn. |
Casino Alkasar
Caldari
|
Posted - 2008.01.14 04:22:00 -
[4]
Edited by: Casino Alkasar on 14/01/2008 04:22:06
its about killing the lag monster that follows the blobs
.. i think ^^
_________________
itze mine |
Xaziar Nortocus
Knights of the Burger King
YTMND.
|
Posted - 2008.01.14 05:04:00 -
[5]
 Originally by: Brea Lafail
Care to explain this jargon for those of us who have to call tech support to plug in a toaster?
It is a technology that connects servers with their RAM-drives, the difference is that InfiniBand does it MUCH faster. To best understand it, you would need a bit of insight into Information Technology or Telecommunications (Or BOTH).
Originally by: Ursula LeGuinn
In other news, water still wet. Film at 11.
|
Zeba
Minmatar
Pator Tech School
|
Posted - 2008.01.14 05:06:00 -
[6]
Originally by: Casino Alkasar
its about killing the lag monster that follows the blobs
.. i think ^^
So the theory goes. 
Originally by: Richard Phallus
 Come live with the common idiot, it's more fun down here.
I did and got banned. |
Tek'a Rain
Gallente
Collegium Mechanicae
|
Posted - 2008.01.14 06:42:00 -
[7]
Edited by: Tek''a Rain on 14/01/2008 06:42:38
think of it like.. replacing a bucket brigade with a set of powerful firehoses. your getting the same thing (water/data) to the same places (the fire/the processors), but faster.
probably a terrible explanation, but its how it was written out to me.
It should help because one of the lag making bottlenecks seems to be getting the information from the (Fast) processors to the (Fast) Ramsan storage drives, and back and forth and back and forth and on and on, which is required for just about most things to happen. scratch that, probably required for Everything.
This should, I think, also effect getting the same important data out to clients, passing from the "internal" nodes that handle the actual working to the "outer" parts of the server as a whole who is handling the connections to the interwebs and everyone else.
computer science. its like magic, with fewer dragons.
|
Tarminic
Forsaken Resistance
The Last Stand
|
Posted - 2008.01.14 06:46:00 -
[8]
Originally by: Tek'a Rain
Edited by: Tek''a Rain on 14/01/2008 06:42:38
think of it like.. replacing a bucket brigade with a set of powerful firehoses. your getting the same thing (water/data) to the same places (the fire/the processors), but faster.
probably a terrible explanation, but its how it was written out to me.
That works pretty well actually.
Quote:
It should help because one of the lag making bottlenecks seems to be getting the information from the (Fast) processors to the (Fast) Ramsan storage drives, and back and forth and back and forth and on and on, which is required for just about most things to happen. scratch that, probably required for Everything.
This should, I think, also effect getting the same important data out to clients, passing from the "internal" nodes that handle the actual working to the "outer" parts of the server as a whole who is handling the connections to the interwebs and everyone else.
computer science. its like magic, with fewer dragons.
Though I'm way too tired to go into all the details, that is actually a decent explanation.
----------------
Tarminic - 31 Million SP in Forum Warfare
Play EVE: Downtime Madness v0.78.2 |
Andrue
Amarr
|
Posted - 2008.01.14 08:10:00 -
[9]
Originally by: Casino Alkasar
Edited by: Casino Alkasar on 14/01/2008 04:22:06
its about killing the lag monster that follows the blobs
.. i think ^^
That'll never happen. The alliances will just make the blobs bigger. It's the same problem as Jita only the objective is combat instead of trade.
--
(Battle hardened industrialist)
[Brackley, UK]
My budgie can say "ploppy bottom". You have been warned. |
Amarria Black
Clan Anthraxx
|
Posted - 2008.01.14 08:34:00 -
[10]
Originally by: Andrue
Originally by: Casino Alkasar
Edited by: Casino Alkasar on 14/01/2008 04:22:06
its about killing the lag monster that follows the blobs
.. i think ^^
That'll never happen. The alliances will just make the blobs bigger. It's the same problem as Jita only the objective is combat instead of trade.
Given, so long as the game mechanics remain such that n+1 ships beats n ships 9 times out of 10.
However, this will still move the upper threshold for "game completely unplayable" to a higher level, and hopefully a level higher than you'll see outside of two megalliances blobbing the hell out of each other. Imagine minimal lag in Jita and none in the Triangle of Death. That can only be a Good Thing, ja?
Originally by: Frug
Your reputation has been entirely redeemed in my eyes. I now want your babies.
|
Mashie Saldana
Minmatar
Hooligans Of War
|
Posted - 2008.01.14 13:26:00 -
[11]
The main point of Infiniband isn't for RAMSAN connectivity but for node to node connectivity, I have a feeling that one already is connected to the database servers using fibre channel.
The main reason to use Infiniband isn't the bandwith but the very low latency compared to Ethernet. Due to that RDMA (Remote Direct Memory Access) is possible, as in node A can read directly from node B's RAM about the status of say a ship. This is essential to be able to break the one node per Sol limitation. Now however just ripping out all Ethernet cards and plugging in Infiniband cards instead won't do much good until more or less all code is rewritten to use the new abilities.
So as SoonÖ as the coding is done I'm sure CCP will deploy it.
|
Grimpak
Gallente
Trinity Nova
|
Posted - 2008.01.14 13:33:00 -
[12]
Originally by: Siigari Kitawa
Edited by: Siigari Kitawa on 14/01/2008 04:12:16
Bandwidth speeds in layman's terms:
Cable: o
InfiniBand: ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
this is the best explanation.
another layman's explanation should be smth like
cable: 100mbit
infiniband: OVER NINE! THOUSAND!!!!!!!!!!!!!!!!11111oneoneoneeleventyone
---
planetary interaction idea! |
Ishina Fel
Caldari
Synergy.
Imperial Republic Of the North
|
Posted - 2008.01.14 13:40:00 -
[13]
Indeed, this upgrade is no small feat - it's a hardware restructuring of a supercomputer spanning a larger room, which cannot be switched off just like that, even partially - just imagine the player reactions if CCP was switching nodes on and off at random just to experiment with their new functionalities. Luckily they have test servers for that, but when Tranquility is upgraded like that, an extended downtime of half a day is probably the least we can expect, and it will be the grand finale of a project spanning months and months of careful planning and preparation.
Imagine you bought a new piece of hardware, like a mainboard, and want to plug it into your system. But you only have a 5 minute window every day where you are allowed to turn the machine off - and you know full well that you have to reinstall your OS and your software as well due to the hardware change. Furthermore, to support that hardware, you need a new version of your OS that you have never worked with before!
That's the kind of project CCP is looking at here. There's pretty much no "just do it" possible for this. And I would honestly prefer if they took all the time they needed rather than ending up with a Tranquility cluster that won't start up or stay stable.
Mind, a devblog about the current plan would be schweet
|
Grimm Myn
Caldari
School of Applied Knowledge
|
Posted - 2008.01.14 13:51:00 -
[14]
Have a look here: http://en.wikipedia.org/wiki/List_of_device_bandwidths
Remember those old Parallel Printer cables?
They transferred smarties at 0.133 MB/s
The faster variant moved smarties at 2 MB/s
USB 2.0 does 60 MB/s
"Normal Network cables" run at 12.5 MB/s (100 mbit)
"Fast Network cables" run at 125 MB/s (1gbit)
Infiniband is nber at up to 12 GB/s (holy sh!t)
(MB/s = MEGABYTE per second)
Then again, 100 gigabit ethernet runs at 12.5 GB/s but thats something else, have fun paying for those network cards. Its under development anyway.
And here the explanation about Infiniband: http://en.wikipedia.org/wiki/Infiniband
"The InfiniBand architecture specification defines a connection between processor nodes and high performance I/O nodes such as storage devices."
Nice thing about Infiniband is QoS (Quality of Service... "Boom effects" > "Services in stations") and Failover. With ethernet its possible but isnt automatically build in.
|
Xaziar Nortocus
Knights of the Burger King
YTMND.
|
Posted - 2008.01.14 14:41:00 -
[15]
Originally by: Mashie Saldana
The main reason to use Infiniband isn't the bandwith but the very low latency compared to Ethernet. Due to that RDMA (Remote Direct Memory Access) is possible, as in node A can read directly from node B's RAM about the status of say a ship. This is essential to be able to break the one node per Sol limitation. Now however just ripping out all Ethernet cards and plugging in Infiniband cards instead won't do much good until more or less all code is rewritten to use the new abilities.
That may be true...but think about this:
Give Ethernet can be connected to any machine via a driver, the only thing that would be needed for the InfiniBand cards are the drivers. Granted they will need tweaking (translation: a LOT of tweaking), it should be no different than us at home adding a new device, provided the server OS is of the Windows/Linux variety (I would guess the latter)
Originally by: Ursula LeGuinn
In other news, water still wet. Film at 11.
|
|
CCP Lingorm
C C P
|
Posted - 2008.01.14 15:28:00 -
[16]
Originally by: Xaziar Nortocus
That may be true...but think about this:
Give Ethernet can be connected to any machine via a driver, the only thing that would be needed for the InfiniBand cards are the drivers. Granted they will need tweaking (translation: a LOT of tweaking), it should be no different than us at home adding a new device, provided the server OS is of the Windows/Linux variety (I would guess the latter)
Actually it's a little more than that.
Infiniband is more like liking your computers together is the 'PCI' bus rather than via a network connection.
You could plug them in and then point all your network coms at the new cards, but it would not do much. The advantage to be gained from Infinband is that it will allow computers to read from other computers, to push work (threads) out to other machines and get the responses back very very quickly. The structure of a Network card is very different in that using it via TCP mean you need to use the TCP stack architecture and this means dealing with the inherent delays with the processing.
The project team doing the infniband implementation as part of the HPC implementation are working very hard to get this out to you as soon as Practicable. But this is a Major recode of large parts of our low level code. We are basically ripping apart our own custom built Cluster management software and replacing it with calls to the HPC system. We are then recoding the software services to make use of the new communications framework allowing rapid transfer of data between Sol nodes and the splitting off of any part of the code that can be threaded.
This is not something we can "Just do it" too. We are working hard on this and will get it out when it is ready.
CCP Lingorm
CCP Quality Assurance
QA Engineering Team Leader
|
|
Amarria Black
Clan Anthraxx
|
Posted - 2008.01.14 15:34:00 -
[17]
Originally by: CCP Lingorm
...
I know this makes me a turbonerd, but the thought of the described final product makes me happy in the pants.
I can't wait until you guys can implement this on live. Keep coding + testing, and I'll try to be patient.
Originally by: Frug
Your reputation has been entirely redeemed in my eyes. I now want your babies.
|
Lag Hon
Minmatar
Enar Nine
|
Posted - 2008.01.14 15:34:00 -
[18]
So does this mean the servers can propogate a database error and fallover even faster?
Quote:
There is no greater power in the universe than the need for freedom. Against that power governments, and tyrants, and armies can not stand.
G'Kar
|
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2008.01.14 17:14:00 -
[19]
Edited by: Akita T on 14/01/2008 17:14:38
Originally by: CCP Lingorm
This is not something we can "Just do it" too.
We are working hard on this and will get it out when it is ready.
On the other hand, feeding us bits and pices of "we are making progress, we are about this much done" every now and then (instead of the usual total silence) would keep "making turbonerds happy in their pants", like Amarria Black put it 
1|2|3|4|5. |
Tarminic
Forsaken Resistance
The Last Stand
|
Posted - 2008.01.14 17:22:00 -
[20]
Originally by: CCP Lingorm
You could plug them in and then point all your network coms at the new cards, but it would not do much. The advantage to be gained from Infinband is that it will allow computers to read from other computers, to push work (threads) out to other machines and get the responses back very very quickly. The structure of a Network card is very different in that using it via TCP mean you need to use the TCP stack architecture and this means dealing with the inherent delays with the processing.
Inherent delays with BUS/Card/Network aside, don't forget that the TCP/IP protocol stack has an entire system of error-checking that must be used to verify every incoming packet. Technically the EVE server could use UDP to avoid the error-checking and speed things up significantly, but you could run into major problems if computers tried to send data farther than their immediate neighbor, since UDP packets can and do arrive out order.
----------------
Tarminic - 31 Million SP in Forum Warfare
Play EVE: Downtime Madness v0.78.2 |
Sazumaan Johnza
Minmatar
Southern Cross Incorporated
Southern Cross Alliance
|
Posted - 2008.01.14 20:41:00 -
[21]
Thank you very much for the update CCP Lingorm!!! :)
According to a couple of sites I have been reading, InfiniBand has become the "natural choice" for supercomputer clusters because it's standardized. A high number of CPU cores per server node platform requires high bandwidth and and low latency. InfiniBand does that really well and is now the preferred interconnect solution for over half of the world's Top 100 Supercomputers...
From an article posted on the Mellanox Technologies website:
"The use of off-the-shelf commodity and standard components has made its mark on the interconnect solutions, making Gigabit Ethernet and InfiniBand the dominant solutions. In the early days, when single-core CPUs were the common solution, Gigabit Ethernet was used mainly for the cases where there was no need for intensive I/O, and InfiniBand for high-performance computing or I/O intensive enterprise applications."
"Since the mid 2000s, multi-core processors have taken over the single-core CPUs due to the ever-increasing need for performance and increasing demand for low power solutions. This new trend pushed the need for fatter standard interconnects that can handle the increasing I/O demands, as more CPUs share the same connection - InfiniBand is the only industry-standard interconnect that provides the required bandwidth, latency, power and utilization characteristics."
"InfiniBand has become the interconnect of choice for high-performance applications and has started to penetrate into the enterprise datacenter as well. With its superior price/performance and power/performance, InfiniBand is expected to continue those trends in the foreseeable future."
Looking forward to sexy InfiniEVE!! 
"Eve is more filling than roast steaks slowly grilling over a rotating fire whilst marinating in a combination of Australian fruity wines and the best imported herbs..." - SChimera [16.4.07] |
Ishina Fel
Caldari
Synergy.
Imperial Republic Of the North
|
Posted - 2008.01.14 20:53:00 -
[22]
Let's hope we'll see Tranquility in the Top 500 soon!
|
Alekzander
Caldari
Shady Buisness
|
Posted - 2008.01.14 20:55:00 -
[23]
Quote:
I know this makes me a turbonerd, but the thought of the described final product makes me happy in the pants.
I can't wait until you guys can implement this on live. Keep coding + testing, and I'll try to be patient.
These aren't your dad's nerds, these are TURBONERDS, more NERDS than a server farm has ROOM FOR! lawl, i couldnt help my self... just in case anyone doesnt get the referance... youtube, and find power thirst.
|
Tarminic
Forsaken Resistance
The Last Stand
|
Posted - 2008.01.14 21:05:00 -
[24]
Originally by: Alekzander
Quote:
I know this makes me a turbonerd, but the thought of the described final product makes me happy in the pants.
I can't wait until you guys can implement this on live. Keep coding + testing, and I'll try to be patient.
These aren't your dad's nerds, these are TURBONERDS, more NERDS than a server farm has ROOM FOR! lawl, i couldnt help my self... just in case anyone doesnt get the referance... youtube, and find power thirst.
FOUR HUNDRED BABIES
----------------
Tarminic - 31 Million SP in Forum Warfare
Play EVE: Downtime Madness v0.78.2 |
MotherMoon
Huang Yinglong
FOUNDATI0N
|
Posted - 2008.01.14 21:17:00 -
[25]
Edited by: MotherMoon on 14/01/2008 21:17:14
Originally by: Tarminic
Originally by: Alekzander
Quote:
I know this makes me a turbonerd, but the thought of the described final product makes me happy in the pants.
I can't wait until you guys can implement this on live. Keep coding + testing, and I'll try to be patient.
These aren't your dad's nerds, these are TURBONERDS, more NERDS than a server farm has ROOM FOR! lawl, i couldnt help my self... just in case anyone doesnt get the referance... youtube, and find power thirst.
FOUR HUNDRED BABIES
you'll have so much fps!
FOUR HUNDRED FPS
plus you'll win at everything forever!
like...
placeholder sig :)
bigger rez |
Tarminic
Forsaken Resistance
The Last Stand
|
Posted - 2008.01.14 21:25:00 -
[26]
Edited by: Tarminic on 14/01/2008 21:25:24
Originally by: MotherMoon
you'll have so much fps!
FOUR HUNDRED FPS
plus you'll win at everything forever!
like...
PVP!
*AAH!*
YOU'LL BE GOOD AT IT!
Okay seriously, back on topic.
Infiniband is sweet, and I'd love to see a Dev Blog on it in the near future. 
----------------
Tarminic - 31 Million SP in Forum Warfare
Play EVE: Downtime Madness v0.78.2 |
MotherMoon
Huang Yinglong
FOUNDATI0N
|
Posted - 2008.01.14 21:26:00 -
[27]
Originally by: Tarminic
Edited by: Tarminic on 14/01/2008 21:25:24
Originally by: MotherMoon
you'll have so much fps!
FOUR HUNDRED FPS
plus you'll win at everything forever!
like...
PVP!
*AAH!*
YOU'LL BE GOOD AT IT!
Okay seriously, back on topic.
Infiniband is sweet, and I'd love to see a Dev Blog on it in the near future.
I pictured your avatar with it's mouth gaping open lawls.
AHHH*
yes it will be sweet and full of win.
placeholder sig :)
bigger rez |
Wild Rho
Amarr
GoonFleet
|
Posted - 2008.01.14 21:37:00 -
[28]
Originally by: CCP Lingorm
stuff
So will this break the limitation of having a maximum of one node to a system that's been a restriction on what systems can ultimately handle player wise?
|
Jenny Spitfire
Caldari
SIVAKASI
|
Posted - 2008.01.14 21:37:00 -
[29]
Edited by: Jenny Spitfire on 14/01/2008 21:38:37
I don't think it can be done with a silver bullet Rho unless they have rewritten the architecture, Eve 2.
---------
Technica impendi Caldari generis. Pax Caldaria!
Recruitment
-KB- |
Ishina Fel
Caldari
Synergy.
Imperial Republic Of the North
|
Posted - 2008.01.14 22:01:00 -
[30]
Originally by: Wild Rho
Originally by: CCP Lingorm
stuff
So will this break the limitation of having a maximum of one node to a system that's been a restriction on what systems can ultimately handle player wise?
This is exactly one of the goals, if I understood the fanfest presentations right. Which one was it again... Hilmar's, I think? You can check it out here.
|
| |
|
| Pages: [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 .. 17 :: one page |
| First page | Previous page | Next page | Last page |