From Twitter:

EVE Online GÇÅ@EveOnline

This is looking to be a potential hardware issue. More updates soon as we learn more. Sorry for the current lack of pew pew!

Please eleborate.

How on earth would a single isolated hardware issue, be a problem to your servers? I would expect your systems to be redundant??

For the record, RvB asked for CCP's permission before we did our 10,000 rifter thing, and they said it would be ok and supposedly had the server reinforced for it. Obviously it was not reinforced enough.

Where's CCP Tuxford? I'm sure he's behind this............................again

It would be nice to have a brief but more detailed explanation other than, "we are having hardware issues." I mean it doesn't need to be a full brief. Either way, glad to see it back up and online I haven't seen a crash this bad in a loooong time. CCP, please keep us informed with the issue in a blog or something. We are PC gamers and though most of us don't fully understand or appreciate the complexities in the creation of EVE, some of us do get the details even if they are a bit technical.

To use a crude metaphor, I can say my computer is having issues. Ok, well that doesnt say anything. I believe CCP is smart enough to know that it is either the ram or HD in a simplistic viewpoint without knowing the actual cause. So maybe like which portion of the hardware failed?

I'm taking bets that a RAMSAN blew up in an nifty new way.... =)

If it's of any interest - I noticed that searches for places started returning zero results about 1/2 hour before the outage.

That was indeed the first symptom.

Ah okay, now this is all starting to make a little more sense. I'm guessing the services used to pull information from the databases failed or decided to go on strike. Or, the storage system decided to freak out for whatever reason that is stll being investigated. I know most of the information is pulled from our HDDs (Hard Disk Drives) like ship, station, planets, moons, etc. models and textures.
Im guessing is more along the line of all the meta data that is handled by the servers like player, market, corp, alliance, etc databases stored on the server HDDs and backup HDDs. It makes sense to me at least and I thank everyone at CCP currently looking into this for us.

Good luck in figuring it all out.

IIRC 99.7% of all data queries can be answered using the RAM. Which explains why it actually ran _somehow_ for a while, til they shut it down.

And I would assume the RAM handling stuff on a demand basis.. e.g. when something is needed the first time, it gets retrieved from the HDD and then stored in the RAM while there's still space. When the RAM is getting used up, there'll be some strategy to decide which data to release, likely the items with the oldest last query are dropped first. (I hope you don't mind, Explorer, but assuming is fun!)

Ok It is 1128am here in blighty having just finished a lovely pair of bacon cobs with the prerequisite tomato sauce and a nice cup of coffee I boot up Eve in the hopes of maybe wasting a couple of hours to see my Eve is not accepting connections..I assumed the problem listed in this thread had been resolved, any confirm or deny?

Ninja edit restarted the client and I'm in

I love the web! So... I'm curious, can you describe " bacon cobs with the prerequisite tomato sauce" in more detail?

(mods forgive the off topic, but I just have to know!)

to bring the whole data center up time percentages into respective (if you've not already done the calcs) there is a rating called Nines ... "four nines" - "five nines", that data centers try to pimp themselves as being. Kind of a rating that attracts clients to use their service. (like ours)

Four nines is 99.99% and works out to be 52.6 (rounded) minutes down time per year !
Five nines would be 99.999% and that is only 5.26 minutes per year !

I had to talk to my wife for like, hours. It was aweful

Bacon on a large crusty bread roll with Heinz tomato ketchup.

Been a couple days now... anything learned from the charred wreckage?

The community team are probably trying to get the virtual worlds team to say something more publishable than "our logs show nothing".

And based on my experience with server techs, they're in for a tough mission ;)