EVE Search - Constant Start-Up Issues: Poor Practice

Check My IP Information

All Channels

EVE General Discussion

Constant Start-Up Issues: Poor Practice

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 2 :: [one page]

Author	Thread Statistics \| Show CCP posts - 2 post(s)
JForce Black Lance Against ALL Authorities	Posted - 2007.03.19 13:01:00 - [1] Of course as soon as I say the words "I work in I.T." I'm bound to be flamed. But I do And funnily enough I work in an area responsible for any changes, and the required release of that change, into the production environment. What I find unusual is the number of startup issues that occur due to changes made. Firstly, most are changes the community isn't told about. We know to set a long skill on patch day, and to be patient till it comes up. We know that it won't come up when you say it will. That is normal. However when you make changes, ANY changes to your production environment, you are taking a risk. And it seems to me that your risk-assessment procedures are poor at best. It's about managing expectations. We expect Tranquility to start up again at 12. Why? Because you haven't messed with it at all. If you're going to mess with it, in any way, then TELL US. That way we can expect to have an extended outage. I'm not going to comment on refunds etc, because updates are needed and outages happen. However CCP IMHO have poor change and release processes, and poor management of customer expectations around them.
Dravin Dread	Posted - 2007.03.19 13:07:00 - [2] Yep, just a litle CYA is all that is needed. CYA always a good idea when making changes.

Timmeh 2000 Forum Moderator Interstellar Services Department	Posted - 2007.03.19 13:07:00 - [3] This topic is already being discussed here. forum rules \| [email protected] \| Our Website! "Formerly known as MiniMeph" \| Do you Role Play in Eve?


Timmeh 2000 Forum Moderator Interstellar Services Department	Posted - 2007.03.19 13:47:00 - [4] bump Topic re-opened. forum rules \| [email protected] \| Our Website! "Formerly known as MiniMeph" \| Do you Role Play in Eve?

Melissa Ravenflame Caldari Aquae-Sulis	Posted - 2007.03.19 14:00:00 - [5] I suppose a lot of it involves having a representative architecture to test changes on. As far as I know (prepare flame retardant material ) CCP do not have a server farm as their pre-production environment which is an exact copy of production. As such, putting in a change, especially a hardware change, does leave them open to problems. The alternative is a very expensive, mostly redundant duplicate test system. However, I've done the Tech Arch for a number of clients which make very good use of that test system so it is possible
Fork Boy	Posted - 2007.03.19 14:02:00 - [6] I'm an old timer system admin too, and there are a few things I'd like to say. 1) Sharkbait showed a short while back that good communication is a great key to keeping your user base calm, even when things aren't working. 2) A little advanced warning would be appreciated. Just mention it to us that something big is going to go down. Probably best not to try to slip it in on the quiet he day before an announced big patch day. 3) Sadly, no matter what pre planning is done, it's not always possible to predict the behaviour of servers under load on a production system. Unless CCP can have an exact mirror of the TQ cluster, same size, same makeup, same simulated load. (At double the cost for subscribers?). We've gone live with a good few rollouts that have not behaved anything remotely like they did under our tests, when we extrapolated load / users etc. I really can't see what more CCP can do in reality, other than let us know what they are planning to do, and keep us up to date when it starts to go wrong. ================================= A fork is a cold, shiny tool To pierce, tear and ingest. Whoever has the fork in hand Controls the meal of their choice.
Jastra Gallente Gallente Venture Inc	Posted - 2007.03.19 14:27:00 - [7] Originally by: JForce Of course as soon as I say the words "I work in I.T." I'm bound to be flamed. But I do And funnily enough I work in an area responsible for any changes, and the required release of that change, into the production environment. What I find unusual is the number of startup issues that occur due to changes made. Firstly, most are changes the community isn't told about. We know to set a long skill on patch day, and to be patient till it comes up. We know that it won't come up when you say it will. That is normal. However when you make changes, ANY changes to your production environment, you are taking a risk. And it seems to me that your risk-assessment procedures are poor at best. It's about managing expectations. We expect Tranquility to start up again at 12. Why? Because you haven't messed with it at all. If you're going to mess with it, in any way, then TELL US. That way we can expect to have an extended outage. I'm not going to comment on refunds etc, because updates are needed and outages happen. However CCP IMHO have poor change and release processes, and poor management of customer expectations around them. The underlying and risk management/mitigation processes may be spot on, they key element missing is communication, though perhaps asking CCP to make public their change schedule is probably asking a bit much since most people will still post here and say "why isnt the server up" :D That being said, you work in IT but how many customers would you notify if a server was getting an upgrade during down time ? - admittedly the window is small here but even so, you might let the IT management chain know, maybe some team leaders that there is a risk, but would you tell everybody ?
Cadela Fria Amarr Eye of God X-PACT	Posted - 2007.03.19 14:37:00 - [8] Yes well, just about everyone in EVE is a Systems admin, game developer, server manager and/or work with the same things CCP do, and somehow everyone always knows better then CCP. Atleast thats the same argument everytime - "Hi I'm joe schmoe and I'm computer smart in one field or another, so therefor I unquestionably know what I'm talking about - This means you can't tell me I'm wrong...that being said, CCP is all wrong about how they do things and should bow their heads in shame and follow my every word! BOW DOWN!" Knowledge is a priviledge, not a right
Ms Freak Amarr NCN Corp Chaos Incarnate.	Posted - 2007.03.19 14:41:00 - [9] Originally by: JForce Of course as soon as I say the words "I work in I.T." I'm bound to be flamed. But I do And funnily enough I work in an area responsible for any changes, and the required release of that change, into the production environment. What I find unusual is the number of startup issues that occur due to changes made. Firstly, most are changes the community isn't told about. We know to set a long skill on patch day, and to be patient till it comes up. We know that it won't come up when you say it will. That is normal. However when you make changes, ANY changes to your production environment, you are taking a risk. And it seems to me that your risk-assessment procedures are poor at best. It's about managing expectations. We expect Tranquility to start up again at 12. Why? Because you haven't messed with it at all. If you're going to mess with it, in any way, then TELL US. That way we can expect to have an extended outage. I'm not going to comment on refunds etc, because updates are needed and outages happen. However CCP IMHO have poor change and release processes, and poor management of customer expectations around them. Good for you!! I'm a developer and agree that CCP change control is somewhat "pap" but theres no need for the whole "Wow i work in IT so i must be right" thing. I'm sure CCP know thier processes/methods/software/coffee machine is not great or working properly. Leave em to sort thier own s**t out.
Akita T Caldari Caldari Navy Volunteer Task Force	Posted - 2007.03.19 14:43:00 - [10] What the OP is saying is that the proxy server change on today's DT should have been announced in a news item a couple of days ago, alongside the announcement for the patch on the 20th. That, and only that. But in a lot more words. _ MySkills \| Module/Rig stacknerfing explained

Miss KillSome	Posted - 2007.03.19 15:17:00 - [11] just gives us some info when u are planning on doing something to TQ that could cause problems. That is hardware upgrades, small software upgrades, anything.. One day ahead i think u know that u are going to add some hardware, its not like its a 10 minute ahead decission..We dont want exact numbers, if u are afraid of some buisness leak (though it sounds fine that u are adding 10 more Blades to the whole system..). Just keep us informed on stuff, it doesnt take u even 10 minutes u write appropriate notice in news..
Jarphen Bard Gallente	Posted - 2007.03.19 15:51:00 - [12] I agree with the OP completely. Have the downtime, got to, no question, but just let us know if there are changes planned in the downtime so we can do a little risk management of our own. De motu corporum in gyrum
JForce Black Lance Against ALL Authorities	Posted - 2007.03.20 01:26:00 - [13] Thanks to the mod team for re-opening my thread And yes of course CCP are doing what they can as far as lessening the risks etc. But they should know better than everyone the environment they're dealing with, and by that I mean the gaming one. One poster asked if I would notify every person if were doing a minor hardware upgrade etc. The answer to that is that yes we would. Every time. We have to. But it's not a valid comparison....because only about 25% of our systems are 24/7. So if we make a change to a system outside of its agreed operating hours then it's a 0 impact change. We all know that Eve players are insane people. We're them. Every single day there are thousands of people sitting around waiting to log in after DT...to play. To change a skill. To complete a market order. Then go to bed/work/out. The best analogy I can think of is that you own a restaurant, a very popular restaurant. Your restaurant opens each day at 12. Every day hundreds of people line up outside to come in. If you know that one day you're having the dining room re-painted, and that the plan is for the painters to finish by 12, but there's always the chance for it to go longer...then you'd tell people. You'd put a sign up a few days before. So your regular customers wouldn't line up outside at 11:45. Maybe they'd plan to eat elsewhere that day, or they'd drop by an hour later. Manage expectations CCP. We still love you, but take note of what is some nice constructive feedback in this thread
Cadela Fria Amarr Eye of God X-PACT	Posted - 2007.03.20 01:54:00 - [14] Why whould they? If you're telling me they don't already know these things, then I laugh. Of course they know all this. It's just like I said in my last post. For some reason everybody is a darn expert and knows better then CCP these days. It's just friggin silly. No offence intended Knowledge is a priviledge, not a right
Viktoria Maher	Posted - 2007.03.20 02:01:00 - [15] Clearly not, as the change was not announced beforehand. ------- Hull tanking? Pffft real men pod tank - Godar Marak
JForce Black Lance Against ALL Authorities	Posted - 2007.03.20 02:13:00 - [16] Originally by: Cadela Fria Why whould they? If you're telling me they don't already know these things, then I laugh. Of course they know all this. It's just like I said in my last post. For some reason everybody is a darn expert and knows better then CCP these days. It's just friggin silly. No offence intended Well the problem is that there ARE people here who are experts at some things. That's not to say that CCP aren't. But I can confidently tell you that CCP aren't following industry best-practise with this. No one says they have to. But that's why you have to endure moaning thread after moaning thread when DT goes long cause an non-advertised change goes wrong.
Lokyi	Posted - 2007.03.20 05:55:00 - [17] Just like I posted in one of the many other threads about this topic (and I think this is actually the most coherent one I've read yet), I think CCP should make sure that at least ONE of these 100 new employees they're advertising for should be given the job of alerting the community to any possible modification being made to the server, either hardware or software. That way we can at least expect something to go silly when the server restarts. Well, expect when it's more likely to happen than normal anyway :P
Kylar Renpurs	Posted - 2007.03.20 06:02:00 - [18] With myself working in IT too, and although I don't diss best practice (though it is a pain), there's one very simple answer. Deployment issues. It doesn't matter how many times you test and re-test in development, when you go live it's murphy's law that there's always some problem which can't be anticipated.
Telemicus Thrace Thrace Inc Ushra'Khan	Posted - 2007.03.20 07:03:00 - [19] For those of us whose prime time is just after DT it is a major issue that seems to be geting more and more frequent. The lack of warning is the real killer, especially in this scenario where we lose Monday evening without warning when many of us were running around to fix stuff up before we lost Tuesday evening to the patch. If I'd known Sunday night I wouldn't be able to log in until Wednesday I would have done a fair few things differently. Contracts, POS fuel, skills. All these things run persistantly to the good or ill of the players. Under those circumstances there should be some warning if I'm going to be cut off for days at a time while others can still come along and blow my stuff up while I'm at work. I understand I'm stuck with mandatory 2 weeks DT a year because of my timezone. I understand that every time a patch comes out I lose a day. What I would like to see is all these little upgrades that routinely bugger up any logistics / operations / contracts / skill changes I have planned rolled into one upgrade. Take the servers offline all evening once a month if you have to and warn us in advance. That would be better than these random blackouts. I know I can't have a level playing field due to my timezone but cut us some slack. Just restrict these upgrades to designated patch nights. >> RECRUITING <<
JForce Black Lance Against ALL Authorities	Posted - 2007.03.22 13:14:00 - [20] Once again we see startup issues, affecting thousands of pilots. Problem Management time? How are these issues investigated and tracked I wonder?

heheheh Singularity.	Posted - 2007.03.22 13:17:00 - [21] Edited by: heheheh on 22/03/2007 13:14:08 Mummy mummy i want it now !!
Presidente Gallente	Posted - 2007.03.22 13:20:00 - [22] Edited by: Presidente Gallente on 22/03/2007 13:16:48 Should I tell you something: I can understand BUT I can live with it. And you should LEARN to. EVE is not the center of my reallife. I pay EUR 10,00 per months which means: 33 euro cent a day. I am working with hardware since 20 years. I know all the simple to complex problems I have with a single PC. So I understand that CCP is dealing with state of the art concepts to make EVE happen. From this POV I see no reason to have the right to demand something because I pay simple EUR 10,00 per month. Let them do their job, be patient ... CCP will not leave us alone. And: an unexpected downtime is a great advantage to let me cool down and concentrate on RL stuff. OVER! OUT! Pres
JForce Black Lance Against ALL Authorities	Posted - 2007.03.22 13:22:00 - [23] Originally by: heheheh Edited by: heheheh on 22/03/2007 13:14:08 Mummy mummy i want it now !! Why dont you try reading the thread? Perhaps then you'll see there's no whining, no "fix it now", just a civil discussion on the way CCP manage their DT startup problems. Try not posting straight out of your ******* next time.
JForce Black Lance Against ALL Authorities	Posted - 2007.03.22 13:23:00 - [24] Originally by: Presidente Gallente Edited by: Presidente Gallente on 22/03/2007 13:16:48 Should I tell you something: I can understand BUT I can live with it. And you should LEARN to. EVE is not the center of my reallife. I pay EUR 10,00 per months which means: 33 euro cent a day. I am working with hardware since 20 years. I know all the simple to complex problems I have with a single PC. So I understand that CCP is dealing with state of the art concepts to make EVE happen. From this POV I see no reason to have the right to demand something because I pay simple EUR 10,00 per month. Let them do their job, be patient ... CCP will not leave us alone. And: an unexpected downtime is a great advantage to let me cool down and concentrate on RL stuff. OVER! OUT! Pres However if you had a dozen POSs to refuel and defend and manufacturing to deal with and things like that, you might expect to come on and play. And if you weren't able to, to be told in advance when they make changes that could go wrong.
50freefly Caldari Purify Namtz'aar k'in	Posted - 2007.03.22 13:25:00 - [25] Originally by: Presidente Gallente Edited by: Presidente Gallente on 22/03/2007 13:16:48 And: an unexpected downtime is a great advantage to let me cool down and concentrate on RL stuff. Obviously not as you are posting on this thread! Oh btw hi =) Originally by: Eight Ace For reasons that have been lost in the mists of time all caldari ships are designed by two people. One does the left hand side and the other does the right. And they never meet.
Barbarellas Daughter Lonely Barbarella	Posted - 2007.03.22 13:33:00 - [26] i work in IT too. but i dont have anything to do with MMO servers. so i stfu.
Presidente Gallente	Posted - 2007.03.22 13:35:00 - [27] Originally by: 50freefly Originally by: Presidente Gallente Edited by: Presidente Gallente on 22/03/2007 13:16:48 And: an unexpected downtime is a great advantage to let me cool down and concentrate on RL stuff. Obviously not as you are posting on this thread! Oh btw hi =) Hehe ... damnit ... I am suffering from withdrawal syndromes and you smashed the truth in my face! An alcoholic would now lie to himself: forum IS rl stuff. I will call my therapist ... Pres
Seph Res PPN United Against ALL Authorities	Posted - 2007.03.22 13:36:00 - [28] i fully agree with the OP but i also can understand CCP in some ways, for example: i work on a matlab project to compute several hundred EEG signals with fast fourier transform... as long as i do not try to reconfigure some parameters all works perfectly fine..then i add some code to test somethin new, e.g only another time base into computing the algorithms, the theory says it will work perfectly cause u only changed a damn time base in a written program which is set up from the dev to run proper at even such changings...and what happens? suddenly the computing time is goin up to hundrets of seconds..u sit on ur machine watchin the busy screen for 20 minutes..just to get the damn message: sorry out of memory...u change 1 value back...voila it works again... the point is that in complicated code sometimes a small change can have big effect, if its that logic or not to happen.. remembers me all time of murphys law... so CCP please if u plan somethin to fix or make improvements to the game..just tell us, we all would understand if its announced...and i even think u have some time schedules what has to be done on a day of work...so most of the time if there are no unforeseen hamster suicides the daily plan will be available at least 1 or 2 days before...i think no one says, hey lets try somethin completely different just 30 mins before he came to work...
Presidente Gallente	Posted - 2007.03.22 13:38:00 - [29] Originally by: JForce Originally by: Presidente Gallente Edited by: Presidente Gallente on 22/03/2007 13:16:48 Should I tell you something: I can understand BUT I can live with it. And you should LEARN to. EVE is not the center of my reallife. I pay EUR 10,00 per months which means: 33 euro cent a day. I am working with hardware since 20 years. I know all the simple to complex problems I have with a single PC. So I understand that CCP is dealing with state of the art concepts to make EVE happen. From this POV I see no reason to have the right to demand something because I pay simple EUR 10,00 per month. Let them do their job, be patient ... CCP will not leave us alone. And: an unexpected downtime is a great advantage to let me cool down and concentrate on RL stuff. OVER! OUT! Pres However if you had a dozen POSs to refuel and defend and manufacturing to deal with and things like that, you might expect to come on and play. And if you weren't able to, to be told in advance when they make changes that could go wrong. Yeah ... that's the dark side of MMORPG's ... they rule and frak up your RL when you want to do big biz and be a big one in those games. At this point: you should learn to live with it. Even CCP can't help you really out. They have to deal with their own unexpected problems. See it that pragmatic. Pres
B0rn2KiLL DEATHFUNK	Posted - 2007.03.22 13:40:00 - [30] Originally by: Cadela Fria Yes well, just about everyone in EVE is a Systems admin, game developer, server manager and/or work with the same things CCP do, and somehow everyone always knows better then CCP. Atleast thats the same argument everytime - "Hi I'm joe schmoe and I'm computer smart in one field or another, so therefor I unquestionably know what I'm talking about - This means you can't tell me I'm wrong...that being said, CCP is all wrong about how they do things and should bow their heads in shame and follow my every word! BOW DOWN!" you are a troll, burn slowly please. as to OP, i agree. --- Originally by: Oveur It's important to understand that EVE is a "PvP" focused game

Hoshi Blackguard Brigade	Posted - 2007.03.22 13:40:00 - [31] Originally by: JForce However if you had a dozen POSs to refuel and defend and manufacturing to deal with and things like that, you might expect to come on and play. And if you weren't able to, to be told in advance when they make changes that could go wrong. You where told about the rev 1.4 patch weren't you? Well there you have it, the server is ALWAYS unstable a a week or 2 after a patch. So you had your warning you just choose to ignore it. ---------------------------------------- A Guide to Scan Probing in Revelations
Jim Pooley Cutting Edge Incorporated	Posted - 2007.03.22 13:41:00 - [32] Edited by: Jim Pooley on 22/03/2007 13:38:12 Originally by: Cadela Fria Yes well, just about everyone in EVE is a Systems admin, game developer, server manager and/or work with the same things CCP do, and somehow everyone always knows better then CCP. Atleast thats the same argument everytime - "Hi I'm joe schmoe and I'm computer smart in one field or another, so therefor I unquestionably know what I'm talking about - This means you can't tell me I'm wrong...that being said, CCP is all wrong about how they do things and should bow their heads in shame and follow my every word! BOW DOWN!" In an attempt to redress this balance, can I just point out I am not an expert in anything IT, programming, software related at all. So, I havent got a fecking clue what is wrong or how it can be done better. What I DO know however, is that Eve is great, beer is great too, and the two together are unpredictable but usually great also. (edit: for spelling) ------------------------------------------ Mines a Pint of Large
Splagada Minmatar Tides of Silence Hydra Alliance	Posted - 2007.03.22 13:41:00 - [33] ill do a "double" Post with your Stuff! ------ Relaxed corp recruiting
Jian Blade	Posted - 2007.03.22 13:43:00 - [34] Originally by: Cadela Fria For some reason everybody is a darn expert and knows better then CCP these days. I can't speak for everyone but some of us a experts and probably do know better than CCP. However, they sure as hell can't afford me and I am happy to let them get on with fixing it
DrAtomic Atomic Heroes The OSS	Posted - 2007.03.22 13:43:00 - [35] One big issue CCP has imo is the sheer size of the production environment which causes it's staging environment not to be an exact copy and therefor invalidates the whole staging environment or at least partly. Another thing at hand seems to be that the Database programmers have SA rights on the production environment, a comon issue as most sysadmins don't have sufficient DBA knowledge. This could be solved by training the sysadmins to have that knowledge or by introducing a true DBA who's only task it is to monitor the database programmers and code. Ow well always nice to type a lot of bla bla bla whilst making assumptions... hehehe ----------------------------------------------- Originally by: CCP Sharkbait we are screwed. My Top 10 List
BritBullet Antares Shipyards Galactic-Federation	Posted - 2007.03.22 13:45:00 - [36] It should be obvious that if the servers go down theres a chance it might not be up in time, it's common sense to put a long skill on and refuel before the server goes down.
Filatov Teg Rebirth Inc. Tau Ceti Federation	Posted - 2007.03.22 13:47:00 - [37] And for skills issue, when I see "Set a long skill" i heard "put a skill at least one week long" cause the stability souhld be chaotic some days after a patch. -------------------------------------- Ride on Shooting staaaaaaaarrrrrrrr!!!
Imiarr Timshae Roid Vandals	Posted - 2007.03.22 13:53:00 - [38] It is becoming more of a joke each day. I'm waiting for the "Patch day, set a long skill!" message to change to "Its DownTime day, set a long skill!". Wait a minuite... downtime happens every day! =( -Imiarr Timshae- Originally by: Nwalmaer Err, he has an Interceptor and I have lost 3 Crows catching Bships personally. Did I mention how much ass you all suck? -Pushing game mechanics since 04/03
RedooM Meed Caldari Chez Mamie Colette	Posted - 2007.03.22 14:05:00 - [39] Edited by: RedooM Meed on 22/03/2007 14:02:18 Listen good folks of Eve When I got in the game in mid 2004 at first I was complaining of issues in the game back then rofl and a player from beta and gave me this piece of wisdom concerning Eve Eve online is a bug and in the c r a c k s here is your game. It's sweet to see that nothing has changed since my first days when I was excited to fly a Merlin. Keep in mind folks that Eve is a big gaming experiment and a programmer challenge. The good news is that you are part of it. For better or worse you have chosen this game because bottom end we just love because it is very mentally challenging and we might need to forgive alot because of this. I vote 9 on 10 from last issues, guys up in Iceland and the server team in England are probably working their butts off to make it appen. Have a good one you all and fly safe.
Eis Bachs Lawless Industries Vis Major	Posted - 2007.03.22 14:06:00 - [40] OMG, hurry CCP my skill finishes in 13d 5h!!! Anyway. I think a lot of you are missing the OP's point. It's not that he expects everything to be perfect in the CCP environment. Its not that there will never be a problem, ever. It's more about the communication aspect of the problems than the problems themselves. And to all the folks that think CCP's environment is OMGUBERHOOOGE!!!!1111oneeleve. Get a grip. This is more medium sized than anything. Try working in an environment where death IS on the line. 911 service anyone? EB

w0rmy Intensive CareBearz	Posted - 2007.03.22 14:07:00 - [41] One would think, with the number of times theyre restarted the servers... Theyd be so well practised theyd be able to get it right. 5,4,3,2,1,5,4,3,2,1,5,4,3,2,1,5,4,3,2,1,5,4,3,2,1... Your signature is inappropriate. Please read the forum rules before reposting- Tirg
BritBullet Antares Shipyards Galactic-Federation	Posted - 2007.03.22 14:09:00 - [42] so change your skill when you log back on
fisty Aliastra	Posted - 2007.03.22 14:12:00 - [43] i dont think anyone gets the idea that any server downtime is for maintenance... Ciao
Crumplecorn Gallente Eve Cluster Explorations	Posted - 2007.03.22 14:13:00 - [44] I'm in a server startup thread. - Originally by: CCP Sharkbait we are screwed.
w0rmy Intensive CareBearz	Posted - 2007.03.22 14:14:00 - [45] Originally by: fisty i dont think anyone gets the idea that any server downtime is for maintenance... maintenance... and screwing with stuff. Its the screwing with stuff bit people would like to be notified about. Your signature is inappropriate. Please read the forum rules before reposting- Tirg
D2O HeavyWater Amarr Synergy. Imperial Republic Of the North	Posted - 2007.03.22 14:16:00 - [46] p1ss1n me off no end this twice ive come out of work to sneak a skill off this week n twice the servers dead. FFs why cant we set skills off via the website?? In preparing for battle I have always found that plans are useless, but planning is indispensable.
Vampr3ss	Posted - 2007.03.22 14:19:00 - [47] this is bol**cks, this sort of thing is happening way to often,with no apology or anything. CCP SORT IT OUT
Liliane Woodhead	Posted - 2007.03.22 14:26:00 - [48] There is no life except Eve ------ Need for Feed. The Hamsters ------
Wyehr Shadow Of The Light R i s e	Posted - 2007.03.22 14:47:00 - [49] First, the Tranquility server is unique. It is simply impossible to test certain things in any meaningful way because there is no way to simulate or replicate thirty thousand people interacting with it. Change management is great, but don't pretend that it will prevent or solve problems like this. Second, you can assume that downtime is used for tons of little activities that probably don't merit any mention to the community. By the time any given change gets interesting enough to mention, the people involved are probably far too busy fixing it to communicate beyond the "broken; fixing ASAP" level. [ 2007.03.18 18:45:59 ] (notify) Typhoon belonging to Gandolf self-destructs.
Ankanos Caldari School of Applied Knowledge	Posted - 2007.03.22 14:47:00 - [50] i'm here 3yrs+ -yeah i'm a private IT too..(big deal) overall... i'm still impressed with the quality of feedback CCP gives to its customers relative to any other company on the planet. -period. (granted prolly more than half the players in this game are techs of some sort and thrive if not demand being informed of pending techinal updates/issues but..) fwiw: i just got a pop-up news item while reading thru this thread (http://myeve.eve-online.com/news.asp?a=single&nid=1399&tid=1 (from my firefox plugin via http://eve-downtime.com -tyvm) so yeah, they could do with giving us a heads up ahead of time, but hey.. they saw the thread, got a clue, and did what you asked.. CCP listens.. i really dont think you could ask for more from any company -you sure as hell don't get this kind of feedback from industry leaders like, um..Microsoft. better late than never, i say.. keep up the good work guys.. -ank ---

CmdrThor	Posted - 2007.03.22 14:50:00 - [51] EVE MotD: "A harddisk will die tomorrow 20 minutes after DT. Because of this, DT will be extended for 37 minutes while we replace the harddisk when it dies. No need to set long skills as we know exactly which HDD will die and thus can replace it immediately." I guess something like this is what all extended-DT-whiners would like, isn't it?
Maam	Posted - 2007.03.22 15:10:00 - [52] People are going well away from the point of the OP, so this will spiral and spiral into Yet Another Generic Whine Thread (tm). The OP's well made point, was just that the day before patch day CCP installed some unanounced servers into the mix, which went wrong and had to be backed out. All he is asking is that CCP mention this to us, so we can make contingency plans and be aware there may be issues. Dear players, we're adding some new servers onto TQ today. Uptime is still expected to be at 12:00, however be warned there is a greater chance of cluster startup problems than usual. That's all.
Jarphen Bard Gallente	Posted - 2007.03.22 15:14:00 - [53] Originally by: Maam People are going well away from the point of the OP, so this will spiral and spiral into Yet Another Generic Whine Thread (tm). The OP's well made point, was just that the day before patch day CCP installed some unanounced servers into the mix, which went wrong and had to be backed out. All he is asking is that CCP mention this to us, so we can make contingency plans and be aware there may be issues. Dear players, we're adding some new servers onto TQ today. Uptime is still expected to be at 12:00, however be warned there is a greater chance of cluster startup problems than usual. That's all. Unfortunately, a lot of the peeps on these threads are flaming to relieve the boredom of the DT and use the DT complaint threads as a baiting exercise. They feel that because they don't care, no-one else should. De motu corporum in gyrum
ToranagaSama	Posted - 2007.03.22 15:29:00 - [54] Originally by: JForce Of course as soon as I say the words "I work in I.T." I'm bound to be flamed. But I do And funnily enough I work in an area responsible for any changes, and the required release of that change, into the production environment. What I find unusual is the number of startup issues that occur due to changes made. Firstly, most are changes the community isn't told about. We know to set a long skill on patch day, and to be patient till it comes up. We know that it won't come up when you say it will. That is normal. However when you make changes, ANY changes to your production environment, you are taking a risk. And it seems to me that your risk-assessment procedures are poor at best. It's about managing expectations. We expect Tranquility to start up again at 12. Why? Because you haven't messed with it at all. If you're going to mess with it, in any way, then TELL US. That way we can expect to have an extended outage. I'm not going to comment on refunds etc, because updates are needed and outages happen. However CCP IMHO have poor change and release processes, and poor management of customer expectations around them. I don't work in I.T., but I know a little. Perhaps the costs are prohibitive for CCP, at this time, but don't you think they s/b running redundant servers; meaning dual/simultaneously running environments. One server goes down, the second remains up and running. Maybe there is a few seconds to minutes of changeover and, perhaps, some relatively intitial small loss of data (backed up and restorable), but ultimately work continues virtually uninterrupted. From my experience, this is not a hardware or software issue, nor, for the most part, not a cost issue, but rather a mindset issue. Within management it has to be acknowledged and accepted that 365/24/7 uninterrupted service is mandatory and normal, as opposed to being exceptional. Once past the "mindset" issues, then its down to meat on the bone----DESIGN, DESIGN, and DESIGN. Here's a quick story: My brother recently took a supervisory network position for a large manufacturer. A couple months ago, one of his Administrators proceeded to initiate a shutdown so that HE could do some maintenance. Yes, the maintenace was needed, but it wasn't necessary at that moment (nor any moment during the working day!). The network shutdown would have cause the shutdown of the manufacturing floor. Now, not only would that facility not be making product, but an entire workforce (save the IT guys) would have been twiddling their thumbs for the rest of the day. My brother caught the guy, BEFORE, he completed his dastardly deed, and it was all my brother could do to keep from slapping him!! The moral of the story is that the Administrator just didn't comprehend what his purpose was, who his master was, and how he and his job fit into the overall scheme of the company. In short, he had an extremely narrow focus---in trying to do his job, he was completely NOT doing his job. In my travels, I've come across this sort of mindset many times, and particularly in IT departments. If it isn't laid out, from the start, emphatically and specifically how IT fits into the scheme of the company's purpose, it just won't be there. I believe CCP exhibits some characteristics of this behavior and mindset. That network is their baby! --- I speak from personal experience in an industry where the network is NEVER allowed to go down and is NEVER taken down (I should say incredible rarely. Perhaps once a year.) There is simply toooooo much money at stake. Last time I think it was down was 9/11! This is why it irks me when CCP's service goes down. It's completely avoidable! Cost and Design are the only real issues. Given that CCP runs a very leading edge technology network----Cost and Design can be prohibitive. Given CCP's level of communiation with its customers, I'm surprised they don't talk about these issue more.
RedooM Meed Caldari Chez Mamie Colette	Posted - 2007.03.22 16:07:00 - [55] Edited by: RedooM Meed on 22/03/2007 16:14:38 Let's all get this straight We have one single server Some login are like 20k to 30k from all over the world in a single environment. Do you have any idea of the technical opposition there is out there? Logic will say there will be major burps and issues for the least. I am completely amazed from my mind and point point of view. CCP has achieved such an working effort so far and from the beginning of the game. From readings of the very beginning of Eve and issues they dealt with from the beginning I know they will. I know this game will get better. Skilling is awfull but they like it that way :( Fly safe you all.
XiticiX Gallente Kudzu Collective	Posted - 2007.03.22 16:19:00 - [56] I also work in I.T. for a major government health organization, and I can tell you we have 3 servers for EVERYTHING. Production, Training, and Testing. Testing is an exact duplicate of production, and is fed information every night from it. I can see how having a testing environment for EVE (which is an exact duplicate of Tranquility) would be difficult with the huge hardware demands the game requires, but I still think it's a NEED rather than a WANT. A few million dollars spent on a Test server would go a long way towards client satisfaction and reduce the number of headaches the developers and managers must suffer from after every patch. It's simple, basic practice to have a test server. And it should be done right. ~~~ This is my sig. Do you like it? ~~~
Ankanos Caldari School of Applied Knowledge	Posted - 2007.03.22 17:33:00 - [57] Originally by: XiticiX I can see how having a testing environment for EVE (which is an exact duplicate of Tranquility) would be difficult with the huge hardware demands the game requires, but I still think it's a NEED rather than a WANT. A few million dollars spent on a Test server would go a long way towards client satisfaction and reduce the number of headaches the developers and managers must suffer from after every patch. It's simple, basic practice to have a test server. And it should be done right. i would agree, but given that is just a game (yes it's a paid service and a business per se, but..) -it still is just a game.. as such..unless they are netting say, 600mil-1bil a year.. i cant see the expenditure worthwhile to appease a bunch of bratty kids (ok granted, Eve is probably the only game that has more adults than kids, but you get my point.) getting bent out of shape because your game server was delayed coming back up is just lain silly iho. -regardless if it is the only thing in your life that is keeping you sane.. -ank ---
XiticiX Gallente Kudzu Collective	Posted - 2007.03.22 19:01:00 - [58] Originally by: Ankanos Originally by: XiticiX I can see how having a testing environment for EVE (which is an exact duplicate of Tranquility) would be difficult with the huge hardware demands the game requires, but I still think it's a NEED rather than a WANT. A few million dollars spent on a Test server would go a long way towards client satisfaction and reduce the number of headaches the developers and managers must suffer from after every patch. It's simple, basic practice to have a test server. And it should be done right. i would agree, but given that is just a game (yes it's a paid service and a business per se, but..) -it still is just a game.. as such..unless they are netting say, 600mil-1bil a year.. i cant see the expenditure worthwhile to appease a bunch of bratty kids (ok granted, Eve is probably the only game that has more adults than kids, but you get my point.) getting bent out of shape because your game server was delayed coming back up is just lain silly iho. -regardless if it is the only thing in your life that is keeping you sane.. -ank Well, I guess my point goes further than just having minor issues after a patch. What if the patch horribly corrupts the database, and it's down for a couple days? Eve-Online is more than just a game. To CPP, it's their business, and their sole source of income atm. If they were really serious about being taken seriously as professionals, they would treat it as such. Hence, having NOT ONLY a secure backup strategy, but a standard testing strategy for rolling out changes. Change Management is huge in the I.T. world, and CCP isn't following it at all. There's a great quote my manager once told me: "'Seat of your pants' is not a management strategy." ~~~ This is my sig. Do you like it? ~~~


Pages: 1 2 :: [one page]

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.