Pages: [1] 2 3 4 5 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 5 post(s) |
Jessica Danikov
Clan Shadow Wolf Fatal Ascension
257
|
Posted - 2014.01.29 10:57:00 -
[1] - Quote
I don't know what's going on inside of CCP, but short of arbitrary deadlines and expectations of progress that aren't being met and bad quality control, here's a few things you should think about and do, if you don't already:
A) All releases should be flexible. If the release isn't good, don't do it, postpone it until it's ready. Player trust and respect is not an infinite resource and every broken update taxes their patience.
B) All releases should be reversible. If the release is bad, roll it back, fix it, only release it when you're sure. The cost of a bad release should be an incentive for people to pipe up and say 'no, if we release now, it's going to cost us'. If anyone thinks a release isn't ready, it shouldn't happen. None of this late night burnout hacking to fix it in time for the next downtime- you'll burn out your employees that way.
C) Test, test, test. Mirror Tranquillity regularly and deploy the entire final release there (in the software industry, we call this a release candidate). Have a mass test to get people playing around with it. Most of the bugs your players ***** about on day 0 of your new release would be caught on Singularity fairly quickly. Once caught, fix those bugs, roll a new release candidate, mirror Tranquility, mass test again. Keep doing this until it's ready.
D) In-house testers. One of the greatest false economies in the software industry is not to have in-house testers. They're relatively cheap compared to your experienced employees and free up whoever is doing your testing to actually spend their time doing whatever it is they're paid to be doing- testing isn't worth their time until a problem has definitive reproduction steps. QA leads can also keep their eyes peeled for the 1/100 talent that ends up in your tester pool.
E) Be aware of your reputation. Unpopular features, for better or for worse, that's the domain of your designers and you have to ride that one out as long as you trust in their process. However, bad software process is entirely avoidable and CCP has a bad history on that front (yes, I know, it's negative bias- we notice everything that goes wrong and not all the things going right, but that's what makes it so important to be rigorous and thorough and minimise the number and severity of what bugs do make it into releases).
F)Be honest, communicative and apologetic. We're all human, we make mistakes. Just be aware if you keep making the same mistakes, you will lose good will (especially from people who know that you should know better).
Again, I don't know how much of this stuff you do, or how well you do it, but obviously there's room for improvement. Maybe a Devblog giving an insight to your development process is long overdue? |
seth Hendar
I love you miners
413
|
Posted - 2014.01.29 11:01:00 -
[2] - Quote
tbh, judjing from the past 2 years releases, the various feedback and results around some features, i'd say CCP has NONE of the above, and doesn't give a crap about it.....
non non regression test is a good example, how many time some "fixed" bug just swing back? |
Kryptik Kai
Red Federation RvB - RED Federation
8470
|
Posted - 2014.01.29 11:02:00 -
[3] - Quote
Its amazing how many software developers play EVE "Shiny.-á Lets be bad guys." -Jayne Cobb |
Kimmi Chan
Tastes Like Purple
876
|
Posted - 2014.01.29 11:05:00 -
[4] - Quote
Oh look another unemployed Software Developer and/or Project Manager looking for a job at CCP. "You should just create one thread and put all of your complaints in it instead of littering the forums with multiple threads." ~CCP Falcon
[Member of The BrownCoat Syndicate] |
seth Hendar
I love you miners
413
|
Posted - 2014.01.29 11:11:00 -
[5] - Quote
Kryptik Kai wrote:Its amazing how many software developers play EVE not really in fact, eve has always gethered a crowd of computer tech ppl, and many of those ARE actually, devellopers, or do work with some.
unlike many other game, i'd say the average eve player has a better understanding of computers, and the % of ppl working in this field is way higher than any other game |
Riot Girl
You'll Cowards Don't Even Smoke Crack
2638
|
Posted - 2014.01.29 11:17:00 -
[6] - Quote
The thing that confuses me a bit is why they released a patch after 6 weeks. Aren't content patches usually every 12 weeks? Well that made me think of something I read in the news about staff members being laid off from WoD and CCP saying they were going to focus more on Eve. Does this mean CCP are going to aim to release content every 6 weeks now? If they are, then idkaf about a few bugs which can be smoothed out later. Not that it ever bothered me anyway. Oh god. |
seth Hendar
I love you miners
413
|
Posted - 2014.01.29 11:23:00 -
[7] - Quote
Riot Girl wrote:The thing that confuses me a bit is why they released a patch after 6 weeks. Aren't content patches usually every 12 weeks? Well that made me think of something I read in the news about staff members being laid off from WoD and CCP saying they were going to focus more on Eve. Does this mean CCP are going to aim to release content every 6 weeks now? If they are, then idkaf about a few bugs which can be smoothed out later. Not that it ever bothered me anyway. actually, if it would be the case, i'd rather see them keep the 12 weeks pace and actually improve the quality of each release, by giving it more testing, and a better integration of player feedback.
this would avoid coming up with things like the ESS or the odyssey exploration, and bugs like the standing thingy we have atm..... |
Nevyn Auscent
Broke Sauce
1002
|
Posted - 2014.01.29 11:29:00 -
[8] - Quote
The biggest bug that people are complaining about got through over a month of testing on Sisi. Stop dumping on CCP and instead have words to all those people on Sisi to actually test stuff, not just fart around in expensive blinged out space ships that they can't be bothered affording or are too scared to ever fly on TQ. |
|
CCP Goliath
C C P C C P Alliance
1971
|
Posted - 2014.01.29 12:00:00 -
[9] - Quote
Jessica Danikov wrote:
C) Test, test, test. Mirror Tranquillity regularly and deploy the entire final release there (in the software industry, we call this a release candidate). Have a mass test to get people playing around with it. Most of the bugs your players ***** about on day 0 of your new release would be caught on Singularity fairly quickly. Once caught, fix those bugs, roll a new release candidate, mirror Tranquility, mass test again. Keep doing this until it's ready.
D) In-house testers. One of the greatest false economies in the software industry is not to have in-house testers. They're relatively cheap compared to your experienced employees and free up whoever is doing your testing to actually spend their time doing whatever it is they're paid to be doing- testing isn't worth their time until a problem has definitive reproduction steps. QA leads can also keep their eyes peeled for the 1/100 talent that ends up in your tester pool. ... Again, I don't know how much of this stuff you do, or how well you do it, but obviously there's room for improvement. Maybe a Devblog giving an insight to your development process is long overdue?
Since you took the time to write this, I will give you as frank a response as possible on the subjects that are under my control.
C - We were on Singularity from the 6th January with this release. Familiar as I am with the term release candidate, being in the software industry myself, I can tell you it was deployed on Monday morning to Sisi, as is par for the course with our deployments. This might seem crazily short to you, but our development method allows us to work right up until then because we do not hamper teams with code/feature freezes and have mature teams which practice 'Whole Team Quality' (more on this later) giving them the responsibility and accountability over the quality of the features they ship. This is a process I have seen evolve in my 2.5 years here to really empower teams, and it is one that I think works well. Further than this though, we deploy daily updates to Singularity to ensure users have the latest stable build available. Combine this with the internal QA coming from the teams, and our outsourcers running a full regression suite every 2 weeks, and you're looking at a pretty robust testing process.
D - I assume on this you are referring to a test house of entry level "testers" working for minimum wage with a 50-90% bimonthly churn. This is firstly unfeasible in Iceland, as the pool of potential employees does not cover the churn that these outfits have. Instead of throwing a large amount of un/semi skilled workers at the problem, we use highly trained and experienced outsourcers to handle our regression testing, and assist with processing bug reports, providing focused testing at team request, and performing targeted exploratory testing (the most valuable type of testing, for me). Our current outsourcing partner has been with us for 3 years now, and I travel to them each year to ensure their methods and practices are keeping to the high standard established. We also have a proven conduit of taking talent from them directly into our own workforce. CCPers who have taken this route include Legion, Sledgehammer, Lebowski, Antiquarian, and even my good self!
Devblog - As the current facilitator of CCP Reykjavik's Agile Community of Practice, I tabled that we write exactly such a devblog not a month ago! We do need to get started (I actually wrote part of it around a year ago, but it got tabled at the time) but there is a plan in place and many keen contributors. CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath |
|
Fix Lag
760
|
Posted - 2014.01.29 12:08:00 -
[10] - Quote
Antiquarian told me my computer sucked when it didn't. He's a big fat phony. :P CCP mostly sucks at their job, but Veritas is a pretty cool dude. |
|
Angelica Dreamstar
Epic Boo Bees
377
|
Posted - 2014.01.29 12:10:00 -
[11] - Quote
This really doesn't help understanding how such a bug can see the light on TQ, CCP Goliath. Your post just makes if look worse... EVE ONLINE: The universe is ours!- Join the Epic Boo Bees! (RP,PvE/PvP,wardecs,new players!)You're at it from day 0! |
|
CCP Goliath
C C P C C P Alliance
1975
|
Posted - 2014.01.29 12:11:00 -
[12] - Quote
Angelica Dreamstar wrote:This really doesn't help understanding how such a bug can see the light on TQ. Your post just makes if look worse...
That wasn't what I was trying to address, so that isn't surprising. I have addressed that in other posts so won't do so again. CCP Goliath | QA Director | EVE Illuminati | @CCP_Goliath |
|
Charlie Firpol
Noob Mercs Monkeys with Guns.
116
|
Posted - 2014.01.29 12:14:00 -
[13] - Quote
Its not a bug, its a feature and a great troll :) |
mr ed thehouseofed
Sebiestor Tribe Minmatar Republic
413
|
Posted - 2014.01.29 12:24:00 -
[14] - Quote
one of these days ccp is going grab the ten biggest complainers in this game, stick in a room and get them to fix everything they whine about ,so that way we can flame them for a change . honestly guys if you think your that good at solving all the problems with the biggest mmo ever ,**** off and make your own bloody game as a non it guy ( i mess up half the emails i send ) i think they do a good trying to balance the game to suit everybody, can't be easy either.
tinfoil hat on real gamers only need one toon-á |
Gregor Parud
158
|
Posted - 2014.01.29 12:25:00 -
[15] - Quote
CCP Goliath wrote:C - We were on Singularity from the 6th January with this release. Familiar as I am with the term release candidate, being in the software industry myself, I can tell you it was deployed on Monday morning to Sisi, as is par for the course with our deployments. This might seem crazily short to you, but our development method allows us to work right up until then because we do not hamper teams with code/feature freezes and have mature teams which practice 'Whole Team Quality' (more on this later) giving them the responsibility and accountability over the quality of the features they ship. This is a process I have seen evolve in my 2.5 years here to really empower teams, and it is one that I think works well. Further than this though, we deploy daily updates to Singularity to ensure users have the latest stable build available. Combine this with the internal QA coming from the teams, and our outsourcers running a full regression suite every 2 weeks, and you're looking at a pretty robust testing process.
That's all really nice but doesn't change the fact that VERY often the most hilarious bugs or issues simply make it through testing, SISI and bug reporting, obviously something isn't working as it should. Minor issues can always happen but CCP really has a vast history of royal fuckups, facepalms and "seriously?" in this regard, ones that if QA would actually play the game a bit (and have a clue on what they're doing) simply wouldn't happen.
What's the chances you have "CCP CAPLOCK" type of clowns working in QA? |
Altrue
Exploration Frontier inc Brave Collective
957
|
Posted - 2014.01.29 12:37:00 -
[16] - Quote
Would be cool to see more people being taking bugs as positively as they can on release date. This is not some regular MMO server with 2k players on. This is Eve, and there are some things that just wont work on TQ where they might work on Sisi. IT magic and stuff.
To me the standing bugs seemed like a pretty fun experience. Okay I wouldn't have liked that if it has been lasting for weeks, but one day, just like that, that's just an interesting experience.
Should they delay a point release date, they'd surely have people complaining about it one way or another. People will complain whatsoever.
Plus, releases have vastly been improved compared to a few years ago. Extended downtime is minimal, deployment is nearly perfect. There are obviously still some issues on the internal-testing side, but overall patch deployment has never been better. Signature Tanking - Best Tanking. |
Adunh Slavy
1328
|
Posted - 2014.01.29 12:46:00 -
[17] - Quote
No boot.ini issues? Ship it. Necessity is the plea for every infringement of human freedom. It is the argument of tyrants; it is the creed of slaves.-á-á- William Pitt |
Treborr MintingtonJr
Quantum Reality R n D Spaceship Samurai
153
|
Posted - 2014.01.29 12:49:00 -
[18] - Quote
I think more of us need to go on SiSi to improve the scope of UAT. |
Sexy Cakes
Have A Seat
712
|
Posted - 2014.01.29 12:49:00 -
[19] - Quote
The inability of CCP to scrap crap game designs that don't sit well with the player base has always been one of their major downfalls.
Anything less than a player riot in-game and CCP keeps their heads buried in the proverbial sand. Not today spaghetti. |
Angelica Dreamstar
Epic Boo Bees
377
|
Posted - 2014.01.29 12:51:00 -
[20] - Quote
CCP Goliath wrote:Angelica Dreamstar wrote:This really doesn't help understanding how such a bug can see the light on TQ. Your post just makes if look worse... That wasn't what I was trying to address, so that isn't surprising. I have addressed that in other posts so won't do so again. Yes. No. Not what I'm getting at.
It doesn't matter if you wanted to address this or not.
Your post is indirectly addressing every single bug out there, but especially all the big ones that still get to TQ. It creates the impression that you hired idiots who have no clue of what they are doing, because huge issues like the standings bug go live.
At the same time you write 'highly trained and experienced outsourcers'. What people perceive is the exact opposite! And that's really bad. People could feel lied to. Some can feel as if you'd think they are idiots. An example of a rather typical reaction is below.
You missed to link to your other post, to help people not jump to ...
'yeah. highly trained and experienced outsourcers... my ass. HOW CAN YOU NOT HAVE NOTICED THIS BUG? WHAT KIND OF IDIOTS ARE YOU HIRING???' etc etc.
Get the point? EVE ONLINE: The universe is ours!- Join the Epic Boo Bees! (RP,PvE/PvP,wardecs,new players!)You're at it from day 0! |
|
Nevyn Auscent
Broke Sauce
1003
|
Posted - 2014.01.29 12:55:00 -
[21] - Quote
Or possibly trolls will be trolls and sensible people actually read the Devs comments and take them on board, while emo raging people will rage no matter what occurs. The standings bug is not as big a deal as people are making it out to be, and it was fixed in a day. |
Treborr MintingtonJr
Quantum Reality R n D Spaceship Samurai
154
|
Posted - 2014.01.29 12:56:00 -
[22] - Quote
@Angelica Dreamstar (hopefully it works):
https://forums.eveonline.com/default.aspx?g=posts&m=4167435#post4167435 |
Angelica Dreamstar
Epic Boo Bees
378
|
Posted - 2014.01.29 13:13:00 -
[23] - Quote
Treborr MintingtonJr wrote: Thanks.
Things can happen. xD Highly trained and experienced ... and no one set as blue. EVE ONLINE: The universe is ours!- Join the Epic Boo Bees! (RP,PvE/PvP,wardecs,new players!)You're at it from day 0! |
Random Woman
Federal Navy Academy Gallente Federation
123
|
Posted - 2014.01.29 13:31:00 -
[24] - Quote
This is what I put into that survey form for whats bad/should change or whatever the last 3 yearly surveys. CCP QA is utter **** and needs to improve.
Either the testers have no idea what they are testing, they are just bad, or theiy are laughting their asses off after handing you test protocols.
I really do not think that there are full regression tests every two weeks, because then the only explanation for the product you ship is, that you do the testing, but ignore the results completly.
Most major patches come with a quality standard that is expected after the first one or two devoloper tests.
Developer tests suck, I do them all the time, and I miss a lot of bugs. Thats why you need a dedicated testing crew, that knows what the product is about in detail. Of cause that means to get told you made a mistake by someone, but it also means you have the chance to correct that mistake before everyone is affected by it.
Also it was on sisi for two weeks is no test either, I dont think that majority of eve knows anything about softwaretesting, they just want to fly ships they cant afford. And the ppl with software testing background probably cant be arsed to try stuff out, because software testing is work. I pay money to play a game, not to keep doing the suff I waste the other 10 hours every day on. |
seth Hendar
I love you miners
416
|
Posted - 2014.01.29 13:32:00 -
[25] - Quote
Nevyn Auscent wrote:The biggest bug that people are complaining about got through over a month of testing on Sisi. Stop dumping on CCP and instead have words to all those people on Sisi to actually test stuff, not just fart around in expensive blinged out space ships that they can't be bothered affording or are too scared to ever fly on TQ. it has gone throught because two things imao.
1- the release brought on TQ was NOT the same tested on SISI 2- there are NO test done in a systematic manner after compiling, and more alarming, before a realease is brought to TQ.
the minimalist of the test, I.E. having ONE person to actually connect to the game on a test server to actually check a few things was clearly NOT done, otherwise this bug wouldn't have slept through.
this one is major, and cannot be missed, we are not talking about an obscure issue occuring for a few ppl in a particular usecase with a probability to happen of 1%, we are talking about a major feature, not working for 100% of the players..... |
Andrea Keuvo
Science and Trade Institute Caldari State
235
|
Posted - 2014.01.29 13:33:00 -
[26] - Quote
I think players would be more forgiving of bugs found on release day if the release wasn't chock full of features that players spent days and sometimes weeks speaking out against while being nearly 100% ignored by CCP. |
Random Woman
Federal Navy Academy Gallente Federation
123
|
Posted - 2014.01.29 13:38:00 -
[27] - Quote
Andrea Keuvo wrote:I think players would be more forgiving of bugs found on release day if the release wasn't chock full of features that players spent days and sometimes weeks speaking out against while being nearly 100% ignored by CCP.
I disagree, i'd rather miss a feature than get 10 ****** features, that negatively affect onther 30 features that worked flawless before.
Something is systematically wrong with CCP QA, and it should change, there are no excuses. |
Kitty Bear
Disturbed Friends Of Diazepam Disturbed Acquaintance
1202
|
Posted - 2014.01.29 13:41:00 -
[28] - Quote
Kryptik Kai wrote:Its amazing how many software developers play EVE
Does 68xxx assembler count ?? |
seth Hendar
I love you miners
416
|
Posted - 2014.01.29 13:42:00 -
[29] - Quote
CCP Goliath wrote: Since you took the time to write this, I will give you as frank a response as possible on the subjects that are under my control.
C - We were on Singularity from the 6th January with this release. Familiar as I am with the term release candidate, being in the software industry myself, I can tell you it was deployed on Monday morning to Sisi, as is par for the course with our deployments. This might seem crazily short to you, but our development method allows us to work right up until then because we do not hamper teams with code/feature freezes and have mature teams which practice 'Whole Team Quality' (more on this later) giving them the responsibility and accountability over the quality of the features they ship. This is a process I have seen evolve in my 2.5 years here to really empower teams, and it is one that I think works well. Further than this though, we deploy daily updates to Singularity to ensure users have the latest stable build available. Combine this with the internal QA coming from the teams, and our outsourcers running a full regression suite every 2 weeks, and you're looking at a pretty robust testing process.
i am sorry, but i think i might share this thought with many others.
your "Whole Team Quality" thing is NOT working, clearly.
there have been enought proof of that for at least the last TWO years.
being accountable for a problem doesn't solve the problem itself, once it is in the wild (understand on TQ) it is already TOO LATE. any efficient QA goal is to PREVENT issues from being released, not to blame someone once it's out.
regarding SISI, i really appreciate the work you do with it, especially the regular updates you bring, but judjing from, again, the past two years, the build released on TQ is NOT a build that happened to have been on SISI, wich allow you to fix a few things in between, i understand, but also bring some other issues, often FAR WORSE.
last point, regarding regression, please have a look at your last two years bug reports, then come here again and tell us you actually do run a regression suite. |
Angelica Dreamstar
Epic Boo Bees
381
|
Posted - 2014.01.29 13:42:00 -
[30] - Quote
Kitty Bear wrote:Kryptik Kai wrote:Its amazing how many software developers play EVE Does 68xxx assembler count ?? AAAAAMMMMIIIIIGGGGGAAAAAAAAAA!!!!!!!!!!!!! :D EVE ONLINE: The universe is ours!- Join the Epic Boo Bees! (RP,PvE/PvP,wardecs,new players!)You're at it from day 0! |
|
|
|
|
Pages: [1] 2 3 4 5 :: one page |
First page | Previous page | Next page | Last page |