Pages: [1] 2 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 10 post(s) |
|
CCP kieron
|
Posted - 2007.04.24 18:45:00 -
[1]
Quality Assurance takes a lot of heat for the work they do. The tasks on their lists are long, varied and not always easy to perform, especially with a code base and community as large as EVE's.
So, what does this have to do with the new blog? Tanis, a former volunteer and current member of the QA team, makes an introduction, provides more insight into the QA process and bug reporting, announces a forthcomin feature and more.
Hit the link and read, Oops! I broke it. Enjoy!
kieron Community Manager, EVE Online |
|
|
Eldo Davip
Forum Moderator Interstellar Services Department
|
Posted - 2007.04.24 18:53:00 -
[2]
woot Tanis! Some excellent improvements on the QA side! Fewer bugs = Fewer complaints on the forums = Happy CRC \o/ ___
EvE +NLINE - T+TALHELLDEATH SUPPORTER
|
|
Lin Dei
Gallente University of Caille
|
Posted - 2007.04.24 19:00:00 -
[3]
Edited by: Lin Dei on 24/04/2007 19:04:56 Seems good, good to hear from you Tanis!
Edit: Just reread it (scrolled through it the first time). - How soon is soon? (and don't say soon lol) - Will the players that get maxed skills on SiSi be publically announced? - Will you ever make a list of things you want tested, as to encourage players to test that specific feature?
------------------ If you see me post it's because either: 1) Silly forums made me do it 2) Because I was too lazy to change characters 3) Because I wanted to 4) <insert mod comment here> |
|
Sahwoolo Etoophie
Forum Moderator Interstellar Services Department
|
Posted - 2007.04.24 19:03:00 -
[4]
I, for one welcome our bugfixing QA overlords!
|
|
Canine Fiend
Minmatar Gekidoku Koroshiya Buntai
|
Posted - 2007.04.24 19:08:00 -
[5]
Great blog Tanis. I really like the part about bug report feedback. I feel that more players would submit reports if they knew that it wasn't just being fowarded to the junk mail folder (even though I know it's not :) ). Keep up the good work QA!
-------------------
|
0raven0
OUTLAWZ IMMORTAL
|
Posted - 2007.04.24 19:38:00 -
[6]
/me puts on bugreporter hat and gets to work
Maxed out skills oh my goodness! ------
Quote: tuxford: AT LEAST ITS SPEELED CORRECTLY tuxford: spelled* Oveur: rofl
|
|
Eldo Davip
Forum Moderator Interstellar Services Department
|
Posted - 2007.04.24 19:39:00 -
[7]
Originally by: 0raven0 /me puts on bugreporter hat and gets to work
Maxed out skills oh my goodness!
425mil SP 4tw ___
EvE +NLINE - T+TALHELLDEATH SUPPORTER
|
|
NVail
Inactive Hardeners
|
Posted - 2007.04.24 20:05:00 -
[8]
We *have* submitted bug reports. For years. Still not fixed. Current main drone issue: If you engage a ship that is beyond your normal drone control range (range is extended through ECM drone skill or through drone link augmentor thingy), and something closer is shooting at you, the drone may decide to go fight it instead. It will reevaluate this decision whenever your main drone control range is crossed.
Last time I heard a talk about improved quality, my drones started killing my gang mates next patch. I'll believe it when I see it. ---
|
Kw4h
Dragon's Rage Intrepid Crossing
|
Posted - 2007.04.24 20:14:00 -
[9]
The feedback solution posted (send an email to the bugreporter) sounds like an awesome idea. I've only posted one or two bug reports (which were rather good ;)) but never got a single clue whether I supplied the right info or not, of if it was good enough to actually fix the bug.
By at least getting an email I would feel more involved, and committed to do some more bugreporting _ Planet Sight Wallpaper - EvE map |
Nisse Owned
The Order of Chivalry
|
Posted - 2007.04.24 20:57:00 -
[10]
Ahh finally an improvement to BR system, have a e-beer! :D
|
|
Ezra
Gallente Calista Industries
|
Posted - 2007.04.24 23:05:00 -
[11]
Will we also get feedback if the bug we report has already been reported and is in your system? (Your blog indicates this is not the case, but it would be really nice to know that it's already being dealt with, and possibly even what status updates there are on the bug.)
While a previous poster is describing one bug, I'm guessing the blog refers to the infamous "buddy agro" bug, often mistakenly referred to as the "drone agro" bug. I've reported that bug numerous times with zero response, so you have no grounds on which to claim that no bug report for that bug has been filed. I'll go submit it again once your new confirmation system is in place, otherwise, I've given up as reporting it in the past has simply been a waste of my time. ------------ Ezra Cornell
|
Par'Gellen
Gallente The Scope
|
Posted - 2007.04.24 23:07:00 -
[12]
Awesome stuff! Thanks Tanis!
After all these years of submitting detailed bug reports just to have them ignored I've lost faith in the current system completely and don't even do it anymore since it seems like such a waste of time. It's not a quick and simple thing to gather all that information together and present it in a step by step reproduction form. To do so and then get the impression that it prints out directly into a garbage bin is disheartening to say the least.
As soon as this bug report feedback system is implemented I'll start submitting bug reports again. Looking forward to it! ---
CCP : Save my mousewheel! |
DHU InMe
Gallente Aliastra
|
Posted - 2007.04.25 00:40:00 -
[13]
Originally by: tanis
ie:
* You submit a bug report, that bug report is for an issue we are not yet aware of, so we create a new entry in our internal defect tracking system based on the information you provided.. you then receive a nice mail informing you that it was a new issue and that it has been entered into our system. * If you submit a bug report but it doesnÆt contain enough useful information you will then get a mail sent to you telling you that we will need more info, and what additional info we would need.
Yes, finally we can see if we make gold or crap... __ UI Overhaul Eve Links http://myeve.eve-online.com/updates/patchnotes.asp HELP |
TOGAKURE Daisuke
Occam's Razor Combine R0ADKILL
|
Posted - 2007.04.25 02:16:00 -
[14]
I'd like to hear more about your bug-testing process and about any automation you can use in conjunction with Eve. My friend is a senior Q&A person in an security product company and he's been showing me a lot of things about bug hunting automation, pretty exciting stuff.
Obviously he's testing desktop apps which don't need 3D acceleration, so I'd like to hear about the differences :)
|
Nate D
New Atlantis Tek Corporation
|
Posted - 2007.04.25 03:09:00 -
[15]
Ah it's great to hear that you're with CCP now Tanis! Congrats!
-Nate
A New EVE Voice In Game Channel |
Kethry Avenger
Pale Riders Incorporated
|
Posted - 2007.04.25 03:40:00 -
[16]
So just a stupid new player question, is that a maxed out character on Singularity or Tranquility? If Tranquility I apparently need to learn how to bug report, well, and quickly.
|
MOS DEF
0utbreak
|
Posted - 2007.04.25 06:47:00 -
[17]
Nice read. Suggesting bug reports is good. Using the drone bugs as example is nothing but hilarious though. YOu imply that noone of you ever played this game if you need a bug report to see what's wrong with drones. Years of feedback on what is wrong with drones really should give you a hitn what is wrong with drones.
|
Sharcy
Sonnema
|
Posted - 2007.04.25 07:28:00 -
[18]
Being able to search through the bugreports database would be very, very helpful. Both for the users and CCP. It would lighten the load on QA, reported bugs could get appended, we'd be able to find solutions to perceived bugs etcetera.
--
|
Bartholomeus Crane
|
Posted - 2007.04.25 09:01:00 -
[19]
Just sending an email about whether the bug is already reported or not, or if more info is needed is not nearly enough. It doesn't let the reporter know if it is being worked on or not, or by who, or if it is fixed and awaiting deployment or further testing.
What this means is that the reporter only gets a confirmation after which the line goes dead. It's more than it was, but I fail to see what good it will do.
Given that most people I know have given up on bug reporting as a waste of time, I wonder if this isn't a case of too little, too late. For it to regain trust you need to expose more information, make developers/bug-testers responsible outward as well as inward. --
|
Dr Aryandi
Hematite Rose Bionic Dawn
|
Posted - 2007.04.25 09:09:00 -
[20]
Feedback FTW!
Seriously that's the single best thing you can do. I used to post bug reports but when it feels like you are just feeding them into a black hole it's very hard to stay motivated to do so.
Blueprint Research Service Available See thread for details.
|
|
Xenofur
Di-Tron Heavy Industries Freelancer Alliance
|
Posted - 2007.04.25 09:31:00 -
[21]
Nice blog.
No cookies for you...
Why?
Because it's bull****ting, lying and marketing blabla.
Fact: Drone bugs have been reported en masse. You claiming they weren't is just a simple lie. So don't tell us to submit bug reports, FIX THE PROBLEM THAT PREVENTS YOU FROM GETTING THEM INSTEAD OF LYING TO YOUR CUSTOMERS: Either actually start reading the bug reports in your submission database or fire the ******* who deletes perfectly fine bug reports.
Regarding feedback: Oooooh, nice, after 3 years you finally realize that a black hole is not worth fly ****. Thank you messiah!!! Not. First off, don't try to push this as you being nice or something, it's not, it's merely one tiny step towards fixing that which has been broken forever. Secondly, it's not even barely enough. You have a guy who is experiencing a bug and instead of giving him life assistance to confirm, verify and enter the bug, you only provide them with "your bug is valid", "it is not valid". That's about as useful as tying up the waiter's eyes and playing warm/cold with him as he is trying to bring you your food. At least one person who is afflicted by a bug should at all stages of it's treatment be informed of all your actions regarding it. If you do not want to do that then you're either to arrogant or too afraid of talking to your customers.
|
murder one
Gallente Death of Virtue
|
Posted - 2007.04.25 10:02:00 -
[22]
Originally by: Xenofur Nice blog.
No cookies for you...
Why?
Because it's bull****ting, lying and marketing blabla.
Fact: Drone bugs have been reported en masse. You claiming they weren't is just a simple lie. So don't tell us to submit bug reports, FIX THE PROBLEM THAT PREVENTS YOU FROM GETTING THEM INSTEAD OF LYING TO YOUR CUSTOMERS: Either actually start reading the bug reports in your submission database or fire the ******* who deletes perfectly fine bug reports.
Regarding feedback: Oooooh, nice, after 3 years you finally realize that a black hole is not worth fly ****. Thank you messiah!!! Not. First off, don't try to push this as you being nice or something, it's not, it's merely one tiny step towards fixing that which has been broken forever. Secondly, it's not even barely enough. You have a guy who is experiencing a bug and instead of giving him life assistance to confirm, verify and enter the bug, you only provide them with "your bug is valid", "it is not valid". That's about as useful as tying up the waiter's eyes and playing warm/cold with him as he is trying to bring you your food. At least one person who is afflicted by a bug should at all stages of it's treatment be informed of all your actions regarding it. If you do not want to do that then you're either to arrogant or too afraid of talking to your customers.
And I thought I had anger issues lol... <I just shoot mission runners to relieve stress, maybe you should too>
Because I said so...
|
|
Tanis.
|
Posted - 2007.04.25 10:06:00 -
[23]
To clarify, skillpoints will be given on Singularity server only!
The point of the blog was not to say that everything is magically going to be fixed, it would be silly to think it would be and I know you guys are smart enough to know better anyway. My blog also says nothing about people not submitting bug reports so I honestly have no idea where people have gotten that impression from. Heck it wasn't even to do with any drone issues, I just picked that randomly.
The point of my blog was to give those of you who wish to see the game improve some information about some improvements to the system that are being made as well as to re-state and re-inforce the fact that if you are going to file a bug report, or if you are going to inquire about when "bug X" will be fixed we will need more details than just "when will you fix the game?".
Or to give another example: A player on the test server once asked me "when will the POS bug be fixed?". Now, at the time he asked me, there were 15 separate defects relating to starbases (aka POSs), that's defects NOT bug reports. Since there were 15 defects and I am no mind reader there was no way I could answer such a vague question without asking for more information. Now had he asked "when will there be a fix for the issue where you would be unable to anchor a turret battery if you did not have sovereignty?" (I made that up BTW) then I would actually have enough information to go on to look into it and give him an answer.
The same would be said if he had submitted a bug report or reported the issue. When we have good information we can get things done faster, with little to no information we have nothing to go on and nothing can be done.
I'm not going to kid you guys, or myself by even pretending that EVE will ever be bug free. As long as the game exists, there will be bugs, this is a fact of life when it comes to programs. Having said that though, as stated in my blog I am doing what I can to help us take steps to reduce the number of bugs that slip through our testing and get more of the existing bugs fixed so that you folks can enjoy a constantly improving game.
To give you a few more numbers to munch on..
- On the average, in the past few months, there have been about 20 bug reports submitted per day, of those 20, 2-5 will become defects or be attached to already known defects.
- On the average day QA creates around 10 new defects (not counting the bug reports that become defects).
- On the average day QA verifies about 10-20 bug fixes.
- Currently there are a minimum of 800 man-hours being spent on testing each week here at CCP. That being split between testing new features/changes, verifying bug fixes, regular regression testing, as well as ad-hoc and unit testing.
- In addition to all of that we also run regular benchmarking and automated testing or the basic game mechanics as well as backend systems. This benchmarking and automated testing is being increased dramatically in the near future as well.
As time goes on I will continue to post more numbers and hard information about testing, QA, bugs, etc. so that you guys can see just what is going on, where we stand, and what progress has been made over time. ____________________________ I break things.
GM Voodoo > That plan really straddles the fine line between genius and idiocy. Tanis. > And that differs from everything else I say how?
[Bug Report Here][How to write a good bug report][Test server rules] |
|
Encad Briht
Minmatar United Univers
|
Posted - 2007.04.25 13:01:00 -
[24]
Just a question, would it be possible to make some kind of simplified public Bugtracker ? Cause then there would be more feedback and information on what you need more and detailed information and it would be also a feedback is certain things are resolved or doesnt need further information. --------------------------------------- Member of UU : Diplomatic Corps |
RedClaws
Amarr Dragon's Rage Intrepid Crossing
|
Posted - 2007.04.25 13:11:00 -
[25]
How many applications for becoming a volunteer ISD bug hunter are there at the moment?
I'm interested in becoming an ISD but on the ISD application form on the eve-o webpage there are radio buttons to select the department you want to apply for. As a result I can only choose one area (I guess it's intended and you don't want me to apply for more than 1 ISD position)
I'm interested in a lot of the ISD positions but bug hunting seems the most relevant to me (being a bachelor in informatics) but if there are too many volunteers already I'll probably apply for forum moderator.
|
|
Tanis.
|
Posted - 2007.04.25 13:27:00 -
[26]
Originally by: RedClaws How many applications for becoming a volunteer ISD bug hunter are there at the moment?
I'm interested in becoming an ISD but on the ISD application form on the eve-o webpage there are radio buttons to select the department you want to apply for. As a result I can only choose one area (I guess it's intended and you don't want me to apply for more than 1 ISD position)
I'm interested in a lot of the ISD positions but bug hunting seems the most relevant to me (being a bachelor in informatics) but if there are too many volunteers already I'll probably apply for forum moderator.
The Bug Hunters are almost always recruiting and we can always use more good people.
If you have any interest in joining ISD, or specifically the Bug Hunter team, I would strongly encourage you to apply.
Applications may take anywhere from a day to several weeks to be processed depending upon the number of applications that are coming in, but we are always looking to expand our team. ____________________________ I break things.
GM Voodoo > That plan really straddles the fine line between genius and idiocy. Tanis. > And that differs from everything else I say how?
[Bug Report Here][How to write a good bug report][Test server rules] |
|
Caius Sivaris
Minmatar Dark Nexxus Free Trade Zone.
|
Posted - 2007.04.25 14:29:00 -
[27]
Originally by: Tanis.
[list] On the average, in the past few months, there have been about 20 bug reports submitted per day, of those 20, 2-5 will become defects or be attached to already known defects.
I submitted a few bug report however I never had any indication that they went anywhere but a black hole, for lack of something as simple as a ticket number. Combined with some announcement of lost mails at time it makes me wonder if they were ever properly entered in a tracking system let alone read by an actual human and qualified.
Are we supposed to get a tracking number? And lets dream on, status updates on the report?
If not, no report from me, whines on forums give more feedback and helps vent frustration better.
Really, if you want people to help you give them something in exchange, an acknowledgement that the issue is looked on goes a long way.
|
Steppa
Gallente Incognito Inc
|
Posted - 2007.04.25 16:03:00 -
[28]
IÆve got to cast a skeptical eye on this whole sector of Eve. I understand that a clear, concise bug report is the key to tracking down and killing a bug. However, take the fighter MWD bug, for instance. I had TuxÆs ear on a forum thread once and laid out very specifically how easy it was to reproduce the exact same behavior every single time in fightersàie, that they do no MWD back to the carrier when recalled. He said something to the effect that he wasnÆt sure it was that cut-and-dry, etc etc. I had submitted, like others, a bug report on this very issue right after the carrier became part of the gameàover a year ago. IÆm not sure whatÆs going on behind the scenesàpossibly I missed somethingàbut they are still flying back to my carrier at slow, non-MWD speed.
|
|
Tanis.
|
Posted - 2007.04.25 16:12:00 -
[29]
Originally by: Caius Sivaris
Originally by: Tanis.
[list] On the average, in the past few months, there have been about 20 bug reports submitted per day, of those 20, 2-5 will become defects or be attached to already known defects.
I submitted a few bug report however I never had any indication that they went anywhere but a black hole, for lack of something as simple as a ticket number. Combined with some announcement of lost mails at time it makes me wonder if they were ever properly entered in a tracking system let alone read by an actual human and qualified.
Are we supposed to get a tracking number? And lets dream on, status updates on the report?
If not, no report from me, whines on forums give more feedback and helps vent frustration better.
Really, if you want people to help you give them something in exchange, an acknowledgement that the issue is looked on goes a long way.
Not meaning to be rude.. but did you read my blog? I believe you will find the answer to this question there. ____________________________ I break things.
GM Voodoo > That plan really straddles the fine line between genius and idiocy. Tanis. > And that differs from everything else I say how?
[Bug Report Here][How to write a good bug report][Test server rules] |
|
Luna Nilaya
|
Posted - 2007.04.25 17:33:00 -
[30]
Finding the bugs doesn't seem to be the problem as much as fixing them. Chimera is still a midget.
|
|
|
|
|
Pages: [1] 2 :: one page |
First page | Previous page | Next page | Last page |