| Pages: [1] 2 :: one page |
| Author |
Thread Statistics | Show CCP posts - 3 post(s) |
|
CCP kieron
   |
Posted - 2007.07.04 20:02:00 -
[1]
With Revelations II and the first fix patch deployed, Tanis thought it time to follow up on his last Dev Blog with some news and updates on EVE's continuing QA process.
Want to read more and find out what is happening, not only now but in the future? Check out More news from the testing pits.
kieron
Community Manager,
EVE Online, CCP Games
Email/Netfang
Look Ma, I'm in a Dev thread! Oh wait... |
|
DarkZombie
Imminent Darkness
Fluff Bites
|
Posted - 2007.07.04 20:27:00 -
[2]
Good blog there m8
|
Paul X365
|
Posted - 2007.07.04 20:54:00 -
[3]
I hope your going to respond in this thread more than you did in the one you referenced in the blog about bug reporting.
1 reply on the day of your initial post and nothing since (7 Days) is about right for CCP but its just not enough, you must realise that if you don't take part in the thread nobody else will.
Bug reporters must be able to track there reports and reply to them if need be.
I see a lot of trumpet blowing about the number of man hours spent checking Tuesdays patch, but no mention of how some of the more obvious errors slipped through. And how's about spending some of those hours updating this page Known issues
Your ideas are very good, but they are seriously overdue.
|
Xaen
Caldari
Caldari Provisions
|
Posted - 2007.07.04 21:21:00 -
[4]
Wonderful (sarcasm).
Now fix the godforsaken ass-backwards broken client user interface please. (See link in sig.)
Support changing the UI here. |
Smakko
Ad Astra Vexillum
Brutally Clever Empire
|
Posted - 2007.07.04 21:57:00 -
[5]
great blog. glad to hear that CCP's getting a handle on this stuff as it ranks up at the top of the list of stuff i'd like to see get done.
|
Helison
Gallente
Times of Ancar
THE R0NIN
|
Posted - 2007.07.04 23:15:00 -
[6]
Nice to see that you are taking up several ideas, which were brought in the linked thread. But weŚll see, when this all is really working.
There are still much too many problems in Eve before I could start to applaud QA.
My main issue is still that CCP is not giving QA (and testers) enough time to work with the finished client. For big patches like Rev 2.0 we need a release-candidate (incl. patch notes) on the testserver for two weeks and not for 3 days. Bugs which are found in the last days (after patch-notes are available) are normally completly ignored, as there is no time for fixing.
|
Franga
Caldari
NQX Innovations
|
Posted - 2007.07.05 00:14:00 -
[7]
Great blog. Thanks for the information.
I'm particularly happy to see a 'mass-fleet' (for lack of a better term) simulation tool being written and put in use. I think that will go a long way to helping figure out alot of those bugs that tend only to appear once the new code is exposed to mass usage on Tranquility.
Again, thanks for the info.
_____________________________
Eldo spanked my sig but I can't be bothered changing it just now. |
Mnengli Noiliffe
|
Posted - 2007.07.05 06:22:00 -
[8]
Just add a hotkey or button in esc menu that would completely re-synchronize the client with server. It should fetch the current position of player's ship in solar system, all the objects surrounding the ship, the states of all modules etc.
To reduce stress, limit its use to, f.ex. 1 time per 5 minutes.
Because even if you fix some issues with desync, there will always be many more, as networks are not ideal and pockets WILL be lost. Think of it - how much reimburse petitions would you avoid if people could simply press the button when weird things start to happen and act on the current situation instead of looking the ship being destructed not being able to do anything (except logoff).
|
Miyamoto Uroki
Katsu Corporation
|
Posted - 2007.07.05 06:39:00 -
[9]
Would like to ask:
How many programmers are working on these benchmarking tools and test suite as their main work?
Is it only that one newly hired guy?
And do you have an ETA on when you hope to start using new benchmarking tools?
Oh, and good work with the bug reports thingy. An oversight of all known issues would be great. Oh wait, there is one already. It just needs more attention ^^
 Originally by: Puupuu
dude... your face
|
Miyamoto Uroki
Katsu Corporation
|
Posted - 2007.07.05 06:56:00 -
[10]
Another thing to mention:
Tanis' blog is a week old? You could have safed your company a lot of trouble on the forums, if you would have given the permission for publishing earlier.
Originally by: Puupuu
dude... your face
|
Takahashi Arran
coracao ardente
Triumvirate.
|
Posted - 2007.07.05 08:18:00 -
[11]
It read like a propaganda piece. Look at all these areas that we're doing so welll on blah blah blah... lots of things coming in the undefined future.. oh yeah thers this lagg issue that we havn't worked out and keeps getting worse but...
|
Miyamoto Uroki
Katsu Corporation
|
Posted - 2007.07.05 08:50:00 -
[12]
Agree with the post above. That's why i ask for numbers. How many programmers acutally involved, and not just 1 hour a week..
Timeframes? Any at all? Although i know that it's hard to calculate these things.
Originally by: Puupuu
dude... your face
|
solbright altaltalt
|
Posted - 2007.07.05 09:55:00 -
[13]
Originally by: Mnengli Noiliffe
Just add a hotkey or button in esc menu that would completely re-synchronize the client with server. It should fetch the current position of player's ship in solar system, all the objects surrounding the ship, the states of all modules etc.
I suspect the problem is solely internal to the client. So, a major rewrite of the client is needed to achieve this without it just getting lost on the refresh.
Quote:
To reduce stress, limit its use to, f.ex. 1 time per 5 minutes.
Vital function. Hotkey will need to be configurable also to change keys on excessive wear and tear. ;)
|
Nisse Owned
The Order of Chivalry
|
Posted - 2007.07.05 12:36:00 -
[14]
Great to see the bugreport system is getting some love finally, hopefully, it will become as useful as mantis and bugzilla soon 
|
fuze
|
Posted - 2007.07.05 13:40:00 -
[15]
Love the feedback on BR. And hopefully those IRC sessions with dev vs player work out fine as well.
And I hope that nasty lagmonster gets nerfed. |
Level5
|
Posted - 2007.07.05 13:46:00 -
[16]
Finally someone with common sense, thankyou for not using Eve Voice for the test round tables.
|
Wardo21
The Arcanum
|
Posted - 2007.07.05 14:30:00 -
[17]
My biggest gripe is with patches that break things. I understand the complexity of testing such a complex system, but things should not get broken by a patch. Moreso if there isn't a reference at all in the patch notes (that I saw anyhow) about those functions being modified in any way. For instance there are at least 2 things that worked pre-revelations patch, that don't work as of last night (post rev2.1 patch).
Warp gang (when in a fleet)
Deliver to (that was a handy right click menu item, instead of opening the corp hangar, then selecting from a list of corp mates, then moving the items in question...)
Fix things by all means, but don't increase your workload by breaking other things that worked before.
Wardo21
|
|
Tanis.
|
Posted - 2007.07.05 14:51:00 -
[18]
Originally by: Paul X365
...
And how's about spending some of those hours updating this page Known issues
...
You mean the Known issues page that was updated with the release of Revelations 2 and revelations 2.01?
Originally by: Helison
...
My main issue is still that CCP is not giving QA (and testers) enough time to work with the finished client. For big patches like Rev 2.0 we need a release-candidate (incl. patch notes) on the testserver for two weeks and not for 3 days.
...
The release candidates on our internal test servers are updated constantly (several times a day at the end of a testing cycle) only when we feel we have a stable build will we put it up on Sisi for public testing. This procedure will not be changed as we will not put out unstable test builds onto a public test server.
Originally by: Helison
...
Bugs which are found in the last days (after patch-notes are available) are normally completly ignored, as there is no time for fixing.
Actually, this couldn't be further from the truth. Allow me to explain how this works.
When a bug is found, we then find how severe it is, i.e. is it a minor issue, or does it break the game entirely? We will never delay a patch just because of a minor issue, that would be ridiculous; we will however, and have in the past, delay a patch to make sure that all major issues that have been found are resolved before releasing it to the live servers. Any major issues found after the patch is released are then considered for a hotfix, or a fix-patch. The less severe issues are then prioritized and we will include fixes for as many of them as we are able to in each future patch. Keep in mind though that programmers and designers can only do so much and the more major issues will always get more immediate attention.
As for people commenting about the de-sync issue. Solbright altaltalt was correct in that it is a much more complicated problem than simply adding a button that would fix it. First we need to find exactly what causes the de-sync and then find a proper way to fix it. There is no magic button you can press to solve all your problems, this is an unfortunate reality when dealing with systems as complicated as EVE.
I would, however, invite those of you who are concerned with the de-sync issues to visit this thread.
____________________________
I break things.
GM Voodoo > That plan really straddles the fine line between genius and idiocy.
Tanis. > And that differs from everything else I say how?
[Bug Report Here][How to write a good bug report][Test server rules] |
|
Helison
Gallente
Times of Ancar
THE R0NIN
|
Posted - 2007.07.05 18:13:00 -
[19]
Originally by: Tanis.
Originally by: Helison
...
My main issue is still that CCP is not giving QA (and testers) enough time to work with the finished client. For big patches like Rev 2.0 we need a release-candidate (incl. patch notes) on the testserver for two weeks and not for 3 days.
...
The release candidates on our internal test servers are updated constantly (several times a day at the end of a testing cycle) only when we feel we have a stable build will we put it up on Sisi for public testing. This procedure will not be changed as we will not put out unstable test builds onto a public test server.
It is not about putting unstable builds on Singularity but about waiting longer with deploying the patch to TQ.
|
Frug
Zenithal Harvest
|
Posted - 2007.07.06 04:02:00 -
[20]
I look forward to these changes.
The last bugreport I submitted was returned to me with a comment that didn't make any sense to me (asking for logs, when the issue had nothing to do with what gets logged). The bug got fixed in the last patch, but it still annoyed me to get the report closed like that. Hopefully a feedback system avoids this sort of thing.
- - - - - - - - - Do not use dotted lines - - - - - - -
If you think I'm awesome, say BOOO
BOOO!! - Ductoris
Neat look what I found - Kreul
Hey, my marbles |
solbright altaltalt
|
Posted - 2007.07.06 06:55:00 -
[21]
Originally by: Tanis.
Solbright altaltalt was correct in that it is a much more complicated problem than simply adding a button that would fix it.
Heh, I was originally making a little ironic humor.
That the client can't keep up with the incoming data is not a good situation.
|
Raquaine
Minmatar
KDM Corp
|
Posted - 2007.07.06 11:06:00 -
[22]
I was interested in the load testing tool you mentioned, to log in 300-500 clients and test e.g. simulated fleet battles.
To get more realistic test profiles, have you thought about recording some of the actual large battles in game? You could set up a way for players to warn you of likely flash points so you could 'turn on the recording', for example. If you can do it technically then coordinating with the community should be easy enough - just need to get the 10 biggest alliances to let their FC's know to drop a note in the help channel or something, and then turn on recording for the relevant system(s). Just a thought. |
Kimo Sabi
Minmatar
United Systems Navy
|
Posted - 2007.07.06 13:48:00 -
[23]
Edited by: Kimo Sabi on 06/07/2007 13:51:33
Originally by: Tanis.
Originally by: Paul X365
...
And how's about spending some of those hours updating this page Known issues
...
You mean the Known issues page that was updated with the release of Revelations 2 and revelations 2.01?
The one that still refers to the previous client build - maybe updating this with new "features" as they're found - and relative to the current build would at least let us know how old the feature is (the build it was first reported in) and what new bugs have been reported
For example:
Build 32517
- xxxx
- xxxx
- xxxx
- xxxx
- xxxx
Build 33752
- xxxx
- xxxx
- xxxx
Build 34381
- xxxx
- xxxx
This would allow us to see that you're working on problems (they would disappear from the list as the new patches are applied I'd hope) and we would also see a positive move in your goal to reducing the total number of new bugs introduced in each new patch.
Nice to see that QA is getting some love
EDIT: Might also be an idea to concatenate the patch notes (bug fixes, not introduction of new stuff) in some way so that we can see all the good work that QA has done since EVE's inception
========================================
Kimo Sabi's Mining Spreadsheet |
|
Tanis.
|
Posted - 2007.07.06 14:12:00 -
[24]
Though I don't see the Known issues list being broken down by build number, it's just too much work to track/update that, there are some plans right now to start tagging issues with something like "wil be fixed in Revelations 2.3" or something like that, assuming that a fix is ready for it, that way people can see which of these issues has already been sorted and when the fix will make its way to TQ.
As for the Bug Reporting feedback system, there may be times when the information you get about your bug report doesn't make sense, or doesn't seem quite right but that doesn't mean that the bug report was not handled properly. In the example given above where you got a reply stating that we needed logs, for most bugs, bugs that would involve a vode fix, client-side logs will almost always be of use of need to us. The logs that is referring to is the ones obtained by running LogServer.exe, you can find more information about this by following the "how to submit a good bug report" link in my signature. So the bug itself may not be about something getting logged (ie. wallet, journal, etc.) but that does not mean that we wouldn't want the client side logs.. those are an entirely different type of logs, logs that tell us what the client is doing, what errors it is throwing, and what it may not be doing right.
Granted, because we have many different people who will filter the bug reports, it is completely impossible to get 100% consistant types of replies, and because of the sheer number of bug reports we handle daily it is not feasable for us to give "live" replies. Also, unfortunately, because of the system we have in place, how we handle the bug reports and defects, how we track the bug reports and defect, and again the sheer number of bug reports we get, we also cannot allow bug reports to be updated after they have been submitted. This may change as we investigate ways to update and improve the system, but not for the forseable future.
I would however like to get your feedback about the system and hte responses you are getting. What other information can we give you to help you know what has been done with your report, or what else we would need from you in order to reproduce your issue? Please post your suggestions and comments here.
____________________________
I break things.
GM Voodoo > That plan really straddles the fine line between genius and idiocy.
Tanis. > And that differs from everything else I say how?
[Bug Report Here][How to write a good bug report][Test server rules] |
|
HeadTrader
|
Posted - 2007.07.06 14:48:00 -
[25]
Edited by: HeadTrader on 06/07/2007 14:53:43
Edited by: HeadTrader on 06/07/2007 14:52:42
Ok, I'm in project management myself, I know the pains but really most important IMHO:
1. Stop update of Known Issues "once in a while hand job no one needs and wants". In this way nobody will ever look at this page anymore. Connect it to your internal bug tracking system, of course you don't need to report there all bugs, but only these with "make public" flag.
2. Do some good PR and flag those resolved in test build ("resolved in test build XXX.YYY" with timestamp). Let ppl know you do something, even if it is not live!
3. Make patch notes with fixed bugs/changes (again, via direct output from internal bug system) to ALL TEST BUILDS. Let your testers know what you changed and what should they test/pay attention.
My 3 cents :)
|
Ikkajo
Minmatar
Illudium Space Products
|
Posted - 2007.07.06 15:16:00 -
[26]
As someone who works almost exclusively in large scale open source development, I have to comment that the inability to update your bug report with more information is disasterous. We suffer a lot of the same problems with people only half reporting an error or the reproduction steps don't work. By asking for more feedback or more targetted testing, we can narrow down problems a heck of a lot quicker. If you can't allow the end user to add more feedback to their own bug report (or even better to any other report), then publically displaying them is almost worthless. At best it will stop others reporting the same bug, but you won't get any idea of the magnitude of the problem then. At worst, you will get a lot of people creating more bug reports to add in their updated info, leading to a lot of extra work.
One other option I haven't seen proposed yet is a voting system. Each account gets X number of votes to distribute amongst the reports they see as being the top priority to fix. (5-10 is the number I've typically seen in big projects). That will give you developers a guide to what the userbase sees as the most important problems to be fixed (ie, camera reset after session change would most likely be at the top of the list).
--
Industrialist Carebear, CEO
Illudium Space Products: Where's the KABOOM!?
LP offers by corp at the LP Store DB |
Emily Spankratchet
Minmatar
Pragmatics
|
Posted - 2007.07.06 15:46:00 -
[27]
A couple of things on my wish list:
Log server: I'm sure it's not easy, but I'd love to be able to attach the log server to an already running client. You've missed out on decent bug reports from me many times because I haven't been able (or wanted) to quit, start logserver, and restart. If you get a bug five stages deep in a mission or being chased by a blob, you don't want to have to restart your client to get a log.
Client/server logging: Not easy, and maybe impossible with current system design. Try and find a way around the dreaded "bla bla bla our logs indicate there wasn't a problem bla bla bla". With the current desync issues, can what is logged (both client and server side) be changed to give a better indication of when and why these problems occur? There's little point us submitting logs if the problems can't be identified from those logs.
|
Hamfast
Gallente
|
Posted - 2007.07.06 17:44:00 -
[28]
Originally by: Tanis.
...
I would however like to get your feedback about the system and hte responses you are getting. What other information can we give you to help you know what has been done with your report, or what else we would need from you in order to reproduce your issue? Please post your suggestions and comments here.
Any update on any of the ideas listed on the above linked thread being implemented?
None of us is as dumb as all of us...
|
Frug
Zenithal Harvest
|
Posted - 2007.07.06 23:42:00 -
[29]
Thanks for the reply. I hear you. I didn't think a logserver file would help, but I guess that's why i don't work as a coder. I also wasn't sure what log you wanted, there also being the dxdiag, screenshot I included, crashdump (which is n/a in this case). I figured I had enough given how small and easy to reproduce the bug was. It felt like an automated brush off, which isn't nice.
- - - - - - - - - Do not use dotted lines - - - - - - -
If you think I'm awesome, say BOOO
BOOO!! - Ductoris
Neat look what I found - Kreul
Hey, my marbles |
DHU InMe
Gallente
Aliastra
|
Posted - 2007.07.07 02:35:00 -
[30]
Help to fix sound ;) ?!
www.audiokinetic.com
Tool that help you manage sound, effect, mesure cpu and memory usage, footprint, stress test etc
Worth alook from sound programmer !
Overhaul //todo remake
EvE Links //todo remake
RTFPN HELP |
| |
|
| Pages: [1] 2 :: one page |
| First page | Previous page | Next page | Last page |