Pages: [1] 2 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 3 post(s) |
|
CCP kieron
|
Posted - 2007.07.09 22:18:00 -
[1]
Following up on Tanis' dev blog of last week, new member of the QA department Lingorm makes his introduction and expands on not only his role on the team, but also tells about forthcoming changes to testing and how they will affect EVE.
For more, read: Testing, Automation, Monitoring and Benchmarking OH MY!
kieron Director of Community Relations, EVE Online EVE Online, CCP Games Email/Netfang Look Ma, I'm in a Dev thread! Oh wait... |
|
Frug
Zenithal Harvest
|
Posted - 2007.07.09 22:45:00 -
[2]
Edited by: Frug on 09/07/2007 22:50:23 Will this have any effect on the heavy lag areas though? Missions and fleet battles. The two important times when ships die and, ironically, lag is the worst. Wandering around space isn't when I experience any lag.
- - - - - - - - - Do not use dotted lines - - - - - - - If you think I'm awesome, say BOOO BOOO!! - Ductoris Neat look what I found - Kreul Hey, my marbles |
Del ReyII
Caldari
|
Posted - 2007.07.09 23:40:00 -
[3]
Glad to see some headway in this area. The game is in a bit of trouble at the moment.
Looking forward to renewing my subs when the improvements are done.
|
Lord Darcy
|
Posted - 2007.07.10 00:05:00 -
[4]
If it would help i run EVE on 3 dedicated machines (yes i'm a nut with 10+ accounts)feel free to fire them up past 3PM EST. My wife says i shouldn't be playing then anyway. I bet there are other nuts that would step up as well if you could get productive real data.
|
Xaen
Caldari Caldari Provisions
|
Posted - 2007.07.10 00:56:00 -
[5]
Admirable goals I know, since I'm a software engineer myself (or glorified code monkey if you don't like PC terms), but what I really wanna know is when are you going to fix the client UI? ------------------------------------------------------------------- Support fixing the EVE UI |
Sevarus James
Minmatar Meridian Dynamics FREGE Alliance
|
Posted - 2007.07.10 00:58:00 -
[6]
I saw this part of the blog, and reading between the lines I'll say thank you for this bit of information. (Pointing to my sig link to explain my smile upon reading this. )
Quote:
This is expanding and we are getting a dedicated group of machines to perform these tests. The machines are not top of the line but are a selected set of hardware that matches our 'Minimum Specification' and our 'Recommended Specification' and will have a number of different Operating Systems installed on them to cover the different OS's are supported by EVE.
Ubuntu 3d Beryl-Linux Desktop+EVE |
Lord Zuku
The Military
|
Posted - 2007.07.10 01:33:00 -
[7]
Originally by: Sevarus James I saw this part of the blog, and reading between the lines I'll say thank you for this bit of information. (Pointing to my sig link to explain my smile upon reading this. )
Quote:
This is expanding and we are getting a dedicated group of machines to perform these tests. The machines are not top of the line but are a selected set of hardware that matches our 'Minimum Specification' and our 'Recommended Specification' and will have a number of different Operating Systems installed on them to cover the different OS's are supported by EVE.
Linux isn't the officially supported platform. I think he meant different versions of Wintendo, like XP SP2 32/64-bit, Vista 32/64, 2000 SP4, 2003 SP2,...
I am happy to see more things being done in the area of performance testing and Python upgrades, however I still think that the solution to a great part of server performance problems lies in the change of server architecture from a fake cluster to a real cluster. Scaling out is the way to go. Scaling up is easy, but eventually you reach the top and then you have a problem (performance issues, expensive hardware).
|
Bein Glorious
GoonFleet GoonSwarm
|
Posted - 2007.07.10 02:33:00 -
[8]
Originally by: CCP Lingorm For some time we have had an automated test that logs a client in, runs a set series of tests and records the results to a database. We now have an internal tool that allows us to graph the results and compare them to other builds, examine trends and make sure that we are getting more out of the client. This is expanding and we are getting a dedicated group of machines to perform these tests. You will soon get to see this on Tranquility. We have a new set of 'agents' that will be flying round EVE recording the performance of EVE. This data will then be used to compare the new development builds against to ensure performance gains. All of these 'characters' are members of the 'CCP Space Monitoring Service' so that they can not be impersonated and will be flying Polaris frigates to prevent them from being targeted and from targeting players.
the devs are macroers |
Dread Operative
Sniggerdly Pandemic Legion
|
Posted - 2007.07.10 03:12:00 -
[9]
Are we going to get downloadable copies of the trimmed version of the client?
|
NeonNero
Gallente Center for Advanced Studies
|
Posted - 2007.07.10 04:56:00 -
[10]
Speaking of stress testing, CCP should seriously consider using virtualization (even something as simple as Microsoft Virtual PC would do) for different software and hardware environments (in terms of available memory and such).
This might also provide useful to see whether the EVE client is still able to support older Windows OS'es (98/Me/2000), even if you're no longer officially supporting them. -- NeonNero Deep within the depths of my mind. |
|
solbright altaltalt
Federal Navy Academy
|
Posted - 2007.07.10 06:49:00 -
[11]
Originally by: Lord Zuku Scaling out is the way to go. Scaling up is easy, but eventually you reach the top and then you have a problem (performance issues, expensive hardware).
Stating the obvious aren't we?!
How about coming up with an outline of how a single battle can be shared across nodes without a massive amount of locking bringing it to a crawl.
|
solbright altaltalt
Federal Navy Academy
|
Posted - 2007.07.10 06:54:00 -
[12]
Originally by: NeonNero Speaking of stress testing, CCP should seriously consider using virtualization ...
Can't say I've tried such a toy but, from what I've heard, if you are trying to test anything more than a database/desktop setup then the VM will wreak havoc on performance. Full-on DirectX games ain't quite in the same ballpark as a web app.
|
solbright altaltalt
Federal Navy Academy
|
Posted - 2007.07.10 06:57:00 -
[13]
Originally by: Frug Will this have any effect on the heavy lag areas though?
That will be the aim. Along with verifying some of more oddball claims.
|
Mamarto
Minmatar Sebiestor tribe
|
Posted - 2007.07.10 07:07:00 -
[14]
Originally by: solbright altaltalt
Originally by: Lord Zuku Scaling out is the way to go. Scaling up is easy, but eventually you reach the top and then you have a problem (performance issues, expensive hardware).
Stating the obvious aren't we?!
How about coming up with an outline of how a single battle can be shared across nodes without a massive amount of locking bringing it to a crawl.
Someone forgot to switch to their dev character before posting...
|
|
CCP Lingorm
|
Posted - 2007.07.10 09:10:00 -
[15]
Hello.
Thought I would jump in and answer some of the questions that have come up.
"Will this have any effect on the heavy lag areas though?" - I assume you are meaning the new Monitoring? in which case the answer is not directly, we will not be putting agents directly into heavily 'lagged' systems, this would only increase the load, which is not what we want. And testing against specific lag, fleet battles and missions, will be conducted on the test servers as we can more actively monitor the entire code coverage.
"I saw this part of the blog, and reading between the lines I'll say thank you for this bit of information." - No problem, and Yes Linux is currently support through the Cedega game engine. But watch this space, is all I am saying.
"the devs are macroers" - err ... sort of? But we macro the Test server and sell isk there for large mark ups and ... oh wait ... everything is really cheap there ... damn there goes my plans for galaxy wide market domination.
"Are we going to get downloadable copies of the trimmed version of the client?" - No.
Now, Virtualization. In its current incarnation Virtualization will not help the performance of the servers in anyway. We do make some use of it internally for testing clients and such, but not much as it is video and audio drivers that need the major testing done against them and the Virtual environment does not really aid with this.
"Fake cluster vs Real Cluster" - err, fake response? Just kidding. The hardware environment for EVE never 'stays still'. As we deploy one incarnation, the next one is in research and/or vendor discussions. EVE is ever evolving and so is its hardware.
CCP Lingorm CCP Quality Assurance QA Engineering Team Leader
|
|
Patch86
Di-Tron Heavy Industries Atlas Alliance
|
Posted - 2007.07.10 10:31:00 -
[16]
Originally by: CCP Lingorm
"the devs are macroers" - err ... sort of? But we macro the Test server and sell isk there for large mark ups and ... oh wait ... everything is really cheap there ... damn there goes my plans for galaxy wide market domination.
Yes yes, I'm sure. I await your appearance on eBay very soon
Anyhow, much sexiness in that blog. The thought of 100 machines running 50 clients each spamming the test server makes me feel warm and fuzzy. And really excellent to hear that you'll be testing under "the different OS's"- we Linux monkeys need love too!
I do have a question though: this trimmed down ultra-lite client- any chance we'll ever see it?! People have been demanding a practically text-based client since time immemorial. If you gave me the option of playing on a client with EVERYTHING except the functional stuff stripped away, I'd be at it like a flash... --------
|
Tareen Kashaar
Jericho Fraction The Star Fraction
|
Posted - 2007.07.10 11:34:00 -
[17]
Nice blog, good to see there are more hands joining the endless battle against the lagmonster :)
Speaking of which, this is probably the wrong thread for it, but... I've noticed that some grids and systems really seem to lag out horribly 5-15 minutes after there has been pewpew combat. We had this in Amarr last Thursday at around 21:30 iirc, when there was a mid sized skirmish at a gate by some other people, yet we on another grid got hit by a few terrible lagspikes a few minutes after the fact.
And then there was a 40vs40, and it seemed like people's wrecks and drones started causing massive lag only after they started dying. I sense some potential for optimization right there, after all dead-in-the-water drones don't have to perform any functions until they're scooped and under someone's control again, right? Well, maybe I'm wrong, but just wanted to give some food for thought.
Good luck with the testing suite! I especially like the thought that all the CCP desktop PCs start playing Eve after office hours... if you don't watch out, they'll develop a hive mind! :) --- WTS: Forum Signatures, price negotiable. Evemail me!
|
solbright altaltalt
Federal Navy Academy
|
Posted - 2007.07.10 11:38:00 -
[18]
Originally by: Mamarto
Originally by: solbright altaltalt
Someone forgot to switch to their dev character before posting...
Nah, I'm just another nerd with a knack for fault finding that got hooked on Eve. My main is expired for the moment.
|
RossP Zoyka
|
Posted - 2007.07.10 11:56:00 -
[19]
Dude, insanely anal engineers who want to fix everything FTW!!!
|
solbright altaltalt
Federal Navy Academy
|
Posted - 2007.07.10 14:04:00 -
[20]
I'll settle for the insanely anal stutter. :P
|
|
Tareen Kashaar
Jericho Fraction The Star Fraction
|
Posted - 2007.07.10 15:40:00 -
[21]
Originally by: solbright altaltalt I'll settle for the insanely anal stutter. :P
That sounds... kinky. --- WTS: Forum Signatures, price negotiable. Evemail me!
|
Pang Grohl
Gallente
|
Posted - 2007.07.10 16:07:00 -
[22]
Originally by: CCP Lingorm
"Fake cluster vs Real Cluster" - err, fake response? Just kidding. The hardware environment for EVE never 'stays still'. As we deploy one incarnation, the next one is in research and/or vendor discussions. EVE is ever evolving and so is its hardware.
Hmm... curiosity piqued at the thought of what potential hardware paths are being considered.
Si non adjuvas, noces (If you're not helping, you're hurting) |
Nemtar Nataal
|
Posted - 2007.07.10 16:39:00 -
[23]
Will you ever need like players to participate in running automated tests like this for testing stress and other issues related to the ammount of players logged in at the time?
|
Sevarus James
Minmatar Meridian Dynamics FREGE Alliance
|
Posted - 2007.07.10 19:47:00 -
[24]
Quote: "I saw this part of the blog, and reading between the lines I'll say thank you for this bit of information." - No problem, and Yes Linux is currently support through the Cedega game engine. But watch this space, is all I am saying.
Yep, we've been WATCHING this space since G3 announcements awhile back. heh heh.
Again, thank you for the information. :)
Ubuntu 3d Beryl-Linux Desktop+EVE |
Koronos
Interstellar eXodus R0ADKILL
|
Posted - 2007.07.10 19:50:00 -
[25]
Since it wasn't explicitly mentioned, I'll just state for the record that post- rev2 desync has become the huge lagishmonster. Here's hoping some of these automated tests can look at that specifically.
Thanks, Koronos
|
Torshin
Prophets Of a Damned Universe
|
Posted - 2007.07.10 20:21:00 -
[26]
the fact that CCP has seen fit to implement a QA team 10 years after the company was founded is a very unsettling fact. Even more upsetting is the fact that desyncing is the only reason they feel they required a dedicated team when issues have been present for years and developers and subscribers could have received much better information and turn around time on system critical issues if there were trained professionals handling things like this on a daily basis.
Backdoor Bandit - Unofficial leader of the new 'Post with your main or STFU' campaign. |
Hamfast
Gallente
|
Posted - 2007.07.10 21:15:00 -
[27]
Edited by: Hamfast on 10/07/2007 21:15:38 From your Blog...
Originally by: CCP Lingorm
Is there still more?
Yes, actually there is. As Tanis mentioned we are planning upgrades to the bug submission system with more Dev interaction into the Game Development forum in an effort to keep the information there more up to date and relevant. Extensions to our testing script system to allow us to automate the testing of a greater variety of in game actions, allowing us to automate the testing of some bugs and the confirmation of the fix, then monitor future changes as we deploy patches, thus reducing the chance of old Bugs reoccurring.
Any word on your updates from your other thread? thread
As in what ideas have you been working on making happen?
None of us is as dumb as all of us...
|
Ionia
Advanced Manufacturing
|
Posted - 2007.07.10 23:15:00 -
[28]
Quote: We are planning a trimmed down client that can have many (think 20-50) clients loaded per machine that will allow us to better simulate player load on the test servers, thus allowing better testing of code under load. Think Jita simulations and fleet battle simulations.
I know you have already said players wont be getting anything like this, but I think that position should be reconsidered. Due to the nature of EVE there are a lot of players with alts etc that would benefit hugely from an option to load a trimmed down client, because as it stands EVE is quite a resource hog.
Even if it had limited functionality such as only having in-station gameplay, it would be quite handy. Having the option of booting EVE in 2 different modes would truly be awesome.
|
Zy'or Tealon
Caldari DarkStar 1 GoonSwarm
|
Posted - 2007.07.11 06:47:00 -
[29]
Originally by: "CCP Lingorm" Following on from my mention in Tanis' blog I thought I would introduce myself and give you some more details on what I am doing here are CCP.
I am the Team Leader for the newly established QA Engineering Team.
Ok you're new. Welcome. Can you answer me this. What ever happened to Ulffr?
Ulffr
Originally by: "Ulffr | 2005.06.14 11:06:32" IĈm Ulfrr the new QA manager at CCP. I would like to use this opportunity to introduce myself and tell you about some of the forthcoming changes in QA here at CCP.
What makes you think you can accomplish what Ulffr couldn't in the 2 years he's been around?
Originally by: "Ulffr" Regression Tests & Automated Tests To minimize the probability introduction of bugs into life systems we will do the following:
* Have better organized regression tests. * Make sure developers unit test their code if they donĈt do so already. * Utilize Singularity better to find bugs and tighten up the release process. * Give players more incentive to test on Singularity. * Introduce automated client tools to perform testing. * Enhancements to the defect tracking system tools.
Originally by: "CCP Lingorm" Automated Regression Tests
One of the side effects of extending the test script system is it will allow us to automate some of the regression tests that get run against all major builds.
Originally by: Clementina
If you bug report it, you get ignored. If you post about it on the forums, you get banned. If you exploit it, you get rich.
|
Frug
Zenithal Harvest
|
Posted - 2007.07.11 11:05:00 -
[30]
Originally by: Zy'or Tealon
What makes you think you can accomplish what Ulffr couldn't in the 2 years he's been around?
He's not Ulffr. What kind of response are you expecting to get?
- - - - - - - - - Do not use dotted lines - - - - - - - If you think I'm awesome, say BOOO BOOO!! - Ductoris Neat look what I found - Kreul Hey, my marbles |
|
|
|
|
Pages: [1] 2 :: one page |
First page | Previous page | Next page | Last page |