EVE Search - Birthday Paradox? 32-bit code? Porkbelly blogs about how they relate

All Channels

EVE General Discussion

Birthday Paradox? 32-bit code? Porkbelly blogs about how they relate

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 [2] 3 4 :: one page
Author	Thread Statistics \| Show CCP posts - 5 post(s)
xeom Veto.	Posted - 2006.08.31 19:37:00 - [31] quick unrelated question kieron,sence you posted this blog today does that mean no tuxford blog? CCP where are our t2 shield power relays?
Skogen Gump	Posted - 2006.08.31 19:42:00 - [32] Originally by: Regat Kozovv I'd be hauling ass to the 7-eleven to buy my lottery ticket! 32bit hashes seem nuts today, but three years ago it probably wasn't too far fetched. Facinating dev blog CCP. Good luck with the fixes... That's very true but yeah like you say, it wasn't a daft thing to do back then
Skogen Gump	Posted - 2006.08.31 19:45:00 - [33] Lol, I just read this on Wikipedia to describe the flexibility of the GUID or UUID ... During a 10 billion year lifetime of the Earth, about 1 trillion UUIDs have to be created every nanosecond to exhaust the number of UUIDs.
John McCreedy Caldari Eve Defence Force Ascendant Frontier	Posted - 2006.08.31 19:54:00 - [34] For the love of god, some of us don't talk geek. WTF is a 32 bit hash and how does that affect my day to day playing of Eve? Make a Difference
Thiedo Mandragorian Gallente	Posted - 2006.08.31 20:00:00 - [35] Edited by: Thiedo Mandragorian on 31/08/2006 20:01:53 Originally by: Skogen Gump I feel that I need to backup that last post with some evidence ... I work for a company that makes a CRM, we use an Object orientated approach to storing data and we use our SQL database to store attributes in rows instead of columns, so we can have as few or as many values for an object as we like. Our main table has over 1.8 million rows in our biggest installation and our primary key (SQL Server 2000) is of type UniqueIdentifier and the formula for it's Identity field is ... =NewID() We've never ever seen a collision. Hey.. isn't an auto-incrementing int (or bigint, depending on need) a better choice for a unique id? From what I understand, a GUID is stored as a string. Wouldn't that completely screw you when you place the clustered index on your identity field? EDIT: Not to mention the extra record overhead storing that 36 (?) character key in various other tables as a foreign key... Thanks! Thiedo Mandragorian President/CEO Mandragorian Mining
Rahn Sohwant	Posted - 2006.08.31 20:01:00 - [36] I think 32 bits ought to be enough for anybody.
Todd Overbeck	Posted - 2006.08.31 20:02:00 - [37] Originally by: John McCreedy For the love of god, some of us don't talk geek. WTF is a 32 bit hash and how does that affect my day to day playing of Eve? It's like a fingerprint. Instead of copying the objects everywhere they are needed, you can store the fingerprint of the object and use that to look up the object when you need it. Unfortunately, if two objects have the exact same fingerprint, the system can't determine which of the two objects is the right one, and is likely to guess wrong. It's a case of mistaken identity
Thiedo Mandragorian Gallente	Posted - 2006.08.31 20:03:00 - [38] Originally by: Todd Overbeck Originally by: John McCreedy For the love of god, some of us don't talk geek. WTF is a 32 bit hash and how does that affect my day to day playing of Eve? It's like a fingerprint. Instead of copying the objects everywhere they are needed, you can store the fingerprint of the object and use that to look up the object when you need it. Unfortunately, if two objects have the exact same fingerprint, the system can't determine which of the two objects is the right one, and is likely to guess wrong. It's a case of mistaken identity OMG!! Identity theft has permeated the Eve Universe!! Thanks! Thiedo Mandragorian President/CEO Mandragorian Mining
Crunch Hardiron Caldari	Posted - 2006.08.31 20:03:00 - [39] Originally by: Rahn Sohwant I think 32 bits ought to be enough for anybody. Is it sad that not only did I get that, I laughed? ---- "Everyone who doesn't believe the sky is falling is an ass-kissing fanboi! EVE is going to fail, mark my words!" -Customers
Lina Morgai Minmatar	Posted - 2006.08.31 20:03:00 - [40] Originally by: John McCreedy For the love of god, some of us don't talk geek. WTF is a 32 bit hash and how does that affect my day to day playing of Eve? The method eve code uses to create unique identifiers is not adequate compared to how many unique identifiers eve needs. Does that clarify the issue any?
Skogen Gump	Posted - 2006.08.31 20:06:00 - [41] Originally by: John McCreedy For the love of god, some of us don't talk geek. WTF is a 32 bit hash and how does that affect my day to day playing of Eve? A hash, or hash table is a structure used in computers to organise Data that you want to be unique. Say you've got a list of unique solar systems in EVE that you need to make sure is unique, you use a mathematical function to look at the stuff in the solar system and create a special number that can only be created by that combination of information - even changing a single letter from upper-case to lower-case would change that number. So you take the data, and you store the number and you know you've got a list of unique data, the other nice thing is that you can then take that number and use it to as an index, so it makes it easy to find things - it becomes a reference number and as it can only have been generated by one set of information, you can trust as (as a computer programmer) that the data you find with that special number is the right Data. What CCP are saying is that the way they made these special numbers has found a way of making the same number from two different pieces of information. That's bad ...
Skogen Gump	Posted - 2006.08.31 20:08:00 - [42] Originally by: Crunch Hardiron Originally by: Rahn Sohwant I think 32 bits ought to be enough for anybody. Is it sad that not only did I get that, I laughed? No and No ... I laughed too
Maya Rkell Forsaken Empire	Posted - 2006.08.31 20:18:00 - [43] Originally by: Skogen Gump Originally by: Maya Rkell I'd imagine that the effect would be quite noticeable, and it's certainly checkable. GUID hash collisions are someone I've encountered before myself... Hey hey come on ... Hash collisions are one thing but Guid hash collisions are alltogether something else entirely, the fact that they're partly based (in standard implementation) on how many milliseconds since the Epoch and a computers mac-address makes the chance of getting a GUID collision extremely small. So, if you've had a Guid hash collision, then unless you're inserting the same item into your hash-table twice, that's a story you've got to tell The engine I work with calls file hashes GUID's. Never realised there was a standard definition/way of generating tbh :P //Maya
Dessa DesPlains	Posted - 2006.08.31 20:39:00 - [44] This explains how one of my alts got German 'skill finished' messages on an english client..
Altaree Gallente Red Frog Investments Daikoku Trade Syndicate	Posted - 2006.08.31 21:39:00 - [45] GREAT BLOG! I love all of this details in it. KEEP UP THE GREAT WORK!! --Altaree Fix The Web Servers
schurem Anarchy Inc. Confederation of Independent Corporations	Posted - 2006.08.31 21:48:00 - [46] So ah, does this explain why the server been crashing like mad lately, or does it explain why the patch has been postponed a couple times? <<<< No Boundaries, No Fences, Fly Free* Or Die Trying* >>>> Typhoon for the win. Nastier than whats faster, faster than whats nastier!
Thario	Posted - 2006.08.31 22:03:00 - [47] Originally by: Thiedo Mandragorian Hey.. isn't an auto-incrementing int (or bigint, depending on need) a better choice for a unique id? From what I understand, a GUID is stored as a string. Wouldn't that completely screw you when you place the clustered index on your identity field? EDIT: Not to mention the extra record overhead storing that 36 (?) character key in various other tables as a foreign key... I think you'r messing up (unique) ID of an item with its hash value. unique ID are what it says: unique number identifying the item. Hash values are typicaly used when you want to store items in a way you can easy access them (fast and without waisting to much memory). I think here we are talking about some resource file and once loaded you put it somewhere in the memory and store its hash value together with a pointer to where you can find it in the memory in an easy access hash table. that way you can later on (when you need it again) see if you already loaded it, or actualy have to go through all the trouble of loading the data you want from your harddisk (or even worse a remote server).
Shiraz Merlot Octavian Vanguard	Posted - 2006.08.31 22:49:00 - [48] Originally by: Jean Perrau Are you saying that your hash table code did not handle collisions ? Wow He didn't describe a hash table, he described an object loader. You must be some junior coder to be so *****y whilst using the wrong terminology. Equivalent of a "speling" flame. Go back to your desk.
Thiedo Mandragorian Gallente	Posted - 2006.08.31 22:57:00 - [49] Edited by: Thiedo Mandragorian on 31/08/2006 23:01:58 Originally by: Thario Originally by: Thiedo Mandragorian Originally by: Skogen Gump I feel that I need to backup that last post with some evidence ... I work for a company that makes a CRM, we use an Object orientated approach to storing data and we use our SQL database to store attributes in rows instead of columns, so we can have as few or as many values for an object as we like. Our main table has over 1.8 million rows in our biggest installation and our primary key (SQL Server 2000) is of type UniqueIdentifier and the formula for it's Identity field is ... =NewID() We've never ever seen a collision. Hey.. isn't an auto-incrementing int (or bigint, depending on need) a better choice for a unique id? From what I understand, a GUID is stored as a string. Wouldn't that completely screw you when you place the clustered index on your identity field? EDIT: Not to mention the extra record overhead storing that 36 (?) character key in various other tables as a foreign key... I think you'r messing up (unique) ID of an item with its hash value. unique ID are what it says: unique number identifying the item. Hash values are typicaly used when you want to store items in a way you can easy access them (fast and without waisting to much memory). I think here we are talking about some resource file and once loaded you put it somewhere in the memory and store its hash value together with a pointer to where you can find it in the memory in an easy access hash table. that way you can later on (when you need it again) see if you already loaded it, or actualy have to go through all the trouble of loading the data you want from your harddisk (or even worse a remote server). Hey Thario! I think you misunderstood. This was a slightly off-topic question I had for Skogen Gump specifically. He was saying that they used the MS-SQL function "NewID()" to generate GUID values for uniquely identitifing database records. I was stating that using numeric identifiers would be more efficient, both in a storage and indexing sense. Thanks! Thiedo Mandragorian President/CEO Mandragorian Mining
Lake The Praxis Initiative Interstellar Starbase Syndicate	Posted - 2006.08.31 22:57:00 - [50] Simple fix. Just reword something on the german character sheet. ducks
Skawl GeoTech	Posted - 2006.08.31 23:12:00 - [51] Originally by: FireFoxx80 Once again, math triumps over man! Math: 1 Man: 0 Surely if math triumphed once again it'd be at the very least Math: 2 Man: 0
Skogen Gump	Posted - 2006.08.31 23:17:00 - [52] Originally by: Thiedo Mandragorian Hey Thario! I think you misunderstood. This was a slightly off-topic question I had for Skogen Gump specifically. He was saying that they used the MS-SQL function "NewID()" to generate GUID values for uniquely identitifing database records. I was stating that using numeric identifiers would be more efficient, both in a storage and indexing sense. Yeah, I'd normally agree - but Microsoft are inlove with the GUID and thusly have an intrinsic data type in SQL Server called the Unique Identifier that can be used for Primary keys and Identity columns; I've not read any literature from MSDN or other sources (and trust me I don't implicitely trust the might word of redmond) that say that using a Unique Identifier as a lookup value causes any performance degredation, but I also don't know how it's stored internally for indexing purposes. Given that nothings actually a type until it's loaded and parsed, and that a guid is basically hexadecimal, it's likely to be faster then a string at anytime; the parentheses, braces and hyphens to make it easily human readable are only added when they're printed as strings.
Magnus Card Caldari	Posted - 2006.08.31 23:30:00 - [53] Originally by: Crunch Hardiron Originally by: Rahn Sohwant I think 32 bits ought to be enough for anybody. Is it sad that not only did I get that, I laughed? Nope, Bill gates reference right? 640kb...
MysticNZ Solstice Systems Development Concourse	Posted - 2006.08.31 23:49:00 - [54] Originally by: Rahn Sohwant I think 32 bits ought to be enough for anybody. Good old Bill Gates -=====- Xorus is teh nub :D
Nerfherder2000 Gallente Nanotechnologists Are Nurturing Outer Space Interstellar Starbase Syndicate	Posted - 2006.09.01 00:34:00 - [55] Originally by: Skawl Originally by: FireFoxx80 Once again, math triumps over man! Math: 1 Man: 0 Surely if math triumphed once again it'd be at the very least Math: 2 Man: 0 Even better Math: n+2 Man: n
bonesbro	Posted - 2006.09.01 01:23:00 - [56] Originally by: Skogen Gump Originally by: Thiedo Mandragorian Hey Thario! I think you misunderstood. This was a slightly off-topic question I had for Skogen Gump specifically. He was saying that they used the MS-SQL function "NewID()" to generate GUID values for uniquely identitifing database records. I was stating that using numeric identifiers would be more efficient, both in a storage and indexing sense. Yeah, I'd normally agree - but Microsoft are inlove with the GUID and thusly have an intrinsic data type in SQL Server called the Unique Identifier that can be used for Primary keys and Identity columns; I've not read any literature from MSDN or other sources (and trust me I don't implicitely trust the might word of redmond) that say that using a Unique Identifier as a lookup value causes any performance degredation, but I also don't know how it's stored internally for indexing purposes. Yes, a clustered index on a GUID is a bad idea. GUIDs are randomly generated, and a clustered index sorts things into order. Thus, inserting a new row will insert into an essentially random place in the table. This will cause significant fragmentation - in fact, you get a table that's perfectly fragmented.
Crunch Hardiron Caldari	Posted - 2006.09.01 01:27:00 - [57] Originally by: Skawl Originally by: FireFoxx80 Once again, math triumps over man! Math: 1 Man: 0 Surely if math triumphed once again it'd be at the very least Math: 2 Man: 0 Math: 2 Man: 1 Math: 2 Man: 2 Math: 2 Man: 3 Man WINS! ---- "Everyone who doesn't believe the sky is falling is an ass-kissing fanboi! EVE is going to fail, mark my words!" -Customers
Jaden Haryl	Posted - 2006.09.01 01:33:00 - [58] Originally by: Skogen Gump Yeah, I'd normally agree - but Microsoft are inlove with the GUID and thusly have an intrinsic data type in SQL Server called the Unique Identifier that can be used for Primary keys and Identity columns; I've not read any literature from MSDN or other sources (and trust me I don't implicitely trust the might word of redmond) that say that using a Unique Identifier as a lookup value causes any performance degredation, but I also don't know how it's stored internally for indexing purposes. Given that nothings actually a type until it's loaded and parsed, and that a guid is basically hexadecimal, it's likely to be faster then a string at anytime; the parentheses, braces and hyphens to make it easily human readable are only added when they're printed as strings. Well SQL server hardly cares whether its a string or not when it indexes, but the size of the key does matter, and 128bit GUID is 4 times the size of a 32bit int, and this definetly affects performance. Also for a table with a high volume of inserts sequential ints will vastly out perform GUIDs as pages do not have to split. There are some white papers out there going into great depth about the performance implications of GUIDs for PKs, heck check out comp.databases.theory and can see it beaten to death, but I wouldnt go there telling them you store attributes in rows, they will tear you apart, as it is such a common elementary mistake to use a relational DB this way. BTW any reason you guys are using a GUID on a table with only 1.8 mill rows, a 32bit int should be more than enough, or go big int, either way better performance. Maybe you guys are replicating? Then again you should just be creating new tables with different rows for different object types and not storing all objects attributes in one big table, but I digress.
jamesw Omniscient Order	Posted - 2006.09.01 02:03:00 - [59] Did someone say there was spare hash?? -- NEW Vid: Domi For the Win!
Alekto Erinys Platinum Investments	Posted - 2006.09.01 03:28:00 - [60] So, uhhh.... for a practical concern How much programming effort will this take to fix? Do you need to rewrite a ton of code? And I think I kind of understood what some of the programmers were saying ... so would it be better to use sequential numbers instead of the (randomly generated?) hashes you currently use?

Pages: 1 [2] 3 4 :: one page
First page \| Previous page \| Next page \| Last page

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.