Pages: [1] :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Lost Hamster
Hamster Holding Corp
|
Posted - 2010.01.09 21:23:00 -
[1]
I don't know how many of you have noticed but the refID value is over: 2 147 483 647 which is the max value what we can process with int32. There are some other types like, unsigned int, then we can process up to refID 4 294 967 295, in the 32 bit world. But what then?
CCP would you mind to degrees the refID to a smaller number? Like you have done in the past? http://www.eveonline.com/ingameboard.asp?a=topic&threadID=936369
|
Vessper
SI Radio
|
Posted - 2010.01.09 23:20:00 -
[2]
Why not just use a 64-bit integer? You don't have to have a 64-bit PC or OS to use them you know. You may be interested in the following link, particularly the part explaining "long" and "ulong":
http://msdn.microsoft.com/en-us/library/exx3b86w.aspx
EveHQ Character App |
Lost Hamster
Hamster Holding Corp
|
Posted - 2010.01.09 23:36:00 -
[3]
Edited by: Lost Hamster on 09/01/2010 23:37:28 Sorry, but I'm not a programer, but I think to be able to use a 64bit integer, you need a 64bit system. At least on the hardware level.
But could be that I'm wrong.
|
Matthew
Caldari BloodStar Technologies
|
Posted - 2010.01.09 23:37:00 -
[4]
They rolled it back before because at that time refID was an int on the server. Last roll was at least a year ago though. Looks like they decided to upgrade the field to bigint, so I doubt they are going to do any more rolls of the refID values now.
This does cause a bit of a problem for quite a few applications though, as only "proper" database applications have implemented bigint at the moment. I have looked into workarounds for Excel and Access (as those are what I use), and have found the following:
Excel
Excel stores its numbers as doubles anyway, which gives you 15 significant figures, so you should not see any immediate problems. However, in theory refID will eventually overflow this.
When it does, excel will only store the most significant 15 digits and you will be unable to work with refID in Excel. At that point, you can persuade excel to store the full value as text, but you lose the ability to manipulate it as a number.
However, considering that the maximum integer value you could fit into excel is 999,999,999,999,999, we are currently at approx 2,147,483,647, and refID is currently growing at approx 140million per month, it would take 595,236 years before you hit an overflow problem.
Access
Access does not support bigint. While you could use the same hack as Excel of storing it as double, this stores an integer in a floating point datatype, which is not good practice. In Access, you have another option, the Decimal data type. A Decimal(19,0) should be big enough to avoid any overflows from a bigint refID. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Matthew
Caldari BloodStar Technologies
|
Posted - 2010.01.09 23:43:00 -
[5]
Originally by: Lost Hamster Edited by: Lost Hamster on 09/01/2010 23:37:28 Sorry, but I'm not a programer, but I think to be able to use a 64bit integer, you need a 64bit system. At least on the hardware level.
But could be that I'm wrong.
You can use 64-bit (and even larger) data types on a 32-bit system. You just suffer a bit of a performance hit when doing so. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Dragonaire
Caldari Corax. New Eden Retail Federation
|
Posted - 2010.01.10 08:07:00 -
[6]
Bigger problem can be that several programming languages don't have any built in support for 64 bit integers yet (Yes I'm looking at you PHP and others) which I worked around in Yapeal by never treat them as integers but always as strings in PHP then let the DB do the conversions as needed. This is where weak typing languages do let you do some simple workarounds that aren't always available with strong typing ones
Trying to do anything dealing with the API in either Access or Excel, which are really just glorified spreadsheets, amounts to cruel and unusual punishment for a programmer IMHO to start with. -- Finds camping stations from the inside much easier. Designer of Yapeal for Eve API.
|
Azrel
Kyoha Shinto
|
Posted - 2010.01.10 11:50:00 -
[7]
Originally by: Lost Hamster Edited by: Lost Hamster on 09/01/2010 23:37:28 Sorry, but I'm not a programer, but I think to be able to use a 64bit integer, you need a 64bit system. At least on the hardware level.
But could be that I'm wrong.
Actually this is not true most programming languages these days provide a 64-bit integer data type, internally they will handle this in a number of ways depending on the platform. In fact it is possible to have integers of any size if you write the code to handle it. For instance most public key crypto systems use integer values in excess of 1024-bits and there are many libraries available to support this.
Most modern database engines will also support large integers too, I don't count Access as a modern database, because it isn't.. Any developer in their right mind shouldn't be using Access.
|
Femaref
Armageddon Day
|
Posted - 2010.01.10 16:43:00 -
[8]
any .net developer here that could give me some advise concerning object.GetHashcode? I used the refID directly (for dicts and the like), but now I need a long, while Gethashcode returns an int. any advise?
|
Ki Tarra
Ki Tech Industries
|
Posted - 2010.01.10 22:05:00 -
[9]
Originally by: Femaref any .net developer here that could give me some advise concerning object.GetHashcode? I used the refID directly (for dicts and the like), but now I need a long, while Gethashcode returns an int. any advise?
Have you thought of calling the GetHashcode function of the bigint?
|
Femaref
Armageddon Day
|
Posted - 2010.01.10 22:57:00 -
[10]
Originally by: Ki Tarra
Originally by: Femaref any .net developer here that could give me some advise concerning object.GetHashcode? I used the refID directly (for dicts and the like), but now I need a long, while Gethashcode returns an int. any advise?
Have you thought of calling the GetHashcode function of the bigint?
funnyly, my work around had the exact same output. case closed.
|
|
Cesar Malari
Dark-Rising IT Alliance
|
Posted - 2010.01.11 05:52:00 -
[11]
I just noticed this issue this evening with a custom app I had written (I'd been ignoring the warnings about overflow for a few days), and changed everything over to use an int64.
However, I now started to get errors when the beforeRefId value I send to the server to page back in time was over 2^31. Is anyone else experiencing this too, or is this my bug? I'm getting this for a wallet that has a *lot* of transactions/day, so 1000 transactions doesn't go back very far.
I have to wait an hour for the API to let me request the initial page again, so I figured I'd ask here while I wait. |
Cesar Malari
Dark-Rising IT Alliance
|
Posted - 2010.01.11 06:45:00 -
[12]
Originally by: Cesar Malari I just noticed this issue this evening with a custom app I had written (I'd been ignoring the warnings about overflow for a few days), and changed everything over to use an int64.
However, I now started to get errors when the beforeRefId value I send to the server to page back in time was over 2^31. Is anyone else experiencing this too, or is this my bug? I'm getting this for a wallet that has a *lot* of transactions/day, so 1000 transactions doesn't go back very far.
I have to wait an hour for the API to let me request the initial page again, so I figured I'd ask here while I wait.
Ok, waited an hour, and it looks like the request is being sent right, the server just doesn't like a beforeRefId > 2^31. |
Ambo
I've Got Nothing
|
Posted - 2010.01.11 12:09:00 -
[13]
I assume they've also update the transaction ID field to 64 bit. I'm wondering though if anything else has changed that might matter? e.g. character ID could now be 64 bit but it's unlikely to ever matter. --------------------------------------
|
|
Chribba
Otherworld Enterprises Otherworld Empire
|
Posted - 2010.01.11 12:48:00 -
[14]
MSSQL (and maybe others) have 'bigint' to use even on the non-64bit versions (which is int64 anyway).
Secure 3rd party service |
|
Matthew
Caldari BloodStar Technologies
|
Posted - 2010.01.12 18:58:00 -
[15]
Originally by: Ambo I assume they've also update the transaction ID field to 64 bit. I'm wondering though if anything else has changed that might matter? e.g. character ID could now be 64 bit but it's unlikely to ever matter.
I'd be surprised if they have, at least at this point. transactionID is nowhere near overflowing an int yet, and is not growing fast enough to mean it would anytime soon (i.e we're talking in terms of being years away). So there wouldn't really be any point taking the performance and storage hit of making it a bigint at this point.
Though it would probably be a safe assumption that at whatever point such ID's do start approaching the limits of int that they will end up being upgraded to bigint rather than the old method of renumbering everything. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Tallasul Matar
|
Posted - 2010.01.13 11:08:00 -
[16]
Hi
I have some problems with the API - WalletJournal. Can anyone confirm that the beforeRefID is broken until now?
Easy Test: call the WalletJournal ... grab 1000 entrys call the WalletJournal again ... error 102 ... With needed beforeRefID cal the WalletJournal with beforeRefID
Error 107: Invalid beforRefID??
Has someone a working Journal System now in his App?
tx
|
Hel O'Ween
Men On A Mission
|
Posted - 2010.01.13 12:03:00 -
[17]
Originally by: Tallasul Matar Can anyone confirm that the beforeRefID is broken until now?
Works for me.
Quote:
Easy Test: call the WalletJournal ... grab 1000 entrys call the WalletJournal again ... error 102 ... With needed beforeRefID cal the WalletJournal with beforeRefID
Error 107: Invalid beforRefID??
Has someone a working Journal System now in his App?
I do it like - First request with beforeRefID=0 - Get all that's there, remember the last refID - Do:If more than 1000 returned call again with beforeRefID=<lastRefID>:Loop until Wallet Journal exhausted
Make sure you're trimming away spaces and such off of lastRefID. tx
-- EVEWalletAware - an offline wallet manager |
Matthew
Caldari BloodStar Technologies
|
Posted - 2010.01.13 19:35:00 -
[18]
Originally by: Tallasul Matar
I have some problems with the API - WalletJournal. Can anyone confirm that the beforeRefID is broken until now?
It looks like beforeRefID isn't accepting values greater than 2^31 (i.e. it is limited to a 32-bit integer). Unfortunately, refID has now exceeded this value. It looks like they made sure the data the API itself returns would handle this, but forgot to change the data type of beforeRefID.
That's why you're getting error 107 (effectively saying it does not recognize your input as a number) rather than just error 102 (which just indicates you have just given it the wrong number). ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Tallasul Matar
|
Posted - 2010.01.13 22:38:00 -
[19]
Edited by: Tallasul Matar on 13/01/2010 22:43:29
Originally by: Hel O'Ween
Works for me.
[qute] I do it like - First request with beforeRefID=0 - Get all that's there, remember the last refID - Do:If more than 1000 returned call again with beforeRefID=<lastRefID>:Loop until Wallet Journal exhausted
Make sure you're trimming away spaces and such off of lastRefID. tx
Hi Hel.. Tx for your response. Yes in my App i do the same thing like you. But for a quick test i check it with a webbrowser to. So there is no trimming and the lastRefID is always the last entry in the responded dataset. So nearly nothing what you can do wrong.
My App uses bigger datatypes anyway (just in case) so i had nothing to fix for the change to bigInt. The only Problem i have is this response with &beforeRefID=<lastRefID> and i have no idear what else i can do wrong.
Can you definitly say that your app is working over 1000 entrys? Would be great if could have a closer look to your steps and maybe there is something i didnt see.
@matthew: yes that was my guess to. But if its work for Hel, maybe it is something other.
Sorry my english is very crummy :(
Tx |
Matthew
Caldari BloodStar Technologies
|
Posted - 2010.01.14 12:55:00 -
[20]
Originally by: Tallasul Matar @matthew: yes that was my guess to. But if its work for Hel, maybe it is something other.
It's entirely possible that Hel hasn't seen this issue yet simply because his journal is not moving fast enough in terms of new records.
If Hel has had less than 1000 new journal entries since refID exceeded 2^31, but more than 1000 in the last 7 days, he will have a second page of data to retrieve, and his value of beforeRefID will still be less than 2^31, so he will be able to retrieve his second page of data successfully.
In theory, some characters will still be able to retrieve all their data via jornal walking until some point over the coming weekend. At that point their beforeRefID must exceed 2^31, otherwise they would not have had sufficient records in the last 7 days to require a second page of data to begin with. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
|
Hel O'Ween
Men On A Mission
|
Posted - 2010.01.14 15:16:00 -
[21]
Originally by: Matthew
It's entirely possible that Hel hasn't seen this issue yet simply because his journal is not moving fast enough in terms of new records.
This is true for myself. However, no other user of EWA has reported something like this (yet). -- EVEWalletAware - an offline wallet manager |
|
|
|
Pages: [1] :: one page |
First page | Previous page | Next page | Last page |