
CCP Garthagk

|
Posted - 2007.10.22 12:18:00 -
[4]
The magic number is the number from this particular incident - next time the number will be something different. It's not static.
Anyway, for a long term solution you could look into doing something like having generations of data, i.e., right now we're in generation 2, that last batch (last 6 months) was generation 1... then have your own primary key.
For example, a journal storage table could be:
CREATE TABLE journal_entries ( localID INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY, ccpGeneration SMALLINT UNSIGNED NOT NULL, ccpRefID INT UNSIGNED NOT NULL, ownerID INT UNSIGNED NOT NULL, /* char OR corp ID */ ...other...
UNIQUE ccpID (ccpGeneration, ccpRefID) )
Pseudo-MySQL, but you get the idea. You would have to keep track of what generation you're in currently, and then when storing the data do a check against the refID and see if they have suddenly changed. For example, here's some pseudo-code for downloading someone's journal:
// setup our variables, this is what we think the current universe is! $currentGeneration = 1; $characterID = 2723742347;
// now get the highest refID we have on file for this user! // and any pedants who want to gripe about not using bound parameters can bite me, // this is pseudo-code only! $latestReferenceID = LocalDatabaseRunSQL( SELECT MAX(ccpRefID) FROM journal_entries WHERE ownerID = $characterID AND ccpGeneration = $currentGeneration );
// now hit the API for this character's journal page, we're assuming // here that there are no errors... really you should always be checking // for errors :-) $newData = API.GetJournalForCharacter( $characterID );
// iterate over each row, find the highest ID in this dump $newMaxRefID = 0; for each $row in $newData { if ( $row->refID > $newMaxRefID ) $newMaxRefID = $row->refID; }
// now, if the highest ID we found is below the highest ID we've already // got, then we can safely assume that the generation has changed. this is // because we should never get back less data than we already have! if ( $newMaxRefID < $latestReferenceID ) { // ... } else { // just store the row in the database, everything's cool }
Long-winded, but you get the idea. You would store our reference IDs only as a way of ensuring no duplicate transactions, but use your own generated IDs internally for your database.
You would then store a table of (ccpGeneration, refIdDecrementValue) so you can properly map between two generations. In the case that the generation has changed, you need to get the new difference ID (which you can calculate manually by comparing two transactions or ask around for). With the new ID, you can compare if a transaction has already been seen by looking at the previous generation - decrementor and seeing if it matches the one you've gotten.
At any rate, this is a suggestion, there might be easier ways of doing this - in fact, I'm sure there probably is? This seems a little longwinded and troublesome. I'd love to hear other ways people are addressing this problem. 
-------------- Garthagk - EVE Software Group - EVE API Guy
To err is human to really ***** up takes admins. -Hellmar |