Pages: [1] :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Jimboga
Did I just do that Test Alliance Please Ignore
|
Posted - 2011.02.03 10:47:00 -
[1]
Hi all, I was wondering if you could shed some light on a problem I'm having. I've been writing a web based application using ALE and it's been working really nicely, except for one problem : I can only get API data for over the last 24 - 48 hours, so it's 10:42am here on the 3rd Feb and the last data I'm seeing here from the journal is from;
<row date="2011-02-01 21:40:00" ...
and that's using http://api.eve-online.com/corp/WalletJournal.xml.aspx directly whilst passing my userID, apiKey, characterID and rowCount = 256. I assumed that the API would list data as it happened too but instead I'm getting entries from the 29th of Feb, then the 27th on the next row, then the 29th on the next row.
Any ideas?
Thanks
|
Azazel Mordred
Minmatar Cloak of Shadows
|
Posted - 2011.02.03 12:52:00 -
[2]
The journal has become a bit of a mess since the last update for Incursion.
You're going to have to call the journal for the first 256 rows, find the lowest ID in the jumble you got back, then make another journal call for the next 256 starting from the lowest ID found. Then just keep repeating that until you get less than 256 records.
Don't forget to sort the whole thing by date, descending, once done, to get it back to the "normal" way it was before the update .
|
Catari Taga
Centre Of Attention Middle of Nowhere
|
Posted - 2011.02.03 12:57:00 -
[3]
Originally by: Azazel Mordred Don't forget to sort the whole thing by date, descending, once done, to get it back to the "normal" way it was before the update .
Oh, so that's what's happening. --
|
Jimboga
Did I just do that Test Alliance Please Ignore
|
Posted - 2011.02.03 13:08:00 -
[4]
Thanks for your reply Azael,
so what I've done to resolve the date issue is I store all of my journal calls to a local SQL database, I then use the fromID which is generated from the last ID of the initial call. When spitting this back to a page, I arrange by date and refID and that seem's to have fixed it. The problem wasn't that though, simply, the data wasn't there ... until I made this post?!
Literally after an hour of this post, 3 days worth of data flooded into my database. CCP, what are you getting up to?
|
Matthew
Caldari BloodStar Technologies
|
Posted - 2011.02.03 14:27:00 -
[5]
Would that 3 days worth of data correspond to around 256 records?
I'm seeing an effect whereby it only pulls 256 new records per caching period. So if my journal is filling with new entries faster than that, it starts to fall behind.
e.g. if I haven't queried the API for a day, and had 1000 new journal records in that day, when I submit the first API call, I only get 256 new records, filling forward from the last record the API had, so the API looks as if it is out of date. If I wait for the CachedUntil time to expire then query again, it will give me the next 256 new records.
So I'm having to ensure I hit the API in enough different caching periods each day to ensure that I don't fall further and further behind. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Zeta Zhul
Caldari Preemptive Paranoia
|
Posted - 2011.02.03 15:36:00 -
[6]
Seriously.
Of all the things to make changes to was there really such a need to make unnecessary changes to the API? Because my wallet journal has suddenly become a major pain in ass. Fortunately I don't worry overly much over taxes, broker fees and such but having gaps in my data irritates me.
|
Iece Quaan
Caldari Dreddit Test Alliance Please Ignore
|
Posted - 2011.02.03 20:05:00 -
[7]
The problem i'm seeing ( and im new to this ) is that the query is only returning ~about the last 12 hours of journal entries, or about 900 lines.
wat?
|
Jimboga
Did I just do that Test Alliance Please Ignore
|
Posted - 2011.02.04 09:42:00 -
[8]
Edited by: Jimboga on 04/02/2011 09:45:46
Iece Quaan - great to see another TESTie on here! If you're only getting the last 12 hours are you sure you're stepping back properly with your fromID passing?
Matthew - those 3 days worth of data was actually 1214 records!?! I'm with you on this one, it's bloody annoying, if I don't call the API at the right times the data goes further and fruther out of date. So now the last entry I can see is 2011-02-03 17:14:00 and yet my wallet from the AccountBalance function shows my wallet as a lot richer than the last 'balance' entry in my WalletJournal does, something is seriously wrong with the WalletJournal. CCP - any help?
|
Matthew
Caldari BloodStar Technologies
|
Posted - 2011.02.04 12:41:00 -
[9]
I seem to have had a big catch-up in data in the queries I ran last night, so presumably something got poked. Will have to wait to see what I get out tonight as to whether this signals an actual fix or if someone just manually poked the cache to temporarily mitigate the issue.
Given the behaviour I'm seeing, and as the journal entries all exist in-game, I'm assuming that there is something wrong with the logic that maintains the results cache on the API server. This would explain how various gaps can appear in the data, which can suddenly fix themselves in a later query. It might also explain why the result set appears to be only partially sorted. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Iece Quaan
Caldari Dreddit Test Alliance Please Ignore
|
Posted - 2011.02.04 15:18:00 -
[10]
Jim- Yes, the logic for stepping back through IDs is working correctly in my script- but it won't step back farther than the timestamp 07:21:00 on the third of February. No idea why.
|
|
Neutrino Sunset
Bene Gesserit ChapterHouse Sanctuary Pact
|
Posted - 2011.02.04 15:18:00 -
[11]
Originally by: Iece Quaan The problem i'm seeing ( and im new to this ) is that the query is only returning ~about the last 12 hours of journal entries, or about 900 lines.
wat?
This is what I'm getting too, and it isn't because I'm not walking the data properly since the server is only returning 110 rows of unsorted garbage data despite the fact that I'm asking for 256.
It's great being used to alpha test someone elses code isn't it! Almost like being at work while I'm at home ffs.
|
Iece Quaan
Caldari Dreddit Test Alliance Please Ignore
|
Posted - 2011.02.04 16:20:00 -
[12]
It almost seems like it started a cache based on when I first starting pulling data. The date and time at which the history stops has been consistent since I started, although I'm getting more data as time goes on because it is letting me pull everything newer.
|
Matthew
Caldari BloodStar Technologies
|
Posted - 2011.02.04 23:18:00 -
[13]
Well, it appears that the catch-up yesterday was a one-off hit. It's started falling behind again today.
Hopefully a more permanent fix is in the works. ------- There is no magic Wand of Fixing, and it is not powered by forum whines. |
Jimboga
Did I just do that Test Alliance Please Ignore
|
Posted - 2011.02.08 10:29:00 -
[14]
I've now had no data since 2011-02-03 18:14:00 and yet there are over 40 pages of entries in the corporate journal since then. This is rediculous.
|
neoe77
Amarr
|
Posted - 2011.02.20 03:01:00 -
[15]
Same delay problem is found in AssetList API. 10 hours passed, some new asset is still not show in the results. --------- check path using eve-path-find |
Catari Taga
Centre Of Attention Middle of Nowhere
|
Posted - 2011.02.20 03:16:00 -
[16]
Originally by: neoe77 Same delay problem is found in AssetList API. 10 hours passed, some new asset is still not show in the results.
cachedUntil wants to have a word with you... --
|
neoe77
Amarr
|
Posted - 2011.02.20 03:32:00 -
[17]
Originally by: Catari Taga
Originally by: neoe77 Same delay problem is found in AssetList API. 10 hours passed, some new asset is still not show in the results.
cachedUntil wants to have a word with you...
cachedUntil in the xml shows there is still 5 hours to wait. how could the cache be so long? my God. --------- check path using eve-path-find |
Johnathan Roark
Caldari The Graduates Morsus Mihi
|
Posted - 2011.02.20 09:41:00 -
[18]
Originally by: neoe77
Originally by: Catari Taga
Originally by: neoe77 Same delay problem is found in AssetList API. 10 hours passed, some new asset is still not show in the results.
cachedUntil wants to have a word with you...
cachedUntil in the xml shows there is still 5 hours to wait. how could the cache be so long? my God.
It should be 1 day, assets can be a huge call.
POS-Tracker 3.0 Hosting |
|
|
|
Pages: [1] :: one page |
First page | Previous page | Next page | Last page |