Pages: [1] :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Max Kolonko
Worm Nation Ash Alliance
19
|
Posted - 2011.12.14 07:41:00 -
[1] - Quote
Those of us that run CRON JOBS to get API data regularly (every hour/day etc...) suffer from the fact that API cache times usually exactly X Hours.
If this can be reduced by few minutes at least, that would make scheduled scripts make less cache request and would always get newest data.
When I set schedule to run every hour and cache is one hour here is what happening:
API cache time = 12:00
12:00 - job should starts. 12:01 - job actually starts (a lot of different jobs on server make for this delay) 12:02 - job finishes it tasks - API cache time somewhere between 13:01 and 12:02
13:00 - api hits cache data 14:00 -- job should starts. 14:01 - job actually starts (a lot of different jobs on server make for this delay) 14:02 - job finishes it tasks - API cache time somewhere between 15:01 and 15:02
So even though my data should be available every hour, script only can get it every other hour.
Reducing API cahce time for all >1h APIs by lets say 5 minutes should fix all delays and script execution times. |
Lutz Major
Austriae Est Imperare Orbi Universo
54
|
Posted - 2011.12.14 08:20:00 -
[2] - Quote
Although I'm with you that some cache timers could be reduced, I must say that your way of (mis)using the cache timers is a bit strange:
You shouldn't fetch new pages unless the cachedUntil expires. That way it is regardless when the timers run out and you don't have to have dozens of different cron jobs. |
Talaan Stardrifter
Universal Exports THE UNTHINKABLES
9
|
Posted - 2011.12.14 12:07:00 -
[3] - Quote
Cron runs cache checker (every 5 minutes, at your pleasure) Cache checker finds expired caches cache checker runs cache refresher for expired caches cache refresher sets new cache expiry at: cacheUntil + 5 minutes
At least, that's how my Agent works (Windows Service, not a cron, but same difference) |
PsyKzz
Bat Country Goonswarm Federation
7
|
Posted - 2011.12.14 14:58:00 -
[4] - Quote
Why not do save the time from the xml response, save to a db.
Then check cron every 5 mins.
If cache time > now then update else ;do nothing end if
Simple.
(i hate having to write code like basic.) Meh. |
Dragonaire
Corax. PURgE Alliance
21
|
Posted - 2011.12.14 15:34:00 -
[5] - Quote
Or simply use an API library that does all of the above suggestions correctly to start with like Yapeal Finds camping stations from the inside much easier. Designer of Yapeal for Eve API. Check out the Yapeal forum thread |
Karbowiak
Sniggerdly Pandemic Legion
31
|
Posted - 2011.12.14 16:06:00 -
[6] - Quote
The way we do it with EVE-KILL, is that we store every api key in the database, attach its last error code to it and its cached till time.
And everytime it runs out we re-request it. For every single key (we have 8k) So, yeah.. lots of requests..
oh and we use Pheal, **** Yapeal :P |
Max Kolonko
Worm Nation Ash Alliance
20
|
Posted - 2011.12.14 19:24:00 -
[7] - Quote
PsyKzz wrote:Why not do save the time from the xml response, save to a db.
Then check cron every 5 mins.
If cache time > now then update else ;do nothing end if
Simple.
(i hate having to write code like basic.)
Un fortunatly i cant run cron every 5 minutes. I dont own server, so i cant set them up at my pleasure. I have available to me interface that allows them to run at most every full hour. so no matter if i check my timer (i doo, i also use pheal) i still can get data 1 hour AFTER the cache expired. |
Tonto Auri
Vhero' Multipurpose Corp
18
|
Posted - 2011.12.14 22:46:00 -
[8] - Quote
Well, it's problem with your server, not with API. Talk to your server provider, probably there's more than one way to skin your cat. |
|
|
|
Pages: [1] :: one page |
First page | Previous page | Next page | Last page |