Hello again,
Thanks for the answer, do you mind if I follow up with a few other questions?
a) If/When you decide to change the way the API works, is it going to break all previous applications? IE, should I wait for the change to happen or is it not in your immediate plans?

b) If I need to grab a large amount of data (all kills for a number of regions in the past 2 weeks for example), what would be the best way to approach it?
Try to get as much data as I can in each request that happens every "x-bin-seconds-between-requests" seconds or get a couple of instances of an application that requests smaller pieces of data every "x-bin-seconds-between-requests" seconds?

Is there any other way to obtain this amount of data? Do you release any dumps?

Thanks again.

Hi guys,

Have you made any progress on API limits or API keys? The current limits are quite tight and hard to work with for realtime apps. Could you provide daily dumps of kills, or some kind of bulk access? This would solve most of my own access problems (I could cache the kills the day before and not have to hammer your server). Many thanks for the great tool.

However, the 2nd version of the API, will ditch the requests pr. second idea, and instead impose a flat request amount, so you default get 5k requests pr. day, which is 1 request every 17.2 seconds, and you'll be in control of how fast you want to request stuff.

We can obv. add more to it on a case by case basis.

5k requests as in max 1m killmails per day (200*5000) or 5k killmails per day?

This would work well for my app. The app I'm working on is a quick kill history checker (get a name, get all kill history, do some searching). If a user gives me a name which has 30 pages of kill history they currently have to wait something like 30 seconds for the app to fetch it all. Under the new API the app can grab those pages in a few seconds then sit inactive for the rest of the day (i.e. the average requests per day will be tiny, but the app will request in bursts of about 30 requests).

If you need any beta testers for the new API let me know :)

I tried to use apiv2 earlier and got an "invalid hash" error. The hash I used came straight from my account page. I guess it's still work in progress though :)

Some more fixes to the API were done last night. This includes removing apiv2 (sorry karbo, but it had issues), and a severe fix to the rate limiting we put on folks using the API. While working on the API I noticed someone hit the API 98 times in 1 second... they had figured out a workaround, and I went ahead and fixed that.

This is unfortunate. When will APIv2 be returning? This is a killer-feature as far as I'm concerned.. opening up more killmail data to the wider community can only lead to good things.

This is unfortunate. When will APIv2 be returning? This is a killer-feature as far as I'm concerned.. opening up more killmail data to the wider community can only lead to good things.

All the data is open, what makes you think it isn't?

I agree it's open, and that's great. I like the idea of apiv2 because of the more flexible request limits. The 10 second timer in apiv1 is quite hard to work with. I know you're still working on these things though :)

Ah... well... apiv2 was going to limit you to about 5k requests per 24 hour period, api v1 gives you 8,640 requests per 24 hour period. So you're actually better off

For interactive purposes it's much worse if more than one page needs to be requested (the user has to wait 10 seconds). For bulk data gathering apiv1 is obviously better as you said.

Do you have a policy on people trying to download all killmails ever? Do you try and stop them or do you permit it as long as they stick to request limits?

Squizz, if I want to make incremental pulls from the database using the AfterKillID to get only kills since my last query is it guaranteed I'll get all killmails (IOW is that a sequential value zkb creates as it parses killmails) or do I risk missing some if they're inserted out of order (IOW is that a CCP-created value that could be out-of-order ie api for killmail #100 is pulled before api for killmail #101)?

Thank you!

I've modified the code to allow 8,640 requests per day at any rate you want. Same limits as before, just not as limiting, heh...

If someone wants to download all our killmails via the API they are more than welcome to do so, we have no policy for or against it. Its all there for anyone to use, we believe in being open about it. However, we aren't going to provide a dump of the killmails, they'll have to use the API. At 1.6 million mails per day it'll take them a couple of weeks, they'll just have to be patient :)

And just to keep everyone updated on what I'm doing - I'm currently evaluating ways to reduce the size of the database. I just removed a 22 gig table and replaced it with a 1 gig table. I'm also looking into how to reduce the size of a 100+ gig table to possibly 30 gigs.

Would be awesome to know what you have done and how did you reduce 22 GB to just 1 GB? Is it possible to get some details about that (private | public - doesn't matter)?

He removed the zz_items table, and used the item data already stored in the raw json files that we store.

As for the big table, it's just indexes that'll be removed, maybe a row or two of data that isn't used that much. :)

Im missing the "all time" top history from players/corps
Its interesting to have it montly but add the option to see the "all time" too!


Edit: Ranks arent updating too.