|
|
| Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Poetic Stanziel
Fweddit
I Whip My Slaves Back and Forth
1851
   |
Posted - 2013.04.08 07:28:00 -
[1] - Quote
Karbowiak wrote:Since last post, we have created a few new things, and fixed some bugs. Any word concerning a fix of the orderDirection/asc problem? Which was causing queries to run for 9000+seconds? (Probably some sort of recursive loop it was unable to get out of until the inevitable stack overflow error.)
I'm currently snagging all the killmails back to Jan 01 2013 ... but once that is done (it will probably take another week or so), then resuming from my most recent KillMailID and working my way into the future, ensuring that I never miss a KM, requires that orderDirection/asc be functioning.
I realize you guys have lives and tonnes of daily work dealing with the killboards ... but when you can get around to it fixing that problem ... I would love you long time.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I Whip My Slaves Back and Forth
1851
|
Posted - 2013.04.09 06:35:00 -
[2] - Quote
DingoGS wrote:Could you add a option to show estimated ISK Value of the Kill? That could open up many interesting analysis to be pulled from the API :) The old EVE Kill API had that information, but the new zKill does not ... which struck me as odd. I asked about it a few weeks ago, and the response was non-committal. I'm sure they have the information in the database, since their new stats API functions give ISK values for losses and kills.
For my own project, I could start accessing the EVE-Central database, getting the mod prices myself, and then constructing ISK values of items on the KM ... but zKill doing it for me would be a time-saver.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I Whip My Slaves Back and Forth
1851
|
Posted - 2013.04.09 06:36:00 -
[3] - Quote
Squizz Caphinator wrote:Poetic, I know I told you I'd look at it last week but that was before I had a pretty big work project dumped on me. I do all of my zkb work in my free time and that was sorely lacking. I will def. try to look into that issue this week. You done fixed it ... as you notified me in an eveMail. Thanks a bunch, dude!
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1861
|
Posted - 2013.04.16 21:29:00 -
[4] - Quote
For people that might want to ease up on the API hits every 60 seconds ... have you thought about releasing a daily XML of a complete days worth of KMs (API-only)? I could download on May 10, for instance, the May 9 killmail file in XML format.
(You could also offer a JSON version for those people that prefer to work in that format.)
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1861
|
Posted - 2013.04.17 19:18:00 -
[5] - Quote
Squizz Caphinator wrote:If you aren't familiar with Stomp, check the README section on zkb's github page. I have no idea what STOMP is (other than being a PHP module), and the following description doesn't say much about its purpose or what functionality it is providing:
https://github.com/EVE-KILL/zKillboard#stomp
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1861
|
Posted - 2013.04.17 20:35:00 -
[6] - Quote
Steve Ronuken wrote:STOMP, for zkill, is providing a stream of kill mails. As they come in, they get flung out onto the stomp stream. Pretty much like what happens with EMDR. Not quite so good for filling in gaps, but handy for keeping up to date. Or if you want to see indicative data (such as: what modules are popular) It's not just PHP. There are stomp clients in many languages. http://stomp.github.io/implementations.html Thanks. That tells me a lot more.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.18 21:58:00 -
[7] - Quote
Corporate API keys for zKill are better than individual API keys.
So that we can encourage more people to supply corporate API keys to zKill, why not supply a page that is searchable, so that we can see if corps have supplied a corp-level API key to zKill/EVEKill? Before somebody posts a personal key, they can check to see if there's already corporate keys in place.
***
Since the corporate API only pulls the last 100 kills ... at what point does it become difficult to ensure that all killmails are grabbed? For a corporation on a particularly busy night, it could be possible to miss killmails, because over 100 kills occurred between API grab A and API grab B?
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.19 08:40:00 -
[8] - Quote
Karbowiak wrote:Since the corporate API only pulls the last 100 kills ... at what point does it become difficult to ensure that all killmails are grabbed? For a corporation on a particularly busy night, it could be possible to miss killmails, because over 100 kills occurred between API grab A and API grab B?
Currently you can only lookup corporate API keys for corps in an alliance.
But good idea. please drop it into: https://github.com/EVE-KILL/zKillboard/issues with a good description.
The solution to the issue of only getting the last 100 kills, you just supplement by either adding more corporate keys (The more keys there are, the more often zKB will fetch for said corp. Say there is 5 keys, it'll fetch every 12 minutes)[/quote]Are you reaching any hardware limitations, though? If you have 100,000 API keys, are you reaching a point where it is becoming difficult to pull all those keys in a single hour?
How many API keys do you guys track?
Do you have a method of prioritizing the keys? Perhaps based on past activity? Keys that generate more activity are given a high priority?
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.19 09:52:00 -
[9] - Quote
Strange problem here:
Check this link -> http://poeticstanziel.blogspot.ca/2013/04/top-100-most-active-corporations-march.html
Second row, check the Red Fed and Blue Rep values for kills and losses.
For some reason my numbers are only half of what's being reported on EVE-Kill and zKillboard. Yet, my numbers for every other corp I've checked are (more or less) on the ball.
This continues for February too. My numbers for Red Fed and Blue Rep are approx. half of what's being reported on the EVE-Kill/ZKill websites.
I'm wondering, if perhaps, the problem is with EVE-Kill/zKill? Perhaps their numbers are being doubled up for some reason? Perhaps there's something else happening, that I'm not aware of.
I can't imagine I'm missing KMs just for the RvB corps ... especially considering I pull my KMs by region.
I'm kinda baffled.
Thanks.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.19 21:00:00 -
[10] - Quote
Karbowiak wrote:Probably some sort of issue with the fetching on your end, or maybe it's cause a lot of their mails are manually posted, dunno :) Are a lot of RvB mails manually posted (not API verified)? Half of them? (Because I am pulling API-only KMs.)
I wouldn't begin to know how to solve the issue, if it is a fetching problem. But why would my fetching problem only be affecting RvB?
Could you run a query for me? Tell me how many of their March 2013 lossmails and killmails are manually posted? And perhaps tell me the KillID (the KillID that CCP uses) range for March 2013 (starting KillID and ending KillID for that month). Maybe I can pinpoint the problem.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
|
|
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.20 03:09:00 -
[11] - Quote
Could Squizz or Karb read the following and set me straight on any errors I might have made?
http://poeticstanziel.blogspot.ca/2013/04/your-api-keys-killmails-and-you.html
Thanks.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.20 07:03:00 -
[12] - Quote
Karbowiak wrote:And as for a public kill feed, zKB will try and fill that gap as best possible both by the use of it's API, and with the use of the STOMP stuff  You keep talking STOMP, and also keep saying you might not use it in the future. So I'll ignore writing any code specific to it, until I know you guys are going to stick with it. Could you describe the benefit of using STOMP tho, versus the traditional API calls to zKill? I think I kind of understand what STOMP is doing, but not quite sure what the benefit of it is over just doing what I'm doing now.
In terms of your API ... let's say I post a manual lossmail to zKill/EVEKill, which I often do. How long, maximum, before that manual lossmail (with the negative ID) is transformed into an API-verified killmail (with a positive ID)? I'm guessing no more than two hours, basically the length of time it will take you to request my kills/losses using the API key I have registered with you folks.
Check the following API call (I don't use this in my code, I was using it to check out some data associated with manual killmails):
https://zkillboard.com/api/xml/no-items/corporationID/1699307293,1741770561/beforeKillID/29598374/page/40/
In the call, all I do is ask for any killmail before ID 29598374 and 40 pages back. Now, there are a lot of manual killmails here, all with the expected negative IDs. Your query obviously isn't just using a "WHERE ID < 29598374" clause, otherwise all those manual IDs wouldn't show up at all. Are you creating a dummy ID (in the expected range) for the manual kills, or are you transforming the "WHERE ID < someID" into a "WHERE date < someDate" filter? Curious.
Also ... would it be possible to add in a /manual-only/ parameter to the API calls, much as you have an /api-only/ parameter? In case I do want to add manual kills in the future, I can filter them specifically, rather than end up grabbing a bunch of api-only data that I already have.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I whip my slaves back and forth
1869
|
Posted - 2013.04.20 07:35:00 -
[13] - Quote
Karbowiak wrote:You set it to beforeKillID, meaning anything before 26598374 (meaning both negative and positive upto 26598374) - also you didn't add /api-only/ - thats why it shows manual kills aswell :) I know that.
I'm wondering why the manual kills are appearing in date sequence order that ... never mind ... you're probably just doing an ORDER BY killDate on all the queries by default.
killIDs (API verified only) are guaranteed to be in sequential order? (I should have asked this weeks ago, but assumed it was the case.) A killID on Feb 01 2013 will always be less than a killID on Feb 02 2013, as an example?
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I Whip My Slaves Back and Forth
1869
|
Posted - 2013.04.20 20:36:00 -
[14] - Quote
Squizz Caphinator wrote:Poetic, yes, when you specify before a killID it is converted into a date. All CCP killiDs are sequential in order and date, so killID 123 happened before killID 321. This does NOT apply to manual killID's, which are purely random because we cannot allocate an ID for a kill we don't know about (obviously).
If you haven't already, create an issue for a manual-only filter on the API and I'll get it done Monday for you. HOWEVER! Any oddities that come from the API involving only manual kills will just be ignored. You have to deal with those yourself :)
( I never wanted manual kills, I still don't, and the less I have to do with them the happier I am ) I'll wait until after Fanfest. There may be news on the API via the devtracks. Seems PrismX wants to redo the Kill Log API.
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I Whip My Slaves Back and Forth
1869
|
Posted - 2013.04.24 03:39:00 -
[15] - Quote
I think one of the problems with a lot of the RvB killmails not being API verified is that many times they are recording greater than 100 kills+losses per hour. Which means killmails that cannot be retrieved (unless you're lucky enough to have other keys which can capture those "lost" killmails.)
I've been talking with Mangala Solaris and have convinced him to register more RvB keys for each CEO/Director they have.
Now, if those keys used within a couple minutes of each other, the problem is not really being solved.
I'm wondering if our code spaces multiple keys from a single corporation with an appropriate timeframe? For instance, let's say RedFed has three keys, are those three keys being pulled twenty minutes apart from each other?
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
Poetic Stanziel
Fweddit
I Whip My Slaves Back and Forth
1869
|
Posted - 2013.04.24 03:42:00 -
[16] - Quote
Karbowiak wrote:So apparently the TEST devs have switched to zKB.. https://zkb.pleaseignore.com/Incase you guys look here, please do send code fixes back incase you fix something Does this mean that they are now registering their API kill log keys with their own personal board, which means you guys won't be able to pull most of the TEST killmails?
Amarr Militia - Fweddit - http://fweddit.com
Poetic Discourse - http://poeticstanziel.blogspot.com |
|
|
| |
|