Pages: [1] :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Vaerah Vahrokha
Vahrokh Consulting
4263
|
Posted - 2013.08.20 12:49:00 -
[1] - Quote
Hello,
I am here to ask the knowledgeable guys lurking in this forum for a suggestion.
I need to attach to an EMDR datafeed.
I am interested only in the data for 1 market in The Forge. So far nothing special.
I need to access the exact price of the lowest current sell order that is currently active for that market.
This process of extracting the current lowest sell order price needs to be repeated at every new order. Every hour the stats are reset.
Is the logic just about checking something like:
if (newly_arrived_order_active && newly_arrived_order_type_is_sell) { add_to_stats(newly_arrived_order_price); }
Because somehow I feel EvE's orders system is more complex than this.
Thanks in advance. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Ydnari
Estrale Frontiers Project Wildfire
222
|
Posted - 2013.08.20 18:33:00 -
[2] - Quote
You'll also want to check (at least) the generatedAt attribute of the EDMR message as messages may arrive way out of order, not least because someone's instance of EVEMon may upload an old cache file because it didn't happen to be running at the time it was generated.
Other than that, pretty much that simple.
You can't request the EMDR datafeed is filtered at source so you get the entire firehose and you have to filter it your side, but that's probably not a problem. -- |
Vaerah Vahrokha
Vahrokh Consulting
4263
|
Posted - 2013.08.20 20:18:00 -
[3] - Quote
Ydnari wrote:You'll also want to check (at least) the generatedAt attribute of the EDMR message as messages may arrive way out of order, not least because someone's instance of EVEMon may upload an old cache file because it didn't happen to be running at the time it was generated.
Other than that, pretty much that simple.
You can't request the EMDR datafeed is filtered at source so you get the entire firehose and you have to filter it your side, but that's probably not a problem.
Thank you so much!
To recap, I am going to accept only orders with active flag + marked as sell order plus date >= the date of my last received order, which in turn must fit within the 1 hour "chunks" I am splitting the data into. As for the "firehose", I'll deal with it. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Steve Ronuken
Fuzzwork Enterprises Vote Steve Ronuken for CSM
1816
|
Posted - 2013.08.20 21:06:00 -
[4] - Quote
If you can read perl
https://github.com/fuzzysteve/EMDR-Consumer/blob/master/sell.pl
May be of interest. It takes all sell orders, tosses them if they're older than the currently stored date, then aggregates the data before storing it. It pays attention to the region, but only for renaming one (for historical (and laziness) reasons). Simple enough just to skip onto the next order if it wasn't the region you wanted. Steve Ronuken for CSM 9!-á I'm starting early :) Handy tools and an SDE conversion Twitter: @fuzzysteve on Twitter |
scrambled
Thirtyplus Spaceship Samurai
8
|
Posted - 2013.08.20 23:49:00 -
[5] - Quote
Steve Ronuken wrote:If you can read perl https://github.com/fuzzysteve/EMDR-Consumer/blob/master/sell.plMay be of interest. It takes all sell orders, tosses them if they're older than the currently stored date, then aggregates the data before storing it. It pays attention to the region, but only for renaming one (for historical (and laziness) reasons). Simple enough just to skip onto the next order if it wasn't the region you wanted.
Any chance you can twiddle EMDR::Consumer to be able to be used with AnyEvent or Mojo::IOLoop? Or rather, mind if I do and submit a pull request? |
Vaerah Vahrokha
Vahrokh Consulting
4263
|
Posted - 2013.08.20 23:51:00 -
[6] - Quote
Thank you too! I am going to vote for you if you really go for CSM.
As for the PERL, this is easy to read. I still recall when I had to setup some PERL payment gateways in the 90s, at the time PERL = a program 10 lines long, 8 of which were 1 regular expression Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Steve Ronuken
Fuzzwork Enterprises Vote Steve Ronuken for CSM
1820
|
Posted - 2013.08.21 02:19:00 -
[7] - Quote
Vaerah Vahrokha wrote:Thank you too! I am going to vote for you if you really go for CSM. As for the PERL, this is easy to read. I still recall when I had to setup some PERL payment gateways in the 90s, at the time PERL = a program 10 lines long, 8 of which were 1 regular expression I try not to obfuscate my code makes it a pain to adjust later.
scrambled wrote:Steve Ronuken wrote:If you can read perl https://github.com/fuzzysteve/EMDR-Consumer/blob/master/sell.plMay be of interest. It takes all sell orders, tosses them if they're older than the currently stored date, then aggregates the data before storing it. It pays attention to the region, but only for renaming one (for historical (and laziness) reasons). Simple enough just to skip onto the next order if it wasn't the region you wanted. Any chance you can twiddle EMDR::Consumer to be able to be used with AnyEvent or Mojo::IOLoop? Or rather, mind if I do and submit a pull request?
Uh, feel free?
I've not looked into any of the event frameworks (as I've not had a reason to, in the past). This has always worked well enough for my goals.
Steve Ronuken for CSM 9!-á I'm starting early :) Handy tools and an SDE conversion Twitter: @fuzzysteve on Twitter |
scrambled
Thirtyplus Spaceship Samurai
8
|
Posted - 2013.08.21 12:10:00 -
[8] - Quote
Steve Ronuken wrote:Vaerah Vahrokha wrote:Thank you too! I am going to vote for you if you really go for CSM. As for the PERL, this is easy to read. I still recall when I had to setup some PERL payment gateways in the 90s, at the time PERL = a program 10 lines long, 8 of which were 1 regular expression I try not to obfuscate my code makes it a pain to adjust later. scrambled wrote:Steve Ronuken wrote:If you can read perl https://github.com/fuzzysteve/EMDR-Consumer/blob/master/sell.plMay be of interest. It takes all sell orders, tosses them if they're older than the currently stored date, then aggregates the data before storing it. It pays attention to the region, but only for renaming one (for historical (and laziness) reasons). Simple enough just to skip onto the next order if it wasn't the region you wanted. Any chance you can twiddle EMDR::Consumer to be able to be used with AnyEvent or Mojo::IOLoop? Or rather, mind if I do and submit a pull request? Uh, feel free? I've not looked into any of the event frameworks (as I've not had a reason to, in the past). This has always worked well enough for my goals.
Yeah, just that I have an existing app that uses Mojolicious that would benefit from having EMDR access - and to keep it fast, non-blocking is the way to go. I'll give it a shot, was working on it yesterday but code that works when you run it straight from a script suddenly refuses to work right when you move it into a module, ZMQ starts throwing errors left right and center.
I'll thwap it some more, will throw a pull request up when it works :)
|
Steve Ronuken
Fuzzwork Enterprises Vote Steve Ronuken for CSM
1820
|
Posted - 2013.08.21 12:13:00 -
[9] - Quote
You might want to keep it separate, and only access the data that you're getting from the feed. For it to be useful data, you'll have to be storing it in something persistent anyway. Steve Ronuken for CSM 9!-á I'm starting early :) Handy tools and an SDE conversion Twitter: @fuzzysteve on Twitter |
Arch Stanton's Neighbour
Forceful Resource Acquisition Inc
69
|
Posted - 2013.08.21 14:39:00 -
[10] - Quote
I have a custom running program that feeds off the EMDR so a couple pointers from me:
1 - you can't just compare the lastest order's price to a minimum price because you have no way of knowing if the order your minimum price came from is still active 2 - therefore you need to know which orders are active. this is actually impossible with EMDR because it doesn't tell you when an order expires 3 - therefore you need to keep a cache of orders that you have recently seen on the data feed (i found 2 hours is the window that works best) and at any given time your mininum sell price will be the mininum from that cache of orders less than two hours old.
HTH. |
|
Vaerah Vahrokha
Vahrokh Consulting
4264
|
Posted - 2013.08.21 18:30:00 -
[11] - Quote
Arch Stanton's Neighbour wrote:I have a custom running program that feeds off the EMDR so a couple pointers from me:
1 - you can't just compare the lastest order's price to a minimum price because you have no way of knowing if the order your minimum price came from is still active 2 - therefore you need to know which orders are active. this is actually impossible with EMDR because it doesn't tell you when an order expires 3 - therefore you need to keep a cache of orders that you have recently seen on the data feed (i found 2 hours is the window that works best) and at any given time your mininum sell price will be the mininum from that cache of orders less than two hours old.
Or just get it from eve-central's api which does the legwork and gives you basically the same result....
HTH.
Thank you for your kind suggestion.
I think I am spared off this scenario, as I only have to record four instant values per given timeframe, it's not important if they become inactive after the sample. Auditing | Collateral holding and insurance | Consulting | PLEX for Good Charity
Twitter channel |
Desmont McCallock
305
|
Posted - 2013.08.21 18:40:00 -
[12] - Quote
Arch Stanton's Neighbour wrote:I have a custom running program that feeds off the EMDR so a couple pointers from me:
1 - you can't just compare the lastest order's price to a minimum price because you have no way of knowing if the order your minimum price came from is still active 2 - therefore you need to know which orders are active. this is actually impossible with EMDR because it doesn't tell you when an order expires 3 - therefore you need to keep a cache of orders that you have recently seen on the data feed (i found 2 hours is the window that works best) and at any given time your mininum sell price will be the mininum from that cache of orders less than two hours old.
Or just get it from eve-central's api which does the legwork and gives you basically the same result....
HTH. Regarding point 2, the expression 'expires' isn't correct. You can know when the order expires as IssueDate and Duration is included in the order's data, but certainly you can not know when that order gets fulfilled. |
Arch Stanton's Neighbour
Forceful Resource Acquisition Inc
69
|
Posted - 2013.08.21 18:53:00 -
[13] - Quote
Yes, that's what I meant, there is no way to know when it is fulfilled or cancelled. Thanks for the addendum. |
|
|
|
Pages: [1] :: one page |
First page | Previous page | Next page | Last page |