Pages: 1 2 [3] 4 5 6 7 8 9 10 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Fovo
Timetravel Enterprises Dark Matter Coalition
|
Posted - 2011.01.07 15:16:00 -
[61]
One thing I just realised, is I am doing system maintenance on my server this weekend and I am not sure how to temporarily disable the data feeds.
It's not an issue on my side really, but no need for your server to waste resources feeding me data when I am off line.
Recommendation. if you receive a 404, stop the feed until re-requested.
Fovo
|
Eve Industrialist
|
Posted - 2011.01.07 15:18:00 -
[62]
I'm going to get behind this. I used eve central for pricing for my mfg app and just when I get price caching working correctly eve central goes awol. Having alternate price source would be good anyway. I used their xml api and yours is close enough. This alt sits in jita and can easily pull prices for you on a daily basis or move to another trade hub. I may or may not be able to help with coding depending on code base and needs.
|
Kallahar
|
Posted - 2011.01.07 18:40:00 -
[63]
Improved the automatic uploader, it no longer checks for T2 ship blueprints, and I removed about 50 items that the CCP export *says* are available on the market, but they actually aren't. It still will check items that often don't have any sellers in your region, but that's by design. There are now 6,133 items checked, which will take about 7 hours to scan the entire market for a region for one client.
I've also decided to discontinue the item.xml/json api. I think that 99% of the people using it are only getting the average price, so that they can update their killboard, etc. However it provides far more info, which is CPU intensive. Therefore, I'm going to kill off that program, and people will have to update their code to use one of these new API's: - http://eve-marketdata.com/developers/item.php for one item lookup - http://eve-marketdata.com/developers/items.php for multiple
I do plan on adding statistical data back, but it will be with a new URL. This way, people who only need the price won't be wasting CPU or bandwidth getting a bunch of extra data they don't need.
Kallahar
|
Greshick
Concordokken.
|
Posted - 2011.01.07 19:18:00 -
[64]
Not sure if you have the server done or something but I just received a 503 error message when I tried to visited the site in my OoG browser.
|
Kallahar
|
Posted - 2011.01.07 19:51:00 -
[65]
Nightly Data Dumps added (and all mysql exports) to http://eve-marketdata.com/developers/
I'll look into the 503 errors, for now just reload the page.
Kallahar
|
Greshick
Concordokken.
|
Posted - 2011.01.08 00:23:00 -
[66]
Thanks for looking into the 503's. I like the work your doing so far and I have been trying to help keep at least Jita's data up to date. A thought we would be to allow filtering by sec status, so could only see in high sec or filter by station. The stations would be nice for selling and buying in Jita.
|
Droxlyn
Caldari
|
Posted - 2011.01.08 03:31:00 -
[67]
I'm mostly looking for the 5% market volume buy effective buy price and the 5% market volume sell price, ideally for a specific system. (IE Jita or Amarr)
That way I can look up my costs for input materials and the profit for the easy sell. I sometimes average the two together to get the middle buy/sell price that should go fast but make me or cost me just a little.
Averages and Medians are not as interesting to me. (Nor are the extremes.)
Drox
|
Dharken Rhaal
Amarr
|
Posted - 2011.01.09 07:58:00 -
[68]
I was using buy/maximum and sell/mininum limited by region (The Forge obviously) from EM. Either I dumb or there is no such possibility with eve-marketdata. "best guess" probably is good for guessing price at killboards, but when you need to decide what to produce/buy/sell - you need more. (tritanium - 3.95?.. definitely need something more precise) Probably I could take it from historic feeds, but that's a lot of data instead of two simple values. Can we have simple maxBuy and minSell taken from most recent upload with region as parameter, please? :)
|
Grimdahl
|
Posted - 2011.01.09 09:53:00 -
[69]
Originally by: Kallahar ... I think that 99% of the people using it are only getting the average price, so that they can update their killboard, etc.
Well -- Evemeep and all it's users are the 1% left it seems. If there are really this less people using this extra data -- why not provide a specific URL exactly for this use? (Evemeep always requests only one item combined with region)
|
otto M8
|
Posted - 2011.01.09 20:07:00 -
[70]
Originally by: Droxlyn I'm mostly looking for the 5% market volume buy effective buy price and the 5% market volume sell price, ideally for a specific system. (IE Jita or Amarr)
That way I can look up my costs for input materials and the profit for the easy sell. I sometimes average the two together to get the middle buy/sell price that should go fast but make me or cost me just a little.
Averages and Medians are not as interesting to me. (Nor are the extremes.)
Drox
First, a tip of the hat and a thanks for taking on this project. I agree with Droxlyn (as a user who got used to EMs simulated prices) that a price on some volume of significance is valuable. A 5% of volume is a great comprise between too little and too much processing/calculation.
Second, as a follow up, what would be even more useful is a standard set of statistics such as you had up briefly. I am most interested in pulling buy/sell prices from one region for quantities that are relevent for me to compare with the corresponding buy/sell prices in other regions.
When faced with the demise EM I started working towards pulling minimum sell, maximum buy by regions for items of interest to me limited by quantities that are useful to me in decision making. For instance, a minimum price is not useful if someone is trying to quickly unload a handful of an item dropped as mission loot if I need 100k units of it. The price I need to pay for those is of much great interest. So I might ballpark a minimum price for 50k units on that item in a specific region.
With EM's simulated price at 5% of the volume on market I got lazy and found that useful enough.
I guess I'm also a little unclear about what the "best guess" price actually means. In a killboard context it is probably not that critical, in an industrial context it is. I'm assuming it is a trimmed/filtered min sell, max buy, but please correct my assumptions if they are incorrect.
Cheers
|
|
Ten Bulls
Minmatar
|
Posted - 2011.01.09 23:20:00 -
[71]
Edited by: Ten Bulls on 09/01/2011 23:23:21 He, site and tool looks promising, good stuff.
I just tried it out, i had some issues, ingame it was checking a lot more prices than where being uploaded from the tool, i stopped getting prices ingame and the uploaded continued uploading the backlog of prices for quite some time, however it stopped without uploading all of them. i.e. there are still files in the cache folder that ts not trying to upload.
I tried enabling verbose output but then the uploader "fails to respond" as windows puts it.
Actually, i did have a torrent going in the background, maybe that was slowing down my uploads and causing problems...
|
vonDuck
Gallente Knights Templar ordre de eve brethren mmx Black Thorne Alliance
|
Posted - 2011.01.10 15:05:00 -
[72]
Something I do is eliminate anything outside of two standard deviations to try to cull outliers from screwing up the prices. You do this by region to ensure you don't get a universe-wide minimum or maximum. You can apply this to volume as well as price, too.
Another thing I do is create a hub price in addition to the region price. I find the system with the most inventory in the market and call that an "item hub". This isn't always the traditional hub as concentrations of drops might occur nearer to the mission system as people try to dump stock and get to the next mission.
Another edit is to ensure the min transaction volume is either 1 or the batch quantity of the item. This avoids the volume market scheme where you buy a bunch of stuff for a great trade just to realize you must have a min volume that is equal to all of the inventory in the region. Good work. Sleep well. I'll most likely kill you in the morning. --Dread Pirate Roberts |
Fovo
Timetravel Enterprises Dark Matter Coalition
|
Posted - 2011.01.10 22:47:00 -
[73]
One thought for those interested in a specific result set. If you are capable of writing the sql query for that specific result and willing to contribute, consider providing it with your request. If I were in Kallahar's shoes, that would make things for me easier..
But then again, Kallahar should probably give this a thumbs up first.
Fovo
|
Kari Trace
Caldari Foyu Trading World
|
Posted - 2011.01.11 04:04:00 -
[74]
Awesome! Plan on my uploading often!
|
Luminocity
|
Posted - 2011.01.11 09:41:00 -
[75]
This..
Originally by: otto M8 First, a tip of the hat and a thanks for taking on this project.
..and this
Originally by: Dharken Rhaal I was using buy/maximum and sell/mininum limited by region (The Forge obviously) from EM. Either I dumb or there is no such possibility with eve-marketdata. "best guess" probably is good for guessing price at killboards, but when you need to decide what to produce/buy/sell - you need more. (tritanium - 3.95?.. definitely need something more precise) Probably I could take it from historic feeds, but that's a lot of data instead of two simple values. Can we have simple maxBuy and minSell taken from most recent upload with region as parameter, please? :)
|
Serene Python
|
Posted - 2011.01.11 10:59:00 -
[76]
Originally by: Kallahar **** HISTORY ADDED **** Much requested, I finally got history added. Here's the quirk, people running the uploader have to have it on the price history tab in order for it to upload, so I've got all of The Forge done, but other regions will take a while to get data.
see http://eve-marketdata.com/developers/item_history.php for info, note that it's *not* eve-marketdata compatible because I forgot to save their formatting before they closed... heh.
Kallahar
Nice to see that movement finally made it in. Thanks
|
randomToon987
|
Posted - 2011.01.11 13:20:00 -
[77]
Originally by: Kallahar
...snip...
I've also decided to discontinue the item.xml/json api. I think that 99% of the people using it are only getting the average price, so that they can update their killboard, etc. However it provides far more info, which is CPU intensive. Therefore, I'm going to kill off that program, and people will have to update their code to use one of these new API's: - http://eve-marketdata.com/developers/item.php for one item lookup - http://eve-marketdata.com/developers/items.php for multiple
I do plan on adding statistical data back, but it will be with a new URL. This way, people who only need the price won't be wasting CPU or bandwidth getting a bunch of extra data they don't need.
Kallahar
I'm having high hopes for this service, but removing the most comprehensive part of the api makes it useless for me, and many others. We might not be the biggest group, but we're developers too, and when we write special solutions for the public or our corp to use we always mention the source of our market data, so a lot of people learn about sites such as eve-metrics(R.I.P.) or eve-central through third party tools. Wouldn't caching extra statistical data be a better solution than removing it altogether?
On a somewhat related note - I'm sure that designing your own api gives you more freedom, and that's ok, but you should analyse eve-metrics api, and try to learn something from the way it was designed. For example - you're not using api version number, which may result in a lot of applications relaying on your site breaking without warning when you decide to change something, using api version number is an easy way to make sure that the data you provide and the application that processes it will work together just fine, and if the version numbers differ it's easy to shut down gracefully.
|
Hel O'Ween
Men On A Mission
|
Posted - 2011.01.11 14:41:00 -
[78]
Originally by: Kallahar
I've also decided to discontinue the item.xml/json api. I think that 99% of the people using it are only getting the average price, so that they can update their killboard, etc. However it provides far more info, which is CPU intensive. Therefore, I'm going to kill off that program, and people will have to update their code to use one of these new API's: - http://eve-marketdata.com/developers/item.php for one item lookup - http://eve-marketdata.com/developers/items.php for multiple
Not sure why you did that.
1) Two different URLs for the same purpose? This way you have to keep two scripts up tp date. Besides, you're already doing the parameter parsing with the second one (multiple items). The single item (single value) is just a special case of multiple items.
2) In terms of (output) format: is that the final result we're getting there already? Because that seems to be some kind of home-brewn result format. Are you planning to return something more standard (like the good old XML)? -- EVEWalletAware - an offline wallet manager |
Jensius Duo
Digital Fury Corporation Northern Coalition.
|
Posted - 2011.01.11 17:48:00 -
[79]
Great honourable effort!
Any chance to have a csv/xml with all the latest price data for every item, in order to be able to look up prices quickly? Eve-Metrics had a latest-basic.csv of ~600kB: type_id,buy_min,buy_max,buy_med,buy_avg,sell_min,sell_max,sell_med,sell_avg,buy_simulated,sell_simulated
|
Azazel Mordred
Minmatar Cloak of Shadows
|
Posted - 2011.01.11 18:01:00 -
[80]
I'd really like to implement this in a project I work on, but as others have mentioned, the missing min and max sell/buy values (per region and otherwise) is unfortunately a deal breaker for me
|
|
Kallahar
|
Posted - 2011.01.11 22:00:00 -
[81]
Thanks everyone!
1) Added a new API method - item_orders - it gives you all the orders for a region, so it should be easy for those of you who want the statistics to grab that and calculate it yourself. If it's not adequate, let me know! 2) Upgrading the database server tonight, so everything should be much faster. 3) SQL queries - YES! If you give me a query it's much easier for me to implement it. See the developers tab under mysql table definitions for the DB layout. 4) XML/JSON formats: if you find any specific errors, let me know. Personally I don't use them. Keep in mind that I can no longer access the eve-metrics API, so I'm not sure exactly how they did it.
Kallahar
|
vonDuck
Gallente Knights Templar ordre de eve brethren mmx Black Thorne Alliance
|
Posted - 2011.01.12 05:05:00 -
[82]
If anybody can explain what they want, I can write the query (or explain why it can't be done ). Good work. Sleep well. I'll most likely kill you in the morning. --Dread Pirate Roberts |
Luminocity
|
Posted - 2011.01.12 07:48:00 -
[83]
This..
Originally by: Hel O'Ween 1) Two different URLs for the same purpose? This way you have to keep two scripts up tp date. Besides, you're already doing the parameter parsing with the second one (multiple items). The single item (single value) is just a special case of multiple items.
Just remove "item price" as "item prices" already accepts both single and multiple values (comma delimited) as the typeID parameter..
Also, good work on the new sell orders API, it's a nice addition.. Now if we could have something in between of the "item prices" (which is ideal for killboards due to being very simple and general) and the new "sell orders" (which is probably good for more in-depth analysis by being quite detailed). Something which would return the latest lowest sell order and highest buy order prices for a specific region. (please see this feature from eve-central as an example)
|
Azazel Mordred
Minmatar Cloak of Shadows
|
Posted - 2011.01.12 07:58:00 -
[84]
This would be helpful if the DB can manage it... Perhaps it would be useful to archive expired orders in the buying/selling tables after a week or two, to keep things a bit faster.
select min(b.price) as buy_min, max(b.price) as buy_max, avg(b.price) as buy_avg, min(s.price) as sell_min, max(s.price) as sell_max, avg(s.price) as sell_avg from eve_inv_types t left join items_buying b on b.type_id = t.type_id and b.region_id = ? and b.date_expires > current_timestamp left join items_selling s on s.type_id = t.type_id and s.region_id = ? and s.date_expires > current_timestamp where t.type_id = ?
|
Luminocity
|
Posted - 2011.01.12 08:16:00 -
[85]
Edited by: Luminocity on 12/01/2011 08:18:00
Originally by: Azazel Mordred This would be helpful if the DB can manage it... Perhaps it would be useful to archive expired orders in the buying/selling tables after a week or two, to keep things a bit faster.
select t.type_id, min(b.price) as buy_min, max(b.price) as buy_max, avg(b.price) as buy_avg, min(s.price) as sell_min, max(s.price) as sell_max, avg(s.price) as sell_avg from eve_inv_types t left join items_buying b on b.type_id = t.type_id and b.region_id = ? and b.date_expires > current_timestamp left join items_selling s on s.type_id = t.type_id and s.region_id = ? and s.date_expires > current_timestamp where t.type_id in (?)
Quick on the draw there Azazel, nice work :) This is pretty much exactly what I had in mind. It might be good to have it accept multiple typeID values (I modified the code above) as the current "items price" method does (comma delimited) so there would be less calls to the method (though potentially more work done internally for each call). Maybe have it limited to a specific amount of accepted typeID's per each call?
Also API versioning is a good idea, I think. Especially as there are probably a lot of modifications in the early days of this project..
Originally by: randomToon987 On a somewhat related note - I'm sure that designing your own api gives you more freedom, and that's ok, but you should analyse eve-metrics api, and try to learn something from the way it was designed. For example - you're not using api version number, which may result in a lot of applications relaying on your site breaking without warning when you decide to change something, using api version number is an easy way to make sure that the data you provide and the application that processes it will work together just fine, and if the version numbers differ it's easy to shut down gracefully.
|
Luminocity
|
Posted - 2011.01.12 10:17:00 -
[86]
On another note. It looks like your methods currently don't accept HTTP POST requests (which I would prefer for multiple reasons). I don't know about the late eve-metrics but eve-central accepts both POST and GET requests, which would be nice in this case aswell..
|
Hel O'Ween
Men On A Mission
|
Posted - 2011.01.12 12:18:00 -
[87]
Originally by: Kallahar
4) XML/JSON formats: if you find any specific errors, let me know. Personally I don't use them. Keep in mind that I can no longer access the eve-metrics API, so I'm not sure exactly how they did it.
First, the announced items.php doesn't seem to exist: http://eve-marketdata.com/api/items.php?type_ids=34 results in a 404.
Whereas http://eve-marketdata.com/api/item.php?type_ids=34 works, but returns a format I'm nor familiar with:
\ntype_id\tbuy\tmin\tmax\tmedian\taverage\tstd_dev\[...] 34 buy0.013.42.41.8621828514751542462764.62665 sell0.013.421.862182851475150.45284745607815
Content type of the above: text/plain
Ah, I see, there's item_prices.xml which just holds a "price"
Now I'm really confused. Why can't you return the same amount of information in the XML (I guess JSON has the same issue) like you do in the item.php?
I'd really appretiate your efforts and I'd love to add your API to EWA, but I guess I'll wait until the "dust has settled" with the API stuff and a stable solution/format has been established. -- EVEWalletAware - an offline wallet manager |
Kallahar
|
Posted - 2011.01.12 17:35:00 -
[88]
Thanks everyone!
1) All API's now accept POST or GET 2) /api/item.* has been removed, it was too CPU intensive for what most people needed. Click on "developers" to see the other API options. 3) Expired orders are now removed within 10 minutes, or sooner if someone checks the market for that item.
Kallahar
|
Colgir
|
Posted - 2011.01.12 18:00:00 -
[89]
Hello,
at first, to make this tool is a great idea and i will upload as much data as i can. But i have found a little Problem: There is a whole region missing. :)
The errormsg: Error: region not found for region_name = Derelik
|
Fovo
Timetravel Enterprises Dark Matter Coalition
|
Posted - 2011.01.12 20:07:00 -
[90]
If anyone has a chance and would be so kind, I posted a related question on the forums here.
I need to create a very specific xml format for an application i would like to use. I will be combining data feeds from eve-marketdata and the eve-api and want to create one page that can run the specific query on the items requested.
Thanks
Fovo
|
|
|
|
|
Pages: 1 2 [3] 4 5 6 7 8 9 10 :: one page |
First page | Previous page | Next page | Last page |