Pages: [1] :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.12 02:10:00 -
[1] - Quote
Anyone have a easy way of getting this? I've looked at; http://eve-marketdata.com/ http://www.eve-cost.eu http://www.evemarketeer.com/
All I want to do is download a bunch of pricing data against TypeID's that have a price(s). In CSV, or MySQL format (preferred).
I read through this; https://forums.eveonline.com/default.aspx?g=posts&t=18466
Basically, where do I find the best snapshot of the market (reasonable avg. price over a whole week) that I run pull against hundreds of items all at once without needing to spam price checks against either a XML request.. or saving that.. having to write my own market uploader/price check system (like eve central).. cause.. that is a little crazy.
Any suggestions are helpful, thanks! |
Ilyk Halibut
Blackwater USA Inc. Against ALL Authorities
1
|
Posted - 2012.04.12 05:22:00 -
[2] - Quote
I'm working on a real-time feed, but you'd need to store the messages yourself. Look for an announcement post in the next week or two if this interests you. If not, best of luck! EVE Alerter - Rally your troops! EVE Mail me to get your own corp/alliance alerter http://evealerter.com |
Snarf Aldes
University of Caille Gallente Federation
1
|
Posted - 2012.04.12 15:17:00 -
[3] - Quote
All the sites you mentioned have an api that lets you do exactly what you want. I can add my site to that list, it will give you current buy and sell prices of items based on typeID. You can construct an url with the typeID's, or you can create a list of items (with the IGB) and pull the prices of those items via the api.
Creator of Eve Addicts |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.12 19:14:00 -
[4] - Quote
Snarf Aldes wrote:All the sites you mentioned have an api that lets you do exactly what you want. I can add my site to that list, it will give you current buy and sell prices of items based on typeID. You can construct an url with the typeID's, or you can create a list of items (with the IGB) and pull the prices of those items via the api.
Nice, while your option is not exactly what I wanted, I can use cURL to pull your datasets, sweet! :) |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 06:19:00 -
[5] - Quote
Snarf Aldes wrote:All the sites you mentioned have an api that lets you do exactly what you want. I can add my site to that list, it will give you current buy and sell prices of items based on typeID. You can construct an url with the typeID's, or you can create a list of items (with the IGB) and pull the prices of those items via the api.
Question (not a double post), do you ever plan on increasing your URI limit on your website to around say.. 65,121 or higher? I've noticed your using Apache 2.2.14 with a Ubuntu installation with probably max (usable) URI limit of 8,177. (Meaning I'd need to query your server 8-9 times per a price check.)
You can add the LimitRequestLine directive in; /etc/apache2/apache2.conf
Of course you will need to restart apache after changing the directive; /etc/init.d/apache2 restart
Alternatively you can run this instead to refresh without turning off the apache daemon process; /etc/init.d/apache2 reload
I believe the upper limit is around 128k or more.
More information on limits can be obtained here; http://httpd.apache.org/docs/2.2/mod/core.html#limitrequestline (the directive you'd use) http://www.danrigsby.com/blog/index.php/2008/06/17/rest-and-max-url-size/ (normal maximums)
Occasionally you may need to recompile with the following option (assuming static install); CFLAG "-D DEFAULT_LIMIT_REQUEST_LINE=(new limit)"
Alternatively.. you could modify your code so it suppports both $_GET, and $_POST variables. Let me know. |
Snarf Aldes
University of Caille Gallente Federation
4
|
Posted - 2012.04.14 09:00:00 -
[6] - Quote
ok, i see the problem here.
Are you trying to construct a big url for the API, or are you trying to insert a large string to build an item list on the market scanner page?
If you are trying to make a big API request, and the items you want to query don't change, then i would suggest you make an item list instead. You only have to enter them once, they are saved in the database, and you can retrieve the prices with your list number + upload key (both visible on the market scanner page).
Creator of Eve Addicts |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 15:54:00 -
[7] - Quote
Snarf Aldes wrote:ok, i see the problem here.
Are you trying to construct a big url for the API, or are you trying to insert a large string to build an item list on the market scanner page?
If you are trying to make a big API request, and the items you want to query don't change, then i would suggest you make an item list instead. You only have to enter them once, they are saved in the database, and you can retrieve the prices with your list number + upload key (both visible on the market scanner page).
I tried to do that, but the Eve Client Browser is too weak. It only did the first 11,000 items. |
Snarf Aldes
University of Caille Gallente Federation
4
|
Posted - 2012.04.14 16:54:00 -
[8] - Quote
in that case you did more than are actually on the market. You took the invTypes table and looked at every item with published=1, that would result in 11402 items. 4476 of those are not on the market, you have to add 'AND marketGroupID IS NOT NULL' to your WHERE clause.
End result should be 6926 items.
Most of the 4476 items are things that are just available via contracts, and hence have no market price. Creator of Eve Addicts |
Snarf Aldes
University of Caille Gallente Federation
4
|
Posted - 2012.04.14 17:39:00 -
[9] - Quote
I've made it easy for you, you can pull the prices from all items on the market via the API. Instead of typeID=x,x, you just add 'getAll=true' to the url. Creator of Eve Addicts |
LifeHatesMe
SKULLDOGS RED.OverLord
4
|
Posted - 2012.04.14 19:06:00 -
[10] - Quote
Nupe.. something must be amiss. I'm using the crucible MySQL database, and when I run;
SELECT COUNT( * ) FROM `invTypes` WHERE `invTypes`.`published` =1
Result is: 11,402
Your CVS format will report null on values it can't find. I suppose that's fine. I wrote my program to dump such data with "0".
List of ID's I'm getting; http://spartan4.net/eve/gen_list_ids.php
The database I imported on my server... (About 500MB of data) Crucible 1.6_65449 in MySQL format |
|
Snarf Aldes
University of Caille Gallente Federation
4
|
Posted - 2012.04.14 20:28:00 -
[11] - Quote
if you do:
SELECT COUNT( * ) FROM `invTypes` WHERE `invTypes`.`published` =1 AND `invTypes`.`marketGroupID` IS NOT NULL
you will get the 6926 items that are actually on the market.
Creator of Eve Addicts |
|
|
|
Pages: [1] :: one page |
First page | Previous page | Next page | Last page |