Pages: [1] 2 3 4 5 6 7 8 9 10 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Kallahar
|
Posted - 2010.12.18 00:43:00 -
[1]
I use eve-metrics nearly every day to check prices in other regions. I couldn't stand to see it die, so I've created a similar site.
http://eve-marketdata.com
It's clearly an homage to eve-metrics. Right now it's got the uploader client (please upload!), a market browser, and it shows all the sellers for each item. I can add in the market trend analysis if needed, but I think most people use eve-metrics for finding out the "correct" price for an item.
It also provides a "live" feed (not AMQP, but HTTP post), and (soon) will have nightly data dumps. My goal is to let people transition easily from eve-metrics to eve-marketdata.
The buggiest part currently is the uploader, if anyone would like to write a better one, please do. The current code is under "developers".
- Kallahar
|
Kallahar
|
Posted - 2010.12.18 01:08:00 -
[2]
oh, and I forgot a HUGE thank you to eve-metrics for providing such a quality service for so many years. You'll be missed!
Kallahar
|
Serene Python
|
Posted - 2010.12.18 14:46:00 -
[3]
Edited by: Serene Python on 18/12/2010 14:55:44 Not sure if I'm the only one no this boat, but I use EM for historical data... Think you implement things like that?
p.s. I mainly used it for the movement data which, not sure how you'd implement this yourself but that's a big feature for me as well as the API for developing my own apps
|
Elaiam Sendara
|
Posted - 2010.12.18 15:17:00 -
[4]
Looks like you have a good start. I hope the community gets behind this project.
|
Serene Python
|
Posted - 2010.12.18 16:33:00 -
[5]
Bugs! When I try to search something I'm being sent to pricecheck.php which results in a 404
|
P3k1
|
Posted - 2010.12.20 00:02:00 -
[6]
I'm interested in helping out if you need help on a project...
Haven't looked closely at uploader code, but i saw it was written in Python.
|
Thibault Etienne
|
Posted - 2010.12.20 21:57:00 -
[7]
Edited by: Thibault Etienne on 20/12/2010 21:59:42 I cant code or anything But ill leave my ships idling overnight and upload the markets. Least I can do. Good luck with this mate. Buyers lists would be nice too
|
Kallahar
|
Posted - 2010.12.20 21:58:00 -
[8]
Ah good find, search bug fixed. Thanks!
On the html-on-one-line, I code everything in PHP, so all outputted HTML is inside a print() statement, so there are no newlines. The back-end php code is very well organized, formatted, and commented :)
Kallahar
|
Serene Python
|
Posted - 2010.12.20 23:49:00 -
[9]
Originally by: Kallahar Ah good find, search bug fixed. Thanks!
On the html-on-one-line, I code everything in PHP, so all outputted HTML is inside a print() statement, so there are no newlines. The back-end php code is very well organized, formatted, and commented :)
Kallahar
Ahh yes PHP coding... I've often had people bug me about my HTML code looking very horrid. You working on having some form of movement being implemented?
|
Johnathan Roark
Caldari The Graduates Morsus Mihi
|
Posted - 2010.12.21 05:28:00 -
[10]
Originally by: Serene Python
Originally by: Kallahar Ah good find, search bug fixed. Thanks!
On the html-on-one-line, I code everything in PHP, so all outputted HTML is inside a print() statement, so there are no newlines. The back-end php code is very well organized, formatted, and commented :)
Kallahar
Ahh yes PHP coding... I've often had people bug me about my HTML code looking very horrid. You working on having some form of movement being implemented?
I use dwoo, makes the php and html clean, plus it makes it easier to fix the display stuff.
Originally by: Kallahar I use eve-metrics nearly every day to check prices in other regions. I couldn't stand to see it die, so I've created a similar site.
http://eve-marketdata.com
It's clearly an homage to eve-metrics. Right now it's got the uploader client (please upload!), a market browser, and it shows all the sellers for each item. I can add in the market trend analysis if needed, but I think most people use eve-metrics for finding out the "correct" price for an item.
It also provides a "live" feed (not AMQP, but HTTP post), and (soon) will have nightly data dumps. My goal is to let people transition easily from eve-metrics to eve-marketdata.
The buggiest part currently is the uploader, if anyone would like to write a better one, please do. The current code is under "developers".
- Kallahar
I commend you for taking this on, but my concern is what happens when you get tired of eve, the site, or the it gets too expensive? I would like to see a team maintain and write what ever replaces eve-metrics. I would even be interested in helping with such a project.
POS-Tracker 3.0 Hosting |
|
Serene Python
|
Posted - 2010.12.21 08:52:00 -
[11]
Originally by: Johnathan Roark
Originally by: Serene Python
Originally by: Kallahar Ah good find, search bug fixed. Thanks!
On the html-on-one-line, I code everything in PHP, so all outputted HTML is inside a print() statement, so there are no newlines. The back-end php code is very well organized, formatted, and commented :)
Kallahar
Ahh yes PHP coding... I've often had people bug me about my HTML code looking very horrid. You working on having some form of movement being implemented?
I use dwoo, makes the php and html clean, plus it makes it easier to fix the display stuff
Never heard of dwoo... I just use notepad++, my PHP code itself it typically good looking just if I got HTML it's very horrid looking typically. I prefer just typing everything rather than letting some program handle everything... Biggest pet peeve in school when we learned HTML... Final project had probably almost 1MB in useless crap being loaded/added to the code.
|
Hel O'Ween
Men On A Mission
|
Posted - 2010.12.21 11:22:00 -
[12]
I seem to be missing the feature that made EC and EM so popular: their own API for integration into other apps/services.
Are you planning on adding something alike? -- EVEWalletAware - an offline wallet manager |
Johnathan Roark
Caldari The Graduates Morsus Mihi
|
Posted - 2010.12.21 18:59:00 -
[13]
Originally by: Serene Python
Originally by: Johnathan Roark
Originally by: Serene Python
Originally by: Kallahar Ah good find, search bug fixed. Thanks!
On the html-on-one-line, I code everything in PHP, so all outputted HTML is inside a print() statement, so there are no newlines. The back-end php code is very well organized, formatted, and commented :)
Kallahar
Ahh yes PHP coding... I've often had people bug me about my HTML code looking very horrid. You working on having some form of movement being implemented?
I use dwoo, makes the php and html clean, plus it makes it easier to fix the display stuff
Never heard of dwoo... I just use notepad++, my PHP code itself it typically good looking just if I got HTML it's very horrid looking typically. I prefer just typing everything rather than letting some program handle everything... Biggest pet peeve in school when we learned HTML... Final project had probably almost 1MB in useless crap being loaded/added to the code.
Dwoo isn't a program, its a library used to separate HTML code from PHP code. Another library that does something similar is Smarty, but its a bit more bloated.
As far as programs, I write my PHP using a program called PHPDesigner for syntax highlighting and some suggestive words. Also use Notepad++ for somethings.
POS-Tracker 3.0 Hosting |
Project 69
|
Posted - 2010.12.22 13:24:00 -
[14]
alot of my php-scrits were relying on eve-metrics price checks.
so if you need any help apart from hosting, please contact me
|
GraphisoDavinci
|
Posted - 2010.12.22 15:32:00 -
[15]
Man i cant thank you enough! Hope this will stay!
Btw i think the primary goal now is autom scanning of the market like evemetrics does. Doing it now with eve metrics and uploading to evemarketdata :)
If you need help in any way (not programming) let me know
|
Kamikazie
Amarr StarFleet Enterprises Systematic-Chaos
|
Posted - 2010.12.23 02:05:00 -
[16]
Kallahar I can write PHP what ever, mail me in-game with what you need done. I would love to help on the project.
|
Lord Helghast
|
Posted - 2010.12.23 13:52:00 -
[17]
Edited by: Lord Helghast on 23/12/2010 13:54:10 #1 code suggestion should be get the automatic market crawling page the eve-metrics had that thing was awesome, saved me from getting click-induced hypertension lol.... the ability to load a page in the ingame browser, and click "upload the needed from market for my region" was just too awesome.
Suggestion beyond that a solid uploader is a must and to just keep the site light weight, don't need anything fancy, graphs are nice but honestly not needed if people want that they can load the xml via an api and do the graphing and calculations themselves, evemetrics was nice but it was easy to see where certain things got bloated (nicely bloated but bloated) mail and personal market orders i'd say even aren't needed features for a market site... keep it light weight, easily updateable and with a solid fast API and your site will be GREAT!!!
historical data is important but precision of historical data is something you might consider, #1 drop all db data for same item within the same say 15 minutes if the age is < 1 month > 6 hours .... if its > 1 month < 2 months drop data under 30 minutes, so that you have only 30 minute updates at the 2nd month, on the third and back to 1 year i'd say drop everything other than hourly updates for each item per region...
this way you can keep your database light but still have useable historical data.
Also when you design the API make it smart allow the developers out their to only request updates from a last key time
so if i ask for historical data for say ITEM999 don't send me a dump of the entire month, let me say " I want ITEM999 pricehistory since 10:15pm yesterday" that way developers of apps can be smart and only request data from the server that they really need and not keep grabbing huge dumps of csv/xml files of data.
same thing for price check api, keep it small and simple, request = ITEM999, type = sellmax, date = recent, region forge ... response 1000isk.... you can have a price full item grab api that sends the sell and buy max min mean for forge for example, but make sure that when you do api that theirs a nice light weight fast way to just get a price that has low overhead and low bandwidth demand.
BTW an idea to reduce leechers, actually implement a credit system, uploading gives credits, more indemand items/regions give higher credit reward, and using the API for calls deducts credits.... and for those that don't want to be uploading for whatever reason or are just over-pulling data, allow them to make ISK donations for credit bundles.
|
Kallahar
|
Posted - 2010.12.23 22:51:00 -
[18]
All excellent ideas, I get the feeling that the most overhead will be in the database. Here are some stats: 66 Regions 7,928 solarsystems 10,860 items available on the market (eve_inv_types.published=1) So that means 716,760 region+items, and each one has a list of buyers and sellers. 102,168 uploads to eve-marketdata so far
I haven't really played the markets, is having historical data needed? Is it different than going in-game and looking at the market "price history" or is it just wanting that data from a different region?
On the open-source comments, I fully support the whole open source community, but I have no interest in being in charge of a project or having to deal with reviewing submitted code. So my best thoughts are to keep the "live feed" open to anyone (ie, have a single uploader that everyone can use), then everyone can implement their own ideas separately. I could also just open up the mysql database, but I have to pay cold hard cash for that+bandwidth...
Thanks for the interest everyone!
Kallahar
|
Serene Python
|
Posted - 2010.12.24 08:38:00 -
[19]
Originally by: Kallahar I haven't really played the markets, is having historical data needed? Is it different than going in-game and looking at the market "price history" or is it just wanting that data from a different region? Kallahar
In terms of what I use it for... YES! One of my apps uses movement/price history grabbed from the API. Then there's people who either can't login to check prices or don't want to start eve up to just check prices. I've also been in that boat because I didn't want to login/load up eve.
|
Kallahar
|
Posted - 2010.12.25 18:01:00 -
[20]
I got api/item.xml and api/item.json working, but I'm not sure how to calculate the following:
- median - kurtosis - skew - variance - standard_deviation - simulated
If anyone knows how to calculate these in mysql/php, please let me know and I'll add them in.
Kallahar
|
|
Serene Python
|
Posted - 2010.12.25 19:42:00 -
[21]
Originally by: Kallahar I got api/item.xml and api/item.json working, but I'm not sure how to calculate the following:
- median - kurtosis - skew - variance - standard_deviation - simulated
If anyone knows how to calculate these in mysql/php, please let me know and I'll add them in.
Kallahar
Those all sound all to familiar from math class...
Median (The middle value?) would be something like
$query = mysql_query("SELECT * FROM eve_item_prices WHERE regionID = $_GET[regionID]") $rows = mysql_num_rows($query) $median = $rows/2
Don't go line for line with that just a rough guess, taking the total number rows in a region and just dividing by 2 to get the middle row. Just got $query[$median] to get the middle value of the prices. If you have an odd number of rows then just round the $median price.
Merry Christmas!
|
Nikolai Kondratiev
Sphere Design Inc.
|
Posted - 2010.12.25 21:53:00 -
[22]
Originally by: Kallahar I got api/item.xml and api/item.json working, but I'm not sure how to calculate the following:
- median - kurtosis - skew - variance - standard_deviation - simulated
If anyone knows how to calculate these in mysql/php, please let me know and I'll add them in.
Kallahar
Median can be done easily in either PHP or MySQL. Here is some code I use in PHP to calculate it before inserting values in database :
sort($values); $n = count($values); $h = intval($n / 2); if($n % 2 == 0) $median = round(($values[$h] + $values[$h-1]) / 2); else $median = $values[$h];
EM's simulated was the highest price you would have to pay to buy 5% of sell orders IIRC. Probably best done in PHP too when you get new order list for an item.
And for all the stat stuff, a simple google search should give you some explanations on how to calculate it (and probably some code for it too)
_ WTS Capital BPOs |
Serene Python
|
Posted - 2010.12.25 23:19:00 -
[23]
http://php.net/manual/en/function.stats-standard-deviation.php
Can't actually code... PHP has some configuration issue I can't be bothered to fix right now
|
woddel
Gallente Canis Industries Ltd Avaricious Cartel
|
Posted - 2010.12.26 14:32:00 -
[24]
hia kallahar
thanks for making this project and hopefully an equal alternative to eve metrics.
one question : do you already have some documentation how your http post service works? meaning the format the data is posted in, variable names etc. that would be great.
also, i am looking forward to a pull api where one can request certain item prices for specific regions. i guess the most important would be 'give me price for tritanium in forge' for example. what would be great is not to simply return the average of all sell order, but try to eliminate all the pseudo-sell orders like shuttles for 100 mil isk - that would greatly falsen the result. in my app, i usually take the weighted average (price * amount) of the top 75% of sell orders and try to ignore orders where the price goes badly above average. weighted average reduces the deviation from like one guy selling a megathron for 50 mil when there are 50 others for 75 mil. the correct price would then be like around 74.5 mil.
last tipp as a developer for myself : keep it simple. do stuff one step at a time. folks (like me) will always want more features - be sure to have that stuff working you do at the moment until you move on the the next module or feature.
good luck and thanks again!
woddel --- creator and maintainer of eve commander - complete web based character information tool and ec agent finder |
Hel O'Ween
Men On A Mission
|
Posted - 2010.12.26 17:53:00 -
[25]
Originally by: Kallahar I got api/item.xml and api/item.json working, but I'm not sure how to calculate the following:
- median - kurtosis - skew - variance - standard_deviation - simulated
More than you perhaps ever wanted to know about statistics: http://en.wikipedia.org/wiki/Statistics.
(Scroll to the bottom of the page for links to algorithms) -- EVEWalletAware - an offline wallet manager |
Kallahar
|
Posted - 2010.12.26 23:47:00 -
[26]
Okay, got median and standard deviation added to the item api, I'll have to figure out how to add the pecl stats extensions to php to get kurtosis and the others working.
For the HTTP callback api, I added documentation for it, just click of "developers".
I'll be adding a simple "what's the "correct" price for an item soon, it will be the best-guess price you could buy/sell an item for in Jita, that should be the price everyone uses as "the price".
Kallahar
|
Serene Python
|
Posted - 2010.12.27 08:12:00 -
[27]
Edited by: Serene Python on 27/12/2010 08:11:54
Originally by: Kallahar Okay, got median and standard deviation added to the item api, I'll have to figure out how to add the pecl stats extensions to php to get kurtosis and the others working.
For the HTTP callback api, I added documentation for it, just click of "developers".
I'll be adding a simple "what's the "correct" price for an item soon, it will be the best-guess price you could buy/sell an item for in Jita, that should be the price everyone uses as "the price".
Kallahar
Do you any intention of putting in the historical movement/historical prices?
|
Hel O'Ween
Men On A Mission
|
Posted - 2010.12.28 09:43:00 -
[28]
Originally by: Kallahar I got api/item.xml and api/item.json working [...]
Sort of, it seems. For me, the closing </xml> tag is missing.
The query (typeID 24702 is the Hurricane) http://eve-marketdata.com/api/item.xml?type_ids=24702 returns
<?xml version='1.0' encoding='utf-8' ?> <evemetrics api_version='2.0'> <!-- EVE MarketData Item API Documentation is available at the same URI with a .html extension. --> <type id='24702'> <name></name> <global> <last_upload>2010-12-26 13:51:46</last_upload> <buy> <minimum>2800</minimum> <maximum>24110122.01</maximum> <median>20170003</median> <average>14267671.157727</average>
<standard_deviation>6673819.0621229</standard_deviation> </buy> <sell> <minimum>22000000</minimum> <maximum>19000000000</maximum> <median>26250000</median> <average>45719842.5672625</average> <standard_deviation>1229254309.9302</standard_deviation> </sell> </global> </type>
</evemetrics>
BTW, is there another place/way to provide feedback to you which you prefer over the EO forums? -- EVEWalletAware - an offline wallet manager |
Catari Taga
Centre Of Attention Middle of Nowhere
|
Posted - 2010.12.28 13:53:00 -
[29]
Edited by: Catari Taga on 28/12/2010 13:55:20
Originally by: Hel O'Ween
Originally by: Kallahar I got api/item.xml and api/item.json working [...]
Sort of, it seems. For me, the closing </xml> tag is missing.
There is no closing </xml> tag in XML since there is no opening <xml> tag either (unless you make it part of the document structure which he didn't). The "<?xml..." part is the doctype definition which is not a block element.
But what is wrong is that the server sends incorrect content type text/html instead of text/xml.
--
|
Hel O'Ween
Men On A Mission
|
Posted - 2010.12.28 16:11:00 -
[30]
Erhm ... yes ... of course.
Obviously coffee level wasn't high enough this morning, when I wrote that ... -- EVEWalletAware - an offline wallet manager |
|
|
|
|
Pages: [1] 2 3 4 5 6 7 8 9 10 :: one page |
First page | Previous page | Next page | Last page |