Pages: [1] 2 3 4 5 6 7 8 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 0 post(s) |
Hexxx
Minmatar
|
Posted - 2009.02.23 08:11:00 -
[1]
http://www.eve-metrics.com/
This is a website similar in function to EVE Central, with one important difference; it includes a tool that automatically uploads any market data you view while playing EVE. It does this by "scraping" the cache files that EVE uses when you view the market details for anything. No fuss, no muss, just automatic feeds of data.
There is also security that prevents false market data from entering the database and some extremely useful reporting and statistic tools. In the future, they plan on having an API for developers to use.
Check out the website, check out the reports, check out how awesome this is!
All this from Ix Forres, the guy who created IskSense.
EBANK - Chairman of the Board | www.eve-bank.net
|
MK Dom
|
Posted - 2009.02.23 09:01:00 -
[2]
Auto loading the market info is wftbbqpwng!
|
Hexxx
Minmatar
|
Posted - 2009.02.23 09:17:00 -
[3]
Originally by: MK Dom Auto loading the market info is wftbbqpwng!
In a word; yes.
EBANK - Chairman of the Board | www.eve-bank.net
|
Kazuo Ishiguro
House of Marbles Zzz
|
Posted - 2009.02.23 09:19:00 -
[4]
So, to clarify, there's no need to spam 'export data' for every item? Sounds good. Has Ix checked that using cache files in this way is EULA-safe?
I'm sceptical about the security features - if you can read cache files, someone else can probably write to them if they really want to try to poison the database. Also, it lacks some of the flexibility of the eve-central API - the option to restrict API queries to a single station or solarsystem seems to be missing.
Having said all that, it certainly looks promising. --- Can't afford that BPO? Look here. 20:1 mineral compression The EVE f@h team |
Daphne Eveningstar
|
Posted - 2009.02.23 09:25:00 -
[5]
Edited by: Daphne Eveningstar on 23/02/2009 09:25:33 Edit: Assuming this is automated and no "export to file" spamming is needed,
There was a guy about this time last year making a tool that used these same files to do some automated price analysis. It was deemed a EULA violation.
tbh I'm concerned that CCP hasn't locked down those data streams to prevent this. Either make it legal and official (and automatic) or do something to disable the functionality (ala Bacon).
|
Hexxx
Minmatar
|
Posted - 2009.02.23 09:25:00 -
[6]
Originally by: Kazuo Ishiguro So, to clarify, there's no need to spam 'export data' for every item? Sounds good. Has Ix checked that using cache files in this way is EULA-safe?
I'm sceptical about the security features - if you can read cache files, someone else can probably write to them if they really want to try to poison the database. Also, it lacks some of the flexibility of the eve-central API - the option to restrict API queries to a single station or solarsystem seems to be missing.
Having said all that, it certainly looks promising.
He uses some kind of checksum to verify authenticity of the cache file. I'm not clear on the details but I believe it to be effective.
In terms of restricting the queries on stations or solar systems...markets operate best when they're efficient. Transparency breeds efficiency. So this program seems to operate in the best interests of a transparent/efficient market.
The most useful function isn't the solarsystem specific sell/buy orders but instead the indexes created from all these orders. I noticed that some of them are down right now, but the promise of nearly automatically created indexes is....well....incredibly promising.
Indexes like those could be used as the foundation for futures contracts on minerals for example. This is just one of many possible uses.
EBANK - Chairman of the Board | www.eve-bank.net
|
Alaki Kant
|
Posted - 2009.02.23 09:26:00 -
[7]
It might be allowed to read the cache files, but it would be an EULA violation to write to them.
|
Hexxx
Minmatar
|
Posted - 2009.02.23 09:28:00 -
[8]
Originally by: Daphne Eveningstar Edited by: Daphne Eveningstar on 23/02/2009 09:25:33 Edit: Assuming this is automated and no "export to file" spamming is needed,
There was a guy about this time last year making a tool that used these same files to do some automated price analysis. It was deemed a EULA violation.
tbh I'm concerned that CCP hasn't locked down those data streams to prevent this. Either make it legal and official (and automatic) or do something to disable the functionality (ala Bacon).
From what I understand he's not messing with the client (the executable, DLLs, etc) but instead clear-text files. This is probably how he's getting around the EULA.
He's not editing anything, just reading some clear-text files.
EBANK - Chairman of the Board | www.eve-bank.net
|
ZeusOnRoids
|
Posted - 2009.02.23 09:41:00 -
[9]
this is a great tool, but its usability depends completely on the number of poeple running the uploader. so, come on trader, let it run
|
Daphne Eveningstar
|
Posted - 2009.02.23 09:50:00 -
[10]
Edited by: Daphne Eveningstar on 23/02/2009 09:50:44
Quote: Browsing market order pages will upload order info, and browsing the history graph/table page will upload history info.
That's from the page for their uploader utility
All I can say is wow. I guess Bacon required the debugger to be running didn't it?
If this is stated by CCP to be a legal use of the client cache than let me be the first to offer up a bounty of 2 billion for a local exe version of the same functionality. I want every market lookup I make (across multiple accounts and hosts/clients) stored in a local MSSQL database for quick analysis of both the current picture and the historic data.
|
|
Marcus D'Eriellius
Gallente Honour Bound Sc0rched Earth
|
Posted - 2009.02.23 10:05:00 -
[11]
You mention there will be developer APIs in the future but the documentation already mentions them.
However, querying via API produces invalid XML:
<?xml version='1.0' encoding='utf-8' ?> <evemetrics api_version='1.1'> <?xml version="1.0" encoding="UTF-8"?> <evemetrics api_version="1.0"> <region name="The Forge"> <type name="Tritanium" id="34">4.14710945273632</type> <type name="Pyerite" id="35">3.90247984496124</type> <type name="Mexallon" id="36">24.9396716981132</type>
</region> </evemetrics> </evemetrics>
Secondly, would it be possible to have some method to ignore the outliers in the data, either copy the minQ parameter from Eve-Central or, better yet, have a way of getting the 95% price as the Jitaindex used to calculate?
Great work, look forward to implementing it :)
|
Hexxx
Minmatar
|
Posted - 2009.02.23 10:58:00 -
[12]
Originally by: Marcus D'Eriellius You mention there will be developer APIs in the future but the documentation already mentions them.
However, querying via API produces invalid XML:
<?xml version='1.0' encoding='utf-8' ?> <evemetrics api_version='1.1'> <?xml version="1.0" encoding="UTF-8"?> <evemetrics api_version="1.0"> <region name="The Forge"> <type name="Tritanium" id="34">4.14710945273632</type> <type name="Pyerite" id="35">3.90247984496124</type> <type name="Mexallon" id="36">24.9396716981132</type>
</region> </evemetrics> </evemetrics>
Secondly, would it be possible to have some method to ignore the outliers in the data, either copy the minQ parameter from Eve-Central or, better yet, have a way of getting the 95% price as the Jitaindex used to calculate?
Great work, look forward to implementing it :)
I think you need a Developer Key to use the API for it.
EBANK - Chairman of the Board | www.eve-bank.net
|
Hexxx
Minmatar
|
Posted - 2009.02.23 10:59:00 -
[13]
Originally by: Daphne Eveningstar Edited by: Daphne Eveningstar on 23/02/2009 09:50:44
Quote: Browsing market order pages will upload order info, and browsing the history graph/table page will upload history info.
That's from the page for their uploader utility
All I can say is wow. I guess Bacon required the debugger to be running didn't it?
If this is stated by CCP to be a legal use of the client cache than let me be the first to offer up a bounty of 2 billion for a local exe version of the same functionality. I want every market lookup I make (across multiple accounts and hosts/clients) stored in a local MSSQL database for quick analysis of both the current picture and the historic data.
Why would you want local? Just grab the uploader, upload tons of stuff, and view the information along with all the other updates from all the other people!!
EBANK - Chairman of the Board | www.eve-bank.net
|
Hexxx
Minmatar
|
Posted - 2009.02.23 11:01:00 -
[14]
Originally by: ZeusOnRoids this is a great tool, but its usability depends completely on the number of poeple running the uploader. so, come on trader, let it run
Yes, the number of people increases the value of the utility. Sounds like Network Effects to me! Bottom line, this thing becomes EXPONENTIALLY more valuable the more people download and install the uploader utility.
EBANK - Chairman of the Board | www.eve-bank.net
|
ChiTrade
|
Posted - 2009.02.23 11:27:00 -
[15]
+1 user here
|
Marcus D'Eriellius
Gallente Honour Bound Sc0rched Earth
|
Posted - 2009.02.23 11:51:00 -
[16]
Originally by: Hexxx
Originally by: Marcus D'Eriellius You mention there will be developer APIs in the future but the documentation already mentions them.
However, querying via API produces invalid XML:
<?xml version='1.0' encoding='utf-8' ?> <evemetrics api_version='1.1'> <?xml version="1.0" encoding="UTF-8"?> <evemetrics api_version="1.0"> <region name="The Forge"> <type name="Tritanium" id="34">4.14710945273632</type> <type name="Pyerite" id="35">3.90247984496124</type> <type name="Mexallon" id="36">24.9396716981132</type>
</region> </evemetrics> </evemetrics>
Secondly, would it be possible to have some method to ignore the outliers in the data, either copy the minQ parameter from Eve-Central or, better yet, have a way of getting the 95% price as the Jitaindex used to calculate?
Great work, look forward to implementing it :)
I think you need a Developer Key to use the API for it.
You do. I've got one.
The invalid bit is that it defines the XML schema twice. One at the beginning <?xml version='1.0' encoding='utf-8' ?> then later on <?xml version="1.0" encoding="UTF-8"?>. My parser is choking on this.
|
Highlord Nicolai
Amarr GK inc.
|
Posted - 2009.02.23 12:10:00 -
[17]
I prophecize that given this works the way I think it does, will end trading as we know it. Margins will be ruined, and only the big fish will remain.
Apocalypes is apon us!
I have spoken.. ------------------------------------------------
"Believe you can or cant, either way your right", If you can figure this sentence out, your life will be complete. Thats the truth! |
No Profit
|
Posted - 2009.02.23 12:34:00 -
[18]
Edited by: No Profit on 23/02/2009 12:36:19
Interesting.
Ah bugger, just read this:
Quote: By default resources are permitted 20 calls per hour at absolute most per (application developer) key
That sinks EMMA right off the bat.
|
Kazuo Ishiguro
House of Marbles Zzz
|
Posted - 2009.02.23 12:48:00 -
[19]
Edited by: Kazuo Ishiguro on 23/02/2009 12:47:50
Originally by: Hexxx In terms of restricting the queries on stations or solar systems...markets operate best when they're efficient. Transparency breeds efficiency. So this program seems to operate in the best interests of a transparent/efficient market.
The most useful function isn't the solarsystem specific sell/buy orders but instead the indexes created from all these orders. I noticed that some of them are down right now, but the promise of nearly automatically created indexes is....well....incredibly promising.
Believe it or not, some of us do actually need detailed and up-to-the-minute information. For low-end minerals in particular, location is crucial, as the costs of transportation are significant. This type of API linked into an XML map is one of the fastest ways to get market data into a spreadsheet; the eve-metrics API could (and should) support such requests, as it will attract more users. --- Can't afford that BPO? Look here. 20:1 mineral compression The EVE f@h team |
Kazuo Ishiguro
House of Marbles Zzz
|
Posted - 2009.02.23 12:49:00 -
[20]
Originally by: No Profit Ah bugger, just read this:
Quote: By default resources are permitted 20 calls per hour at absolute most per (application developer) key
That sinks EMMA right off the bat.
No, it just means you have to generate a single query for lots of items at once. You're not limited to a single typeID (or region) per query.
--- Can't afford that BPO? Look here. 20:1 mineral compression The EVE f@h team |
|
Ricdic
Caldari
|
Posted - 2009.02.23 13:21:00 -
[21]
My market metric addon client thingo died on me whilst playing COD4 |
Krathos Morpheus
Gallente Legion Infernal Wildly Inappropriate.
|
Posted - 2009.02.23 13:29:00 -
[22]
Originally by: Hexxx
Originally by: ZeusOnRoids this is a great tool, but its usability depends completely on the number of poeple running the uploader. so, come on trader, let it run
Yes, the number of people increases the value of the utility. Sounds like Network Effects to me! Bottom line, this thing becomes EXPONENTIALLY more valuable the more people download and install the uploader utility.
I discovered this tool some time ago, but discouraged by it having less data than eve-central, I'll give it another try hoping that it gets enough users. This is a great tool. EVE Knowledge |
Verite Rendition
Caldari F.R.E.E. Explorer Wildly Inappropriate.
|
Posted - 2009.02.23 13:39:00 -
[23]
Edited by: Verite Rendition on 23/02/2009 13:38:52 How is it I've never heard of this before? It looks like a great tool.
The website needs a bit of work however. It's not pruning old data (I'm seeing things from 2008) and the inability to select what region you want to check without logging in is a PITA. All the data in the world won't help if your website sucks. ---- FREE Explorer Lead Megalomanic EVE Null-Sec Player Influence Map http://dl1.eve-files.com/media/corp/Veritefw/FWinf |
Hexxx
Minmatar
|
Posted - 2009.02.23 14:05:00 -
[24]
Originally by: Verite Rendition Edited by: Verite Rendition on 23/02/2009 13:38:52 How is it I've never heard of this before? It looks like a great tool.
The website needs a bit of work however. It's not pruning old data (I'm seeing things from 2008) and the inability to select what region you want to check without logging in is a PITA. All the data in the world won't help if your website sucks.
I'll shoot a note to Ix Forres about the old data issue. The requirement to login may be a bit PITA, but the cost is pretty low considering all the data you get.
And to a previous point; the utility of EVE Metrics increases exponentially the more users are on it. Hopefully we can drive increased adoption of the uploader utility to really give this tool a boost.
I'm running the uploader in the background everytime I start up my computer.
EBANK - Chairman of the Board | www.eve-bank.net
|
Muscaat
Eve University Ivy League
|
Posted - 2009.02.23 14:06:00 -
[25]
Originally by: Hexxx The most useful function isn't the solarsystem specific sell/buy orders but instead the indexes created from all these orders. I noticed that some of them are down right now, but the promise of nearly automatically created indexes is....well....incredibly promising.
Indexes like those could be used as the foundation for futures contracts on minerals for example. This is just one of many possible uses.
I'm keeping an eye on it as a second data source for EVE Markets - I've not yet found the time to have a play (nor a headscratch on how to integrate it).
|
Hexxx
Minmatar
|
Posted - 2009.02.23 14:16:00 -
[26]
Originally by: Muscaat
Originally by: Hexxx The most useful function isn't the solarsystem specific sell/buy orders but instead the indexes created from all these orders. I noticed that some of them are down right now, but the promise of nearly automatically created indexes is....well....incredibly promising.
Indexes like those could be used as the foundation for futures contracts on minerals for example. This is just one of many possible uses.
I'm keeping an eye on it as a second data source for EVE Markets - I've not yet found the time to have a play (nor a headscratch on how to integrate it).
EVE Markets = EVE Central.
They get their data from the EVE Central API so it suffers from the same problems EVE Central has; that is a lack of data due to manual uploads and data integrity (false export records).
EBANK - Chairman of the Board | www.eve-bank.net
|
Nick Turing
|
Posted - 2009.02.23 14:55:00 -
[27]
Edited by: Nick Turing on 23/02/2009 14:57:04
Originally by: No Profit Edited by: No Profit on 23/02/2009 12:36:19
Interesting.
Ah bugger, just read this:
Quote: By default resources are permitted 20 calls per hour at absolute most per (application developer) key
That sinks EMMA right off the bat.
That's just absurd. If they find that sort of limit to be necessary, I'm fairly sure they're Doin It Wrong. Any reasonable query ought to be indexable, and thus able to be answered efficiently, so much much higher limits should be practical.
That said, if they want to start offering dumps like eve-central do (eg, an email or other feed of all updates), I'd be more than happy to put up a query interface with much more reasonable limits.
Edit: Regarding the 'security' aspect: At best, it can only be obfuscation. Given that they don't have an authoritative source to check against, and given that the user has complete control over the client, there's literally no way to actually prevent people from forging data. You can make it extremely difficult, but that's it.
One method that would be practical, though: Only displaying data that has been independently uploaded by at least two sources. That would certainly raise the bar for anyone wanting to inject false data.
|
Hel O'Ween
Aliastra
|
Posted - 2009.02.23 15:32:00 -
[28]
Originally by: Kazuo Ishiguro
Originally by: No Profit Ah bugger, just read this:
Quote: By default resources are permitted 20 calls per hour at absolute most per (application developer) key
That sinks EMMA right off the bat.
No, it just means you have to generate a single query for lots of items at once. You're not limited to a single typeID (or region) per query.
And how should we know if and when one of our users clicks on the button to retrieve data?
As it reads, the limit is per developer key, per hour. This means that if *one* user does 20 lookups, *all* other users are out of business for the next one hour.
This for sure can't be the intended outcome. -- EVEWalletAware - an offline wallet manager |
Daphne Eveningstar
|
Posted - 2009.02.23 16:53:00 -
[29]
Originally by: Hexxx
Why would you want local? Just grab the uploader, upload tons of stuff, and view the information along with all the other updates from all the other people!!
Well ya see, I didn't build up a billion a day business to go leaking my data all over some guy's website. Same reason I turn off Eve-Central lookups in EMMA. My data is my data. The hosts of these sites don't get to datamine my lookups.
But having a local history of all my lookups and what cost how much where? Not having to jab the Export button each time to do it? Almost priceless to me!
My offer stands.
|
Clair Bear
Perkone
|
Posted - 2009.02.23 17:13:00 -
[30]
Originally by: Daphne Eveningstar [ But having a local history of all my lookups and what cost how much where? Not having to jab the Export button each time to do it? Almost priceless to me!
Exactly this. There are only 6 stations whose market data I care about, and I can populate my own data with the alts I have installed in those regions.
Now that I know this is possible and how to do it I think I'll give the offline sc****r implementation a shot. Unfortunately it's worth a lot more than 2B not to distribute it once done. =)
|
|
|
|
|
Pages: [1] 2 3 4 5 6 7 8 :: one page |
First page | Previous page | Next page | Last page |