| Pages: [1] 2 3 4 5 6 7 :: one page |
|
|
| Author |
Thread Statistics | Show CCP posts - 14 post(s) |
|
CCP Fallout
   |
Posted - 2011.08.30 14:06:00 -
[1]
Attention market addict! Dr.EyjoG's newest dev blog gives us information about a new initiative: providing historic market data to capsuleers. This is a test run, so please do share your feedback with both Research & Statistics and Team Sleeper Cell in this thread.
Fallout
Associate Community Manager
CCP Hf, EVE Online
Contact us |
|
Gnulpie
Minmatar
Miner Tech
|
Posted - 2011.08.30 14:08:00 -
[2]
Just awesome! |
Squizz Caphinator
Woopatang
|
Posted - 2011.08.30 14:09:00 -
[3]
yes! yes! yes!
--
Eve Who
EveChatter
KillWhore Killboard |
Femaref
Armageddon Day
WE FORM VOLTRON
|
Posted - 2011.08.30 14:09:00 -
[4]
omg, this is epic.
From the two, I prefer csv of course, but others would be better.
|
Lost Hamster
Hamster Holding Corp
|
Posted - 2011.08.30 14:11:00 -
[5]
Nice one. Please keep the SQL version. I think that's the easiest to use.
--------------------------------------------------------------------------------------------
Shields are like pants, they're supposed to come off. Armor is like the condom once its gone ur ****ed |
Mors Magne
Astral Adventure
|
Posted - 2011.08.30 14:12:00 -
[6]
Quote: "Fly safe but remember to destroy stuff - it is good for the EVE economy!"
|
Femaref
Armageddon Day
WE FORM VOLTRON
|
Posted - 2011.08.30 14:16:00 -
[7]
 Originally by: Lost Hamster
Nice one. Please keep the SQL version. I think that's the easiest to use.
It's not sql, it's mssql backup. Put another way: PITA.
|
Myxx
Atropos Group
|
Posted - 2011.08.30 14:17:00 -
[8]
I prefer Csv.
is it possible to keep both for those of us who like one but not the other?
--
Originally by: CCP Explorer
(and if you guys would also stop using Drakes it would be really appreciated, kthxbye).
Originally by: Tom Gerard
Then again... I am a moron.
|
Arkady Sadik
Minmatar
Electus Matari
|
Posted - 2011.08.30 14:21:00 -
[9]
Thank you for this data. It's very useful. So useful, in fact, that we already collect it :o)
When you view the market history in-game, the data is written to a cache file. As cache reading is legal, we have a program that reads the cache file and uploads that data to our server. We also have a JavaScript tool for the IGB that opens the market window for a number of "interesting" types. This used to be provided by eve-metrics, but they closed down. :-( We have most of the data from eve-metrics, with some item histories going back as far as May 2008.
What we use it for
We generate a 7-day "index" price for materials in the regions we operate in. We buy materials below that index, calculate our production cost based on that index, and sell products above that calculated production cost. This allows us to easily cooperate between multiple players (some buy, some sell) without complicated record keeping.
The resulting index is public and can be found here:
http://www.electusmatari.com/gmi/
What format would be useful?
Well, it ends up in an SQL table here. Getting it in SQL directly is good. But to translate .bak files into SQL requires some MS program, and for the official DB dump, we actually rely on a 3rd party (zofu for president) to translate it to PostgreSQL format. I suspect a lot of others use MySQL. MSSQL backup format is pretty much unusable for most people, and CSV is easier to translate into something sensible.
Have you considered providing an API interface instead of file dumps?
|
|
Chribba
Otherworld Enterprises
Otherworld Empire
|
Posted - 2011.08.30 14:22:00 -
[10]
There we go.
Secure 3rd party service | in-game 'Holy Veldspar' Now /w voice |
|
|
|
Dierdra Vaal
Caldari
Veto.
Veto Corp
|
Posted - 2011.08.30 14:26:00 -
[11]
I think this is a great initiative. I hope the developers are also looking at services like eve-central which are providing this sort of information already :)
Veto #205
* * *
Director Emeritus at EVE University
* * *
CSM1 delegate, CSM3 chairman and CSM5 vice-chairman
|
rofflesausage
|
Posted - 2011.08.30 14:32:00 -
[12]
Edited by: rofflesausage on 30/08/2011 14:32:34
Out of the two formats listed, CSV.
.bak files are a pain to work with in my experience.
|
Rees Noturana
Red Rock Mining Company
|
Posted - 2011.08.30 14:36:00 -
[13]
CSV means simpler access to a lot of different programming languages.
MSSQL backup format means evil MS land, angry, angry, flame war. I think we've had enough of those lately. I don't need the added hassle of converting the data into something simple and readable before I run my script on it. Open standards beats weird MS garbage any day.
|
Meissa Anunthiel
|
Posted - 2011.08.30 14:39:00 -
[14]
Wonderful,
This has been a long time coming but is very very welcome.
-----
Member of CSM 2, 3, 4, 5 and 6. My blog
|
Jareck Hunter
Rubicon Legion
|
Posted - 2011.08.30 14:47:00 -
[15]
About the Delay.
I think i'm not only speaking for myself here, but most of the time, i'm not interrested in the price of tritanium 3 months or 2 years ago, if i want to feed my sheets for production and other things, i want to know how much something cost today, maybe yesterday or last week, so i can work with those numbers.
I think sites like eve-metrics or eve-central, got used a lot cause they provided actuall data, that could be read out automaticly.
So an update every downtime for items that get traded a lot, every 2-3 days things that get traded sometimes and maybe every week for low transaction items could be a way to go.
If you could archive something like that, that would be nice.
-------------------------------------------------
Sorry for my bad english^^ |
Marcel Devereux
Aideron Robotics
|
Posted - 2011.08.30 14:50:00 -
[16]
Any text format (CSV, JSON, XML, etc..) over a MSSQL backup. If you go with MSSQL backup we have to rely on someone with a copy of MSSQL to do the conversion for us. Seeing as you already have it please do it for us. Thanks!
|
Face ofBoe
|
Posted - 2011.08.30 15:01:00 -
[17]
Anyone who can restore an mssql bak can just as easily consume a csv. Just provide the csv and we can take it from there.
|
Mechael
Minmatar
Helhest 1st Prospectors' Collective
|
Posted - 2011.08.30 15:04:00 -
[18]
.csv format would be greatly preferred. MSSQL is a pain in the ass. I'd like to request JSON as I tend to code in node.js and mongodb, but that wasn't an option.
As to a reasonable interval, I'd like to request as often as possible. An API would be preferred, actually. Please, don't make us wait longer than one week between updates.
This will be a huge, huge help and very powerful tool. With what tools we players currently have, in order to get complete market data we have to have someone (or multiple people) manually look up every single item in every single region so that we can read the cache files. It's not very feasible, and very obviously a large headache.
|
|
CCP Stillman
|
Posted - 2011.08.30 15:08:00 -
[19]
Originally by: Dierdra Vaal
As such, is it possible that this will be upgraded to an API call? Where, similar to eve-central, I can query one or more itemIds (as well as location parameters, time parameters, etc) and get their up-to-date price statistics back?
Originally by: Arkady Sadik
Have you considered providing an API interface instead of file dumps?
It's no coincidence that the API team has been working with Dr. EyjoG and his team on this. Our ultimate goal is to expose this through an API with more up to date data. This release is to gather feedback and find out how you guys can best benefit from a such potential API.
|
|
Avall
|
Posted - 2011.08.30 15:11:00 -
[20]
This is awesome!
1) Any text format is preferable to .bak. Personally I prefer XML.
2) Please please please make this an API call, per item. This way it could be used in web apps or mobile apps live, without having to download a large file or a lot of data.
3) Delay should be in the range of hours, absolute maximum is one day. Otherwise the data won't be used very much since more up to date data is available from crowd sourced cache reading repositories like eve central and others.
Looking forward to integrating this into Leaky Capsule, my iPhone app (yes, that was a shameless plug :)
|
|
|
Dierdra Vaal
Caldari
Veto.
Veto Corp
|
Posted - 2011.08.30 15:16:00 -
[21]
Edited by: Dierdra Vaal on 30/08/2011 15:15:54
Originally by: CCP Stillman
Originally by: Dierdra Vaal
As such, is it possible that this will be upgraded to an API call? Where, similar to eve-central, I can query one or more itemIds (as well as location parameters, time parameters, etc) and get their up-to-date price statistics back?
Originally by: Arkady Sadik
Have you considered providing an API interface instead of file dumps?
It's no coincidence that the API team has been working with Dr. EyjoG and his team on this. Our ultimate goal is to expose this through an API with more up to date data. This release is to gather feedback and find out how you guys can best benefit from a such potential API.
If you do I'll buy you so much beer at fanfest <3
For my project the eve-central service is sufficient. Min, Max, Avg and Median prices per item, while allowing filtering per region or time period and allowing me to query multiple itemIDs at once.
The biggest thing I'm missing is similar price statistics for faction and deadspace items (basically anything that cannot be traded on the market), but I suspect thats a whole different set of problems.
Veto #205
* * *
Director Emeritus at EVE University
* * *
CSM1 delegate, CSM3 chairman and CSM5 vice-chairman
|
Vanessa Vansen
|
Posted - 2011.08.30 15:21:00 -
[22]
Edited by: Vanessa Vansen on 30/08/2011 15:21:35
About the format ...
please use the most general format, i.e. CSV over MSSQL backup
however, I would vote for xml
|
|
CCP Stillman
|
Posted - 2011.08.30 15:23:00 -
[23]
Originally by: Avall
Delay should be in the range of hours, absolute maximum is one day. Otherwise the data won't be used very much since more up to date data is available from crowd sourced cache reading repositories like eve central and others.
I just want to address this one already.
From a technical point of view, this is completely impossible I'm afraid. The amount of data it takes to generate this sort of data is absolutely huge. There has to be a delay, because of that. And it has to be measured in days.
From a design point of view, there's also the concern about people getting a huge advantage from getting this sort of information.
So this leaves an interesting question I'd be curious to hear. Say that we developed an API for this, which would give you the data for any set of regionID and typeID for 7 days ago. How useful would that be?
|
|
Sgt Napalm
Veto.
Veto Corp
|
Posted - 2011.08.30 15:23:00 -
[24]
How much will the subscription based license cost in Aurum?
|
Arkady Sadik
Minmatar
Electus Matari
|
Posted - 2011.08.30 15:24:00 -
[25]
Well, as mentioned, I think the most interesting data is roughly the last 28 days at most. There are uses for older data, but they're rare. For example, I have used old market data in the past to figure out which patch introduced a change that caused a significant price changes (there's a nice bulge in Tritanium which lasted for a year and starts and ends at specific patches, say).
That is, most applications will be interested only in the last few days of data. Our own application would do a daily query of about 380 types (if this is API-queryable, maybe even all types with a marketgroupid) once per day to get the last 7 days of data for four regions, nothing else.
|
|
CCP Stillman
|
Posted - 2011.08.30 15:24:00 -
[26]
Originally by: Dierdra Vaal
The biggest thing I'm missing is similar price statistics for faction and deadspace items (basically anything that cannot be traded on the market), but I suspect thats a whole different set of problems.
I'm afraid that's outside the scope of this project, and is a very difficult problem in general. Sorry.
|
|
Eve Industrialist
|
Posted - 2011.08.30 15:25:00 -
[27]
Looking forward to this. Api similar in form to eve cental xml would be best. Csv is doable as well. My first instict was to say sql but then I realized I am not likely to keep it up to date as such. A delay or csv only release would keep the eve central guys in business... tho I agree a delay longer than a day on mineral prices makes this data not as good as eve central.
|
Dierdra Vaal
Caldari
Veto.
Veto Corp
|
Posted - 2011.08.30 15:25:00 -
[28]
Originally by: CCP Stillman
So this leaves an interesting question I'd be curious to hear. Say that we developed an API for this, which would give you the data for any set of regionID and typeID for 7 days ago. How useful would that be?
Given how quickly markets can change, I think there will be many cases where 7 days will be too long a delay. Could it be a daily updated thing?
Veto #205
* * *
Director Emeritus at EVE University
* * *
CSM1 delegate, CSM3 chairman and CSM5 vice-chairman
|
Nahkep Narmelion
Gallente
CALIMA COLLABORATIVE
Atrox Urbanis Respublique Abundatia
|
Posted - 2011.08.30 15:26:00 -
[29]
CSV is preferred, but the file is too big for Excel. I'll likely use it in SAS for analysis, and csv files can be directly exported into SAS. But my guess is not many people will have access to something as powerful as SAS, so if the data could be broken up it might be even more useful.
|
Arkady Sadik
Minmatar
Electus Matari
|
Posted - 2011.08.30 15:27:00 -
[30]
Stop posting while I post! <.<
Originally by: CCP Stillman
So this leaves an interesting question I'd be curious to hear. Say that we developed an API for this, which would give you the data for any set of regionID and typeID for 7 days ago. How useful would that be?
For our uses, not useful at all. We can get up-to-date data with our existing toolchain, and we use the data from the last 7 days. Anything before that is interesting mostly for historical reasons, or for items that are traded very rarely.
|
|
|
|
|
| |
|
| Pages: [1] 2 3 4 5 6 7 :: one page |
| First page | Previous page | Next page | Last page |