EVE Search - EVE data export, rmr edition

Check My IP Information

All Channels

EVE General Discussion

EVE data export, rmr edition

» Click here to find additional results for this topic using Google

Monitor this thread via RSS [?]

Pages: 1 2 3 4 5 6 7 8 9 10 [11] 12 13 .. 13 :: one page

Author	Thread Statistics \| Show CCP posts - 12 post(s)
Amarr Menza Amarr Viziam	Posted - 2006.09.20 03:11:00 - [301] Hi What you have written has already been covered earlier. It is better if you use something like : mysql < dbo.sql this will batch import in the SQL in the fastest possible way, do this through command line and skip any tools. Though it is better to actually do through the CSV as you will still get MySQL errors if the values are not present in the SQL data. Check out my Wiki as I am doing exports of the database in MySQL 5 format that should provoke some debate at lease on how the indexing and field types should be set up. ------ EVE Online Devs Wiki Triangle Solutions ------
Amarr Menza Amarr Viziam	Posted - 2006.09.20 03:13:00 - [302] Originally by: Matalino Originally by: Amarr Menza Can we have our own Forum - Called 'Game Data Developers' or something. Many people that are trying to develop stuff for eve will post in this forum. "A forum for threads with words like: deterministic, continuations, generators, extrapolation, vertex shader, bi-directional reflectance distribution, etc." Would be nice if we had our own specific forum shouldnt take long to set up that deals with the Data dumps exclusively. ------ EVE Online Devs Wiki Triangle Solutions ------

Eldo Davip Forum Moderator Interstellar Services Department	Posted - 2006.09.24 17:32:00 - [303] Originally by: Amarr Menza T20 Can we have our own Forum - Called 'Game Data Developers' or something. This thread seems to contain hundreds of posts that can be answered by people correctly if they were properly organised. This way I can also find the posts by yourself within this mess. THere is one. Its called the Eve Technology Lab Email Us (Report a bad post) \| Forum Rules - Read 'em! \| Website

Amarr Menza Amarr Freelance Co Op	Posted - 2006.09.26 06:44:00 - [304] Hi, The EVE data wiki is now coming on great quite a few of the Tables have been optimised and outputted in MySQL 5, I am now adding optimised CSV downloads and will add XML and SQL very soon. Plus a decent schema of the DB. Looking for support on this so if anyone wants to help out click below log in and get stuck in. EVE Data Help Cheers Amarr ------ EVE Online Devs Wiki Triangle Solutions ------
Raquel Smith Caldari Ferengi Commerce Authority	Posted - 2006.09.26 08:41:00 - [305] Originally by: Amarr Menza Hi, The EVE data wiki is now coming on great quite a few of the Tables have been optimised and outputted in MySQL 5, I am now adding optimised CSV downloads and will add XML and SQL very soon. Plus a decent schema of the DB. Looking for support on this so if anyone wants to help out click below log in and get stuck in. EVE Data Help Cheers Amarr Not really sure what the goal of the project is. There is already a guide to migrating the TSQL to MySQL and Postgres.
Amarr Menza Amarr Freelance Co Op	Posted - 2006.09.26 09:30:00 - [306] Hi Ok show me a link to this guide please. The goal is to not provide a 'Guide' but to provide updated optimised MySQL / Postgres / any DB schemas and data exports. As you can see from EVE MySQL Exports This is starting to happen now, I find the SQL generated by CCP is not indexed correctly, the fields types are obviously all a miss - so I am simply correcting all these importing in the CSV and then exporting this out again in an optimised fashion. ------ EVE Online Devs Wiki Triangle Solutions ------
Raquel Smith Caldari Ferengi Commerce Authority	Posted - 2006.09.26 09:35:00 - [307] Originally by: Amarr Menza Hi Ok show me a link to this guide please. The goal is to not provide a 'Guide' but to provide updated optimised MySQL / Postgres / any DB schemas and data exports. As you can see from EVE MySQL Exports This is starting to happen now, I find the SQL generated by CCP is not indexed correctly, the fields types are obviously all a miss - so I am simply correcting all these importing in the CSV and then exporting this out again in an optimised fashion. The guide exists in this thread! I agree that CCP's schema is rubbish. I spent weeks optimising it for my own needs.
Amarr Menza Amarr Freelance Co Op	Posted - 2006.09.26 14:51:00 - [308] And there lies the problem I have been through this thread with a fine tooth comb and a lot of the ideas are laborious and sometimes incorrect, they pay no attention to what makes a DB tick the indexing and field types, they just seem to care about getting the Data loaded up. I am sure there are dozens of DBs around that are very poorly created. If you have done the porting do you want to share your schema? What DB do you use? MySQL? The point of the collaboration is for everyone to find out the best way to do things, and to create some debate. Instead of everyone doing the same thing 50% lets try and work together and make it excellent, it will save a lot of peoples time and allow some really good tools to be developed faster. ------ EVE Online Devs Wiki Triangle Solutions ------
Jean Toroque InterSec	Posted - 2006.09.30 21:31:00 - [309] I've seen the question asked a few times, but not answered yet: where's the data concerning the amount of each mineral yielded when refining ore? I understand that you can get the requirements for manufacturing from TL2MaterialsForTypeWithActivity (winner: Academy Award for Worst Table Name in a Database), and that the requirements for manufacturing are normally the same as the output from reprocessing. But that doesn't help with ore, because there is no blueprint type for any of the ores, and there is no data in TL2EtCetera for any of the ore types. For example, Veldspar is typeid 1230, and there are no rows in TL2 where typeid = 1230, for any activity. The information is shown in a perfectly intelligible format in the Item Database ... seems strange that it wouldn't be in the data dump. Does anyone have The Clue on this one?
Pantheon Lea Farmer Boyz	Posted - 2006.09.30 22:02:00 - [310] Edited by: Pantheon Lea on 30/09/2006 22:02:44 http://www.eve-online.com/itemdatabase/manufactureresearch/ore/default.asp Pantheon Lea

Jean Toroque InterSec	Posted - 2006.10.01 00:40:00 - [311] Pantheon Lea, I get that it's in the Item Database. Refer to my post above: Originally by: "Jean Toroque" The information is shown in a perfectly intelligible format in the Item Database ... seems strange that it wouldn't be in the data dump. My question is whether the info is in the data dump, and if it's not, how come?
Pantheon Lea Farmer Boyz	Posted - 2006.10.12 22:22:00 - [312] Originally by: Jean Toroque My question is whether the info is in the data dump, and if it's not, how come? It's not in the data dump, probably because it's not stored in the database, there is really no need to store it in the database because it's very static and also used very often, it's probably hard coded in the server, client or both. Pantheon Lea
FlyBaby	Posted - 2006.10.16 22:38:00 - [313] Originally by: Utoxin After careful inspection of the currently available CSV and SQL data files, it appears that some data was either intentionally or accidentally removed when they were last updated. The base price data for MANY modules and ships is 0 ISK. After speaking a bit with the EVEmon developer on their forums, it appears that they originally got their data /from/ the RMR data exports, but it is definately not present in the files now. Can anyone else confirm the removal of this information? this is something that interrest me also. is there anybody at all that has an awnser on this? and if possible a solution? note: I also think we need a separate forum section for application development. this thread is getting bigger and bigger, and harder to find awnsers/questions back without reading it all over again. and people start double posting question.
Tor Manx	Posted - 2006.10.27 03:19:00 - [314] Been playing with the data a bit today, and can't seem to find where I'd retrieve the "Mining Amount" total's for the mining lasers. Anyone know off hand how I'd go about pulling that information out?
Vessper Indicium Technologies	Posted - 2006.10.27 12:58:00 - [315] Originally by: Tor Manx Been playing with the data a bit today, and can't seem to find where I'd retrieve the "Mining Amount" total's for the mining lasers. Anyone know off hand how I'd go about pulling that information out? The "Mining Amount" is Attribute Type number 77. If you search for this in the "attributeID" column of the invTypeAttributes table, it will give you the item IDs and mining amount of all the items with this attribute, including mining lasers and mining drones.
Tor Manx	Posted - 2006.10.28 05:01:00 - [316] Originally by: Vessper The "Mining Amount" is Attribute Type number 77. If you search for this in the "attributeID" column of the invTypeAttributes table, it will give you the item IDs and mining amount of all the items with this attribute, including mining lasers and mining drones. Strangely enough, it turns out my query was correct and there are just more than one "near" identical rows for many of the items. For instance, with this query I get multiple listings for each object due to the fact there are often many items with the same type and attribute id's. SELECT * FROM Dbo_invTypes it, Dbo_dgmTypeAttributes dta where it.typeID = dta.typeID and dta.attributeID = 77 and it.groupID = 54 So I ran this query hoping it would narrow it down, but still no luck. SELECT * FROM Dbo_invTypes AS it, Dbo_dgmTypeAttributes AS dta WHERE it.typeID=dta.typeID AND it.typeID in (3,5,11,12,21,22,25,482,483) And dta.attributeID=77 and it.groupID = 54 Not really getting their data model 100% yet. But I'll keep working on it.
Vessper Indicium Technologies	Posted - 2006.10.28 09:49:00 - [317] Originally by: Tor Manx Strangely enough, it turns out my query was correct and there are just more than one "near" identical rows for many of the items. For instance, with this query I get multiple listings for each object due to the fact there are often many items with the same type and attribute id's. I use the following query and it seems to give me a list of all items with a mining amount (no duplications either): SELECT invTypes.typeName, dgmTypeAttributes.valueInt FROM invTypes INNER JOIN (dgmAttributeTypes INNER JOIN dgmTypeAttributes ON dgmAttributeTypes.attributeID = dgmTypeAttributes.attributeID) ON invTypes.typeID = dgmTypeAttributes.typeID WHERE (((dgmAttributeTypes.attributeID)=77)); Unfortunately, I'm not very good with SQL so I don't know if this is the best way to do it...but it works.
Fubar Amarr Eagle Eye Inc.	Posted - 2006.10.28 17:08:00 - [318] Originally by: Vessper SELECT invTypes.typeName, dgmTypeAttributes.valueInt FROM invTypes INNER JOIN (dgmAttributeTypes INNER JOIN dgmTypeAttributes ON dgmAttributeTypes.attributeID = dgmTypeAttributes.attributeID) ON invTypes.typeID = dgmTypeAttributes.typeID WHERE (((dgmAttributeTypes.attributeID)=77)); Unfortunately, I'm not very good with SQL so I don't know if this is the best way to do it...but it works. You can simplify the statement to.. SELECT invTypes.typeName, dgmTypeAttributes.valueInt FROM invTypes INNER JOIN dgmAttributeTypes ON invTypes.typeID = dgmTypeAttributes.typeID WHERE (((dgmAttributeTypes.attributeID)=77));
Vessper Indicium Technologies	Posted - 2006.10.29 09:19:00 - [319] Originally by: Fubar Originally by: Vessper SELECT invTypes.typeName, dgmTypeAttributes.valueInt FROM invTypes INNER JOIN (dgmAttributeTypes INNER JOIN dgmTypeAttributes ON dgmAttributeTypes.attributeID = dgmTypeAttributes.attributeID) ON invTypes.typeID = dgmTypeAttributes.typeID WHERE (((dgmAttributeTypes.attributeID)=77)); Unfortunately, I'm not very good with SQL so I don't know if this is the best way to do it...but it works. You can simplify the statement to.. SELECT invTypes.typeName, dgmTypeAttributes.valueInt FROM invTypes INNER JOIN dgmAttributeTypes ON invTypes.typeID = dgmTypeAttributes.typeID WHERE (((dgmAttributeTypes.attributeID)=77)); Much neater, thanks
Tor Manx	Posted - 2006.10.31 00:56:00 - [320] Thanks for your idea's. I kinda ran into a problem with the csv export though. It seems that some of the csv data has overrun their rows (or so it seems). In the most recent csv extract that I downloaded, around like 160 everyting goes to hell because a blank line and unformatted text follows it. I'll re-download and see if it's any better.

Tor Manx	Posted - 2006.10.31 01:07:00 - [321] Should state that the export file in question is the dbo_invTypes. Definitely looks like a bad export to me, but if someone else could check it would be appreciated.
Aelena Thraant Shadows of the Dead Xelas Alliance	Posted - 2006.11.02 21:53:00 - [322] Is there a list of what resources a BP uses without any ME research? I couldn't find it in the RMR data export, but was wondering if I missed the info from somewhere.
Vessper Indicium Technologies	Posted - 2006.11.03 00:38:00 - [323] Originally by: Aelena Thraant Is there a list of what resources a BP uses without any ME research? I couldn't find it in the RMR data export, but was wondering if I missed the info from somewhere. Unfortunately not. You will have to use some math to calculate the wastage and add it on to the "perfect" production figures that are given in the file export.
Irongut M'8'S	Posted - 2006.11.03 12:36:00 - [324] Originally by: Tor Manx Should state that the export file in question is the dbo_invTypes. Definitely looks like a bad export to me, but if someone else could check it would be appreciated. Yeah that file is totally screwed. There are a few other files with the same problem but they're all small enough to manually fix them. I'm slowly working on fixing dbo_invTypes but since there are 16,000+ items it's going to take forever.
Vessper Indicium Technologies	Posted - 2006.11.03 16:44:00 - [325] Originally by: Irongut Originally by: Tor Manx Should state that the export file in question is the dbo_invTypes. Definitely looks like a bad export to me, but if someone else could check it would be appreciated. Yeah that file is totally screwed. There are a few other files with the same problem but they're all small enough to manually fix them. I'm slowly working on fixing dbo_invTypes but since there are 16,000+ items it's going to take forever. I couldn't get on with the original CSV files that CCP provided so I've reworked the SQL export files to produce a group of (what seems to be) decent CSV files. If anyone is interested, I'd be happy to share the whole collection. I've even added mineral/ore data into the TL2Materials file
Lufio II Amarr Marangrio Space Services Inc.	Posted - 2006.11.03 18:15:00 - [326] Edited by: Lufio II on 03/11/2006 18:17:33 Originally by: Irongut Originally by: Tor Manx Should state that the export file in question is the dbo_invTypes. Definitely looks like a bad export to me, but if someone else could check it would be appreciated. Yeah that file is totally screwed. There are a few other files with the same problem but they're all small enough to manually fix them. I'm slowly working on fixing dbo_invTypes but since there are 16,000+ items it's going to take forever. I used textpad and worked my way into regular expressions. I played around a bit and finally managed to cough a regexp up which eventually fixed all existing problems after doing a replace with it. The problem is that some Item descriptions contain line breaks, and this breaks the CSV-Format for almost all of the freely available CSV-Imports, heck, even for most of the professional ones (they always assume that one record is one line, and if a description contains a line break, that usually ends a line prematurely and breaks that specific record ;) ) Sadly I don't remember the regular expression I was using right now else I'd share it with you...
Camladar	Posted - 2006.11.12 20:50:00 - [327] staStations is missing a lot of stations? There definitely seems to be stations missing from the staStations table. I went as far as running the stationID's in my market logs through the table manually and and there's a lot that simply are not in there. Any workarounds, ideas, suggestions, or new data for this info?

t20	Posted - 2006.11.13 01:15:00 - [328] You'll all get the new data when Revelations (Kali) hits the deck.

Gigi Barbagrigia Latent Appliance Fetishists	Posted - 2006.11.13 07:00:00 - [329] I seriously hope somebody remembered and got you a drink or three at FF.
Raquel Smith Caldari Ferengi Commerce Authority	Posted - 2006.11.13 07:48:00 - [330] Originally by: t20 You'll all get the new data when Revelations (Kali) hits the deck. You, sir, are my hero.



Pages: 1 2 3 4 5 6 7 8 9 10 [11] 12 13 .. 13 :: one page
First page \| Previous page \| Next page \| Last page

COPYRIGHT NOTICE
EVE Online, the EVE logo, EVE and all associated logos and designs are the intellectual property of CCP hf. All artwork, screenshots, characters, vehicles, storylines, world facts or other recognizable features of the intellectual property relating to these trademarks are likewise the intellectual property of CCP hf. EVE Online and the EVE logo are the registered trademarks of CCP hf. All rights are reserved worldwide. All other trademarks are the property of their respective owners. CCP hf. has granted permission to EVE-Search.com to use EVE Online and all associated logos and designs for promotional and information purposes on its website but does not endorse, and is not in any way affiliated with, EVE-Search.com. CCP is in no way responsible for the content on or functioning of this website, nor can it be liable for any damage arising from the use of this website.