Pages: [1] 2 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 9 post(s) |
|
CCP Stillman
|
Posted - 2010.12.08 12:22:00 -
[1]
HereÆs the Static Data Dump and Image Dump for Incursion 1.0.0
Static Data Dump: http://content.eveonline.com/data/Incursion_1.0.0_37398_db.7z Image dump: http://content.eveonline.com/data/Incursion_1.0.0-imgs.7z
Enjoy!
|
|
Ilyk Halibut
Blackwater USA Inc. Random-Violence
|
Posted - 2010.12.08 17:19:00 -
[2]
Wow, that was quick.
Any general idea of what's new/different?
|
Hel O'Ween
Men On A Mission
|
Posted - 2010.12.08 17:25:00 -
[3]
Originally by: Ilyk Halibut Any general idea of what's new/different?
At minimum: - Noctis (+BPO) - Revamped market groups -- EVEWalletAware - an offline wallet manager |
Desmont McCallock
|
Posted - 2010.12.08 18:07:00 -
[4]
Has anybody else problem to open the Image dumb zip file?
No matter how many times I dl it and with whatever way (Save as, GetRight, via Chrome), I can't get it to open.
Corrupt file?
|
derivativo
|
Posted - 2010.12.08 18:17:00 -
[5]
Originally by: Desmont McCallock Has anybody else problem to open the Image dumb zip file?
No matter how many times I dl it and with whatever way (Save as, GetRight, via Chrome), I can't get it to open.
Corrupt file?
What software are you using to decompress the file? Everything worked fine here using 7-zip.
|
Desmont McCallock
|
Posted - 2010.12.08 18:21:00 -
[6]
Funny as it sounds I use 7zip and alt WinRar.
Static Data Dump caused me similar issues but after dl it several times I managed to get a good copy.
|
Desmont McCallock
|
Posted - 2010.12.08 20:07:00 -
[7]
Looks like CCP fixed it.
(No word though, you moles ).
|
Catari Taga
Centre Of Attention Middle of Nowhere
|
Posted - 2010.12.09 00:04:00 -
[8]
Edited by: Catari Taga on 09/12/2010 00:05:51
Originally by: Desmont McCallock Looks like CCP fixed it.
(No word though, you moles ).
I downloaded the image dump 2 days ago already from the community toolkit page and my copy has no issues, so I'd double check my local RAM/HDD/etc. for something that might cause data corruption if I was you.
PS: Thanks for the dump. I thought the official name was to be changed to Static Data Export btw. ;) --
|
Desmont McCallock
|
Posted - 2010.12.09 07:55:00 -
[9]
Edited by: Desmont McCallock on 09/12/2010 07:55:53 TBTH with you, I too had DLed the image dumb as soon as Hanathor mentioned that they are available at toolkit and it was ok. SDD at that time was also diff as it had 2 db copies in it.
So once CCP Stillman made this thread, I re-dl them to see if they made any changes. And they did. But the image dumb zip file was corrupt as I mentioned in my OP.
So I reported it here and after a couple of hours, I retried to dl it and it opened up fine.
I'm no "yesterday's" guy (been dealing with comp about 30 years now and being an EVEMon senior Dev for over a year now) so I can destinguish when things go wrong and at what side.
Thanx though for your input Taga.
|
Lukas Rox
Torchwood Archive
|
Posted - 2010.12.09 08:49:00 -
[10]
There is no usual Jercy's conversion yet, so I tried to make one myself:
MySQL Incursion 1.0.7z
It's my first attempt, so I dont guarantee it's complete, it should import no problem though. ---
Faction Ships Listing | Lukas Rox |
|
Nikolai Kondratiev
Sphere Design Inc.
|
Posted - 2010.12.09 14:51:00 -
[11]
will give out a try to that MySQL version a bit later today _ WTS Capital BPOs |
derivativo
|
Posted - 2010.12.09 18:19:00 -
[12]
Why CCP doesn't release the data dump in csv format? Would be _so much_ better! I use Linux and consider waiting for some MySQL/SQLite dump frustrating, as I still need to further export that to PostgreSQL.
With an open format everyone would benefit. Could CCP say something about this, please?
|
Desmont McCallock
|
Posted - 2010.12.09 18:48:00 -
[13]
FYI.
zofu incursion mysql conversion
Just providing the link, haven't checked it.
|
Ruziel
Minmatar Twilight Military Industrial Complex
|
Posted - 2010.12.09 20:11:00 -
[14]
Originally by: Desmont McCallock FYI.
zofu incursion mysql conversion
Just providing the link, haven't checked it.
I know for a fact that version was converted off of the first release of the dump (the double backup one), and he hadn't run any checks on it yet, hence the warning.
I haven't come across any issues so far using it.
|
Jercy Fravowitz
School of Applied Knowledge
|
Posted - 2010.12.09 23:42:00 -
[15]
the first dump (v1) was in a .zip file, not announced on the forum, did not contain the schema file and had minor issues with the schema change to typeID being 32bit if you insisted on using foreign keys with a backend that requires fked columns to have the same type. it also had a 650MB .bak file, containing two backups. one was missing all rows in one of the ram* tables. the other one was done like 20 minutes later and did have no missing rows. i got told thats what happens when you do a mssql backup dump, and the target file already exists: it appends instead of replaces. so new dev on the job presses button, button goes red and MEEP, stuff gets fixed, dev presses button again -> broken and fixed dump in same .bak file.
now there is a new dump (v2) in a .7z file, proper 300MB size, contains schema blurb, the mentioned minor fk problem fixed.
someone decided keeping the filename for those two very different .bak files the same would be cool. so if you want to figure out which one you are holding, check the wrapper or the filesize. hooray.
still, this wasnt really terrible for a "my first datadump".
those of you who grabbed the v1 file and do not have any FK issues (you would know if you had, trust me on that one), just keep using the -v1, it is fine.
v2 conversions:
mysql singlefile (you are most likely looking for this one) this one should also load just fine in php-webloader-scripts, if it whines about something with timestamps at the end, ignore it. it should now also have bearable performance for loading into innodb. (added no-autocommit to the dump opts)
mysql one-file-per-table (if you dont need/want the whole thing)
sqlite singlefile (also for EVE Asset Manager)
postgres-compatible unquoted
whine here (if you prefer to be publicly ridiculed) or at me ingame (if you prefer to be silently ignored) in case there are any issues or you need custom dumps (partial sqlite, csv, ...) or something...
hints on how to use the split materials data may be found in the dom100 thread.
script of the changes i made compared to the ccp mssql can be found here.
|
Arkady Sadik
Minmatar Electus Matari
|
Posted - 2010.12.10 00:07:00 -
[16]
Apparently, invflags.flagname and invflags.flagtext got swapped around in the dump. Quick fix:
UPDATE invflags SET flagname = flagtext, flagtext = flagname;
|
|
CCP Explorer
|
Posted - 2010.12.10 00:19:00 -
[17]
Originally by: Catari Taga PS: Thanks for the dump. I thought the official name was to be changed to Static Data Export btw. ;)
The official names have always been the Static Data Export Database, or SDEDB or SDE for short, and the Image Export Collection, or IEC.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.10 00:22:00 -
[18]
Originally by: derivativo Why CCP doesn't release the data dump in csv format? Would be _so much_ better! I use Linux and consider waiting for some MySQL/SQLite dump frustrating, as I still need to further export that to PostgreSQL.
With an open format everyone would benefit. Could CCP say something about this, please?
It was detailed in this dev blog more than 2 years ago.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.10 00:23:00 -
[19]
Originally by: Jercy Fravowitz the first dump (v1) was in a .zip file, not announced on the forum, did not contain the schema file and had minor issues with the schema change to typeID being 32bit if you insisted on using foreign keys with a backend that requires fked columns to have the same type. it also had a 650MB .bak file, containing two backups. one was missing all rows in one of the ram* tables. the other one was done like 20 minutes later and did have no missing rows.
now there is a new dump (v2) in a .7z file, proper 300MB size, contains schema blurb, the mentioned minor fk problem fixed.
We didn't announce what you refer to as v1 since it was broken, as you rightfully pointed out.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
CCP Explorer
|
Posted - 2010.12.10 00:24:00 -
[20]
Originally by: Arkady Sadik Apparently, invflags.flagname and invflags.flagtext got swapped around in the dump. Quick fix:
UPDATE invflags SET flagname = flagtext, flagtext = flagname;
I've forwarded this to the developers.
Erlendur S. Thorsteinsson Software Director EVE Online, CCP Games |
|
|
Sarah Fourleaf
|
Posted - 2010.12.10 01:21:00 -
[21]
Hey,
Image pack seems to not be the Incursion one. I can't find any new icons (with top left corner cursor for deadspace, pirate faction, officer, etc.)) in it.
Maybe i made something wrong, but icons (ships + items) seems to be the same as Dominion's ones. |
Celebrain
1st Steps Academy Fidelas Constans
|
Posted - 2010.12.10 03:56:00 -
[22]
Originally by: Sarah Fourleaf I can't find any new icons (with top left corner cursor for deadspace, pirate faction, officer, etc.)) in it.
You have to figure this out for yourself and add them, if you want them. That's the way it's always been for T2 and T3 corners, just now they have new faction, deadspace, etc ones. The new corners themselves (by themselves) are in the new image dump for us to use.
|
Celebrain
1st Steps Academy Fidelas Constans
|
Posted - 2010.12.10 04:01:00 -
[23]
Originally by: derivativo I use Linux and consider waiting for some MySQL/SQLite dump frustrating
I use Linux as well. That's why I set up my own windoze virtual machine and conversion script, so that I'm not totally dependent on a whole chain of other volunteers necessarily. This is the way I recommend it for anyone who wants it in a time sensitive manner and doesn't like to wait for conversions.
|
derivativo
|
Posted - 2010.12.10 08:42:00 -
[24]
Originally by: CCP Explorer It was detailed in this dev blog more than 2 years ago.
Thank you, and "sorry" about not reading _all_ the dev blog posts ever. However, "That method isnÆt as maintainable as we would desire" sounds more like "it is easier to click on create backup". I would really like to know why, technically, CCP has chosen this over CSV which is the most common way to import tabular data since... forever.
Originally by: Celebrain I use Linux as well. That's why I set up my own windoze virtual machine and conversion script, so that I'm not totally dependent on a whole chain of other volunteers necessarily. This is the way I recommend it for anyone who wants it in a time sensitive manner and doesn't like to wait for conversions.
Question is not if I can, ultimately, convert from whichever platform-specific format they release, but why can't they release it under a platform-independent format in the first place. (and hopefully a free, open, non software-restricted format, like csv)
Originally by: Celebrain That said, eve has a rich community of volunteers, and it's great! So feel free to use their conversions if it saves you time too... it's just a matter of weighing how dependent you want to be on other volunteers doing something for you first, vs how much time you want to save yourself, vs how much you like "getting it from the horse's mouth" or "cutting out the middleman" so to speak...
It is not about saving time, weighing dependency, etc, but choice. This concept alone is responsible for the great variety found in free and open source software. If everyone liked the same things there would be only one ship in EVE... and millions of players.
|
Lutz Major
Austriae Est Imperare Orbi Universo
|
Posted - 2010.12.10 09:38:00 -
[25]
Originally by: derivativo blah blah blah
You really think CSV is suitable for exporting database tables and entries? I pity you!
And being the nice guy here, I will not write the other five insulting statements I thought about you and your 'bright' idea.
|
Celebrain
1st Steps Academy Fidelas Constans
|
Posted - 2010.12.10 11:28:00 -
[26]
I'm just happy that CCP releases any data and api's at all... regardless of the format! Most other game companies are far less progressive. This is one of the things that has drawn me to eve in the first place.
The fact that it's in a non-open-standard format, well.. as long as I can get to it relatively easily without paying tons of money, then I won't complain too much.... I use VirtualBox (free), WinXP (old license laying around), and MS SQL Server 2008 Express R2 (free)... CCP is obviously a windows shop.
And yeah, CSV has more variations on "the standard" than you can imagine... it's no real standard at all, unless you use MS Excel as your measurement of a standard, and that's hardly free and open. Remember also this data isn't really "tabular" so much as it's "relational"... :)
|
derivativo
|
Posted - 2010.12.10 16:49:00 -
[27]
Originally by: Lutz Major
Originally by: derivativo blah blah blah
You really think CSV is suitable for exporting database tables and entries? I pity you!
And being the nice guy here, I will not write the other five insulting statements I thought about you and your 'bright' idea.
As far as most database servers use different data types and table information, CSV is a suitable format to export and import the data between them. If you know a better way, please, be kind to share.
Originally by: Celebrain
I'm just happy that CCP releases any data and api's at all... regardless of the format! Most other game companies are far less progressive. This is one of the things that has drawn me to eve in the first place.
I'm also very happy with that. But why can't they be better?
Originally by: Celebrain
The fact that it's in a non-open-standard format, well.. as long as I can get to it relatively easily without paying tons of money, then I won't complain too much.... I use VirtualBox (free), WinXP (old license laying around), and MS SQL Server 2008 Express R2 (free)... CCP is obviously a windows shop.
I'm not quitting the game because the database format is not open or something, I would just like to have it that way and not bother about running virtual machines, waiting for volunteer dumps, etc.
Originally by: Celebrain
And yeah, CSV has more variations on "the standard" than you can imagine... it's no real standard at all, unless you use MS Excel as your measurement of a standard, and that's hardly free and open. Remember also this data isn't really "tabular" so much as it's "relational"... :)
Actually, CSV is a pretty simple format. Most of the variations are about how to interpret commas and special characters, which might be the reason why CCP doesn't bother to export, import and check for data integrity. Also, the data inside a CSV format file is tabular, what is relational is the database server table.
|
Ambo
Tribal Liberation Force
|
Posted - 2010.12.10 17:29:00 -
[28]
CSV is an archaic nightmare of a method to use for transferring database data around.
Thanks CCP for the export - nice and timely as I've come to expect. --------------------------------------
|
Jercy Fravowitz
School of Applied Knowledge
|
Posted - 2010.12.10 18:43:00 -
[29]
Edited by: Jercy Fravowitz on 10/12/2010 18:44:21
Originally by: derivativo Actually, CSV is a pretty simple format. Most of the variations are about how to interpret commas and special characters, which might be the reason why CCP doesn't bother to export, import and check for data integrity. Also, the data inside a CSV format file is tabular, what is relational is the database server table.
if anyone actualy asks for it, i can do excel-format csv dumps with three lines of cut-n-paste. but i do not in general consider it useful to have a GB of CSV sitting around. same goes for XML variants. or custom format dumps, even binary ones. if you ask for it in a specific enough way, most likely no problem. i have a camel and am not afraid to use it.
generating CSV from any DB is trivial, like search for excel here.
generating any DB from CSV is a pain in the rear of bigger girth than the 5 minutes of playing mousepusher-DBA to "convert" the .bak to a GB+ utf16 sql script file with two "free" (beer) tools right off microsoft.com. the annoying part of doing db conversions is matching up the schema to the target flavor. we wouldnt have any less of that with CSV being the delivery format. probably more. the mssql backup at least carries its data structure along in a way that usualy doesnt get desynced, overly violated or lost.
sure as hell not what i would have picked, but works better than what we had before.
(edit: cut-n-paste fsckup)
|
derivativo
|
Posted - 2010.12.10 20:16:00 -
[30]
Originally by: Ambo CSV is an archaic nightmare of a method to use for transferring database data around.
Yes, but is a paid-software-unrestricted one because althought SQL Server Express is free, Windows is not. So you are also eliminating the native import for MacOS users, and there is an official MacOS client. What should they do?
As for MySQL, PostgreSQL and SQLite, some of the most used free database servers, they run under many operating systems. Therefore, why not officially release a dump for any, or all, of these databases? I hardly believe that most 3rd party developers use MSSQL anyway, and there got to be someone in CCP who do know how to convert database formats...
Again, I'm not saying that I'm unhappy with the dev kit, but it could be better for us, developers. If weren't for outstanding volunteers like Jercy there would be much less interest from developers to first convert the data and further program something.
Originally by: Jercy Fravowitz
if anyone actualy asks for it, i can do excel-format csv dumps with three lines of cut-n-paste. but i do not in general consider it useful to have a GB of CSV sitting around. same goes for XML variants. or custom format dumps, even binary ones. if you ask for it in a specific enough way, most likely no problem. i have a camel and am not afraid to use it.
What you already do is pretty good, and the MySQL dump is, indeed, what most people will be likely looking for. But thanks for volunteering.
Originally by: Jercy Fravowitz
generating CSV from any DB is trivial, like search for excel here.
Exactly.
Originally by: Jercy Fravowitz
generating any DB from CSV is a pain in the rear of bigger girth than the 5 minutes of playing mousepusher-DBA to "convert" the .bak to a GB+ utf16 sql script file with two "free" (beer) tools right off microsoft.com. the annoying part of doing db conversions is matching up the schema to the target flavor. we wouldnt have any less of that with CSV being the delivery format. probably more. the mssql backup at least carries its data structure along in a way that usualy doesnt get desynced, overly violated or lost.
sure as hell not what i would have picked, but works better than what we had before.
(edit: cut-n-paste fsckup)
I understand it is better than nothing. And of course that 99%+ of players don't even care about this, as they don't develop anything. And also, the EVE API is pretty much the most amazing thing some MMO company have created to its developer players. But, why not ask a little more? Or, perhaps, a little less, as converting from free cross-platform formats is much easier than do anything under Windows only, virtualized or not. I for one consider that a pain...
|
|
|
|
|
Pages: [1] 2 :: one page |
First page | Previous page | Next page | Last page |