| Pages: 1 2 3 :: [one page] |
|
|
| Author |
Thread Statistics | Show CCP posts - 16 post(s) |
Desmont McCallock
223
   |
Posted - 2012.12.05 19:33:00 -
[1] - Quote
Please post here any issues found with Retribution 1.0 Static Data Export. |
ItsmeHcK1
Burning Napalm
Northern Coalition.
35
|
Posted - 2012.12.05 19:40:00 -
[2] - Quote
Some types in the invTypes table have NULL as a name. I don't recall this being the case in Inferno.
Either way, broke my data exporter for a while until I figured it out.  |
Desmont McCallock
223
|
Posted - 2012.12.05 19:41:00 -
[3] - Quote
So I found some time today to fetch the SDE from CCP's PrismX twitt. And I tried to create EVEMon's data files. Little did I know that my attempt would end up in disaster.
So here's what I have found so far.
- Spelling errors in invMarketGroups for ids: 1688,1689, 1691 "Aplifiers" instead of "Amplifiers".
- In same table ids: 1688,1689,1690,1691 have as parentGroupID id:550 but that id number is nowhere to be found in the table.
I'll will update the post with any new founding. |
Vessper
Indicium Technologies
17
|
Posted - 2012.12.05 20:15:00 -
[4] - Quote
The invMarketGroups table also has parentGroupID entries for 1610, 1625, 1626, 1627 and 1633, all of which don't exist as marketGroupIDs. The first four of those are racial subsystems and the last is for ganglinks.
In addition, item types which have new icons ingame don't have the correct iconID in the invTypes table - they still point to the old icons. This is mainly for shield/armor hardeners/resist amps, the Large Micro Jump Drive and the orbital bombardment ammo. |
Quantix BlackStar
The Praxis Initiative
Gentlemen's Agreement
3
|
Posted - 2012.12.05 21:51:00 -
[5] - Quote
The 'Fleet Assistance Modules' market group (listed as groupId 799 in the SDE), has no sub groups so ganglinks, portal generators, etc are in groups that do not exist currently.
Lead Developer for EveHQ: http://www.evehq.net
|
Desmont McCallock
223
|
Posted - 2012.12.05 21:53:00 -
[6] - Quote
Lets face it. This SDE is a bust. |
Louis Vitton
Jita Trade and Industry
20
|
Posted - 2012.12.06 05:25:00 -
[7] - Quote
I am hanging out for them to re-release it with fixes |
|
CCP Prism X
C C P
C C P Alliance
1005
|
Posted - 2012.12.06 08:50:00 -
[8] - Quote
Sorry for that guys.
In my defense I no longer maintain the SDE code nor have anything to do with its generation. I just knew people would want to get their hands on it ASAP so I Tweeted it.
But I think the new SDE guy might be busy with post expansion fallout. I'll poke at him and if he is I'll just get stuff sorted myself. 
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
|
CCP Prism X
C C P
C C P Alliance
1006
|
Posted - 2012.12.06 11:23:00 -
[9] - Quote
Quantix BlackStar wrote:The 'Fleet Assistance Modules' market group (listed as groupId 799 in the SDE), has no sub groups so ganglinks, portal generators, etc are in groups that do not exist currently.
Not sure what you mean. Group 799 is a hasTypes=1 group so subgroups are not to be expected. Instead there are types in invTypes that have a marketGroupID of 799. Am I misunderstanding something?
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Muscaat
EVE Markets
12
|
Posted - 2012.12.06 12:09:00 -
[10] - Quote
CCP Prism X wrote:Quantix BlackStar wrote:The 'Fleet Assistance Modules' market group (listed as groupId 799 in the SDE), has no sub groups so ganglinks, portal generators, etc are in groups that do not exist currently. Not sure what you mean. Group 799 is a hasTypes=1 group so subgroups are not to be expected. Instead there are types in invTypes that have a marketGroupID of 799. Am I misunderstanding something?
799 is present and correct AFAICT; it's the group for blueprints of items that live in subgroups of the missing 779. |
|
|
|
CCP Prism X
C C P
C C P Alliance
1006
|
Posted - 2012.12.06 12:34:00 -
[11] - Quote
Muscaat wrote:CCP Prism X wrote:Quantix BlackStar wrote:The 'Fleet Assistance Modules' market group (listed as groupId 799 in the SDE), has no sub groups so ganglinks, portal generators, etc are in groups that do not exist currently. Not sure what you mean. Group 799 is a hasTypes=1 group so subgroups are not to be expected. Instead there are types in invTypes that have a marketGroupID of 799. Am I misunderstanding something? 799 is present and correct AFAICT; it's the group for blueprints of items that live in subgroups of the missing 7 79.
Ah! Now we're talking!
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Zifrian
Licentia Ex Vereor
Intrepid Crossing
439
|
Posted - 2012.12.06 15:02:00 -
[12] - Quote
All I've found so far is these two typeId's in the invTypes table that don't have typeNames or any other data - 32894,351611. First one seems to be some sort of celestial (from the icon), not sure on the other.
Maximze your Industry Potential! - Get EVE Isk per Hour! |
Quantix BlackStar
The Praxis Initiative
Gentlemen's Agreement
3
|
Posted - 2012.12.06 17:47:00 -
[13] - Quote
CCP Prism X wrote:Quantix BlackStar wrote:The 'Fleet Assistance Modules' market group (listed as groupId 799 in the SDE), has no sub groups so ganglinks, portal generators, etc are in groups that do not exist currently. Not sure what you mean. Group 799 is a hasTypes=1 group so subgroups are not to be expected. Instead there are types in invTypes that have a marketGroupID of 799. Am I misunderstanding something?
whoops...my mistake on that. I was trying to find where the ganglink items group information was in the group tree (and where the break was), and got that crossed with the blueprint group.
Lead Developer for EveHQ: http://www.evehq.net
|
Louis Vitton
Jita Trade and Industry
20
|
Posted - 2012.12.07 04:52:00 -
[14] - Quote
CCP Prism X wrote:But I think the new SDE guy might be busy with post expansion fallout. I'll poke at him and if he is I'll just get stuff sorted myself.
Cheers for this mate.
|
Artem Valiant
NecroRise Squadron
17
|
Posted - 2012.12.07 06:58:00 -
[15] - Quote
I exported invMarketGroups table from the game client. It contains the missing market groups.
http://dl.eve-files.com/media/1212/invMarketGroups.sql.zip
Strangely, it also contains a spelling error. |
|
CCP Prism X
C C P
C C P Alliance
1010
|
Posted - 2012.12.07 08:28:00 -
[16] - Quote
Just checking in so nobody thinks I forgot about you.
Yesterday was quite intensive on meetings with regards to going over the deployment, the development cycle leading up to it and starting to line up our ducks for the next cycle. In between those I was looking at the SDE (and other things) and I'm pretty confident I'll be done with it by noon GMT.
I'll try and shanghai some people into releasing it ASAP and post the new link here as well as twitter.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Desmont McCallock
227
|
Posted - 2012.12.07 08:30:00 -
[17] - Quote
CCP Prism X wrote:Just checking in so nobody thinks I forgot about you.
Yesterday was quite intensive on meetings with regards to going over the deployment, the development cycle leading up to it and starting to line up our ducks for the next cycle. In between those I was looking at the SDE (and other things) and I'm pretty confident I'll be done with it by noon GMT.
I'll try and shanghai some people into releasing it ASAP and post the new link here as well as twitter.
Any news is good news. |
|
CCP Prism X
C C P
C C P Alliance
1011
|
Posted - 2012.12.07 11:13:00 -
[18] - Quote
New SDE, probably with new bugs in it!
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Lluerssen
Finaka Industrial Group
Ouroboros Group
5
|
Posted - 2012.12.07 11:50:00 -
[19] - Quote
New invTypes: 16593
Old invTypes: 20186
is this ok?
Sister of EvE: http://eve.bubonicpestilence.ru/
Loot History Analyzer:-áhttp://eve.bubonicpestilence.ru/loot_history/ |
|
CCP Prism X
C C P
C C P Alliance
1011
|
Posted - 2012.12.07 11:57:00 -
[20] - Quote
Hmm, not 100% sure atm but it sure does look like something got sanitized out of the types that shouldn't have. I'll beat at it with a stick for a while and see what happens.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
|
|
Lluerssen
Finaka Industrial Group
Ouroboros Group
5
|
Posted - 2012.12.07 11:59:00 -
[21] - Quote
http://cl.ly/1308150r0p1E
Removed invTypes, omg -.-
> SELECT count(*) FROM `invTypes` WHERE typeName LIKE '%blueprint%';
3053 vs 112
Sister of EvE: http://eve.bubonicpestilence.ru/
Loot History Analyzer:-áhttp://eve.bubonicpestilence.ru/loot_history/ |
|
CCP Prism X
C C P
C C P Alliance
1011
|
Posted - 2012.12.07 12:01:00 -
[22] - Quote
Yeah I accidentally put an inner join on the description, change log clearly states that was a mistake. Gimmie a few moments.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Vessper
Indicium Technologies
18
|
Posted - 2012.12.07 12:07:00 -
[23] - Quote
mapSolarSystems has no rows in the new SDE?
|
|
CCP Prism X
C C P
C C P Alliance
1011
|
Posted - 2012.12.07 12:10:00 -
[24] - Quote
Vessper wrote:mapSolarSystems has no rows in the new SDE?
Dunno about the broken one I generated but the one I'm uploading now seems cool:
SELECT COUNT(*) FROM mapSolarSystems
7929
(1 row(s) affected)
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Vessper
Indicium Technologies
18
|
Posted - 2012.12.07 12:12:00 -
[25] - Quote
Yes, it was the one released an hour or so ago.
|
|
CCP Prism X
C C P
C C P Alliance
1011
|
Posted - 2012.12.07 12:19:00 -
[26] - Quote
Yeah that makes sense. The sun types do not have a description authored. Therefore they did not get dumped into the types table, and thus all solar systems were removed as they have a sunTypeID authored and all sun types were deleted.
However:
New fixedfixed version uploaded to same addy and CDN cache purged.
That's attempt two, I'm aiming for "Third time's the charm!" though! 
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Vessper
Indicium Technologies
18
|
Posted - 2012.12.07 12:51:00 -
[27] - Quote
The eveUnits table is missing unitIDs 103, 122 and 129, and 122 is used by many of the attributes (103 and 129 appear unused). The only thing I can see that these have in common is that the unit display name is null (or was at least in the Inferno SDE).
It's easy to add that one line in manually but just thought I'd raise the issue anyway.
|
Vessper
Indicium Technologies
18
|
Posted - 2012.12.07 13:19:00 -
[28] - Quote
The dgmAttributeTypes table is missing attributes for the low, mid, hi and rig slots of a ship (IDs 12,13,14,1137) and the dgmTypeAttributes table is missing all entries containing these IDs.
Not sure if it's related to the above bearing in mind unitID 122 is "Fitting Slots".
|
|
CCP Prism X
C C P
C C P Alliance
1012
|
Posted - 2012.12.07 13:21:00 -
[29] - Quote
Probably is.
And this is why we should be releasing the SDE before deploying the expansion. I'm going to wait for some more issues to crop up before generating another one.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Sidrat Flush
Low Sec Pharmacies
21
|
Posted - 2012.12.07 13:23:00 -
[30] - Quote
When the tables are converted to yaml will errors be easier or harder to spot?
I am not my skill points but http://eveboard.com/pilot/Sidrat_Flush |
|
|
Dei
Mamaleek
3
|
Posted - 2012.12.07 20:13:00 -
[31] - Quote
Just out of interest. How much does data structure and data change in the days before a release? |
Dread Nanana
Action Super Dupper Test Corp
7
|
Posted - 2012.12.08 16:38:00 -
[32] - Quote
Sidrat Flush wrote:When the tables are converted to yaml will errors be easier or harder to spot?
Yes, YAML - a structure known for its strong consistency checks and resilience to mistakes 
The straight MSSQL dump is much superior, at least if it had any constraints maybe. Without foreign key constraints, we get what we get.
|
Dei
Mamaleek
3
|
Posted - 2012.12.08 16:59:00 -
[33] - Quote
Not sure why there aren't any integrity scripts. It sounds like this is a fairly manual process to get us the metadata. Is the actual datastructure that horrible to work with? |
Sidrat Flush
Low Sec Pharmacies
21
|
Posted - 2012.12.08 17:12:00 -
[34] - Quote
Dei wrote:Not sure why there aren't any integrity scripts. It sounds like this is a fairly manual process to get us the metadata. Is the actual datastructure that horrible to work with?
There's data they don't want us to have only thong comes to mind is the NPC rat information although it used to be included.
I'm sure there's other stuff just because would be interesting to get more information though.
So YAML and I haven't used it will be worse to work with, is a yaml sheet easy to convert to an access table or better yet a straight to excel pivot table?
I am not my skill points but http://eveboard.com/pilot/Sidrat_Flush |
Desmont McCallock
228
|
Posted - 2012.12.10 16:51:00 -
[35] - Quote
IDs 363369, 363370, 363371 are missing from invMarketGroups related to IDs 350916, 363355, 363356, 363357, 363358, 363359 in invTypes. All those are DUST 514 related data though. |
Zifrian
Licentia Ex Vereor
Intrepid Crossing
442
|
Posted - 2012.12.11 20:52:00 -
[36] - Quote
So a week later and still no fixed data?
Maximze your Industry Potential! - Get EVE Isk per Hour! |
Desmont McCallock
228
|
Posted - 2012.12.13 18:12:00 -
[37] - Quote
No news, is bad news. (Also bumping thread) |
Dei
Mamaleek
3
|
Posted - 2012.12.13 20:32:00 -
[38] - Quote
PrismX this morning said he was waiting on more reports on what was wrong. :S |
Desmont McCallock
228
|
Posted - 2012.12.14 05:42:00 -
[39] - Quote
He already stated that in his last post. On one hand he's justified. On the other hand I'm about to explode by the criticism I receive why I haven't updated EVEMon yet. I'm between a rock and a hard place. |
darius mclever
44
|
Posted - 2012.12.14 12:15:00 -
[40] - Quote
+1 on releasing a dump now
and then maybe using the waiting time for writing a script that validates that the dump is consistent. |
|
|
Zifrian
Licentia Ex Vereor
Intrepid Crossing
444
|
Posted - 2012.12.14 12:56:00 -
[41] - Quote
Desmont McCallock wrote:He already stated that in his last post. On one hand he's justified. On the other hand I'm about to explode by the criticism I receive why I haven't updated EVEMon yet. I'm between a rock and a hard place.
Not to mention that be said he's not the data dump guy anymore either and he's given us more than him.
Maximze your Industry Potential! - Get EVE Isk per Hour! |
Golden Gnu
EVE University
Ivy League
70
|
Posted - 2012.12.14 22:55:00 -
[42] - Quote
I hope they learn a lesson: (PrismX + API/SDE) or fail (J/K  )
Anyway, hope they improve the process for next release.
One should think it could be tested and automated pretty easy (one could of course be wrong)
if EveMon is not updated, Eve-Online is not fully updated.
Everyone knows EveMon just as much a part of the game as the eve client IMHO.
Also, I'm sorry if I'm being rude... 
Download is the meaning of life, upload is the meaning of intelligent life
http://eve.nikr.net - home of jEveAssets |
mramo
Pator Tech School
Minmatar Republic
1
|
Posted - 2012.12.15 16:59:00 -
[43] - Quote
Golden Gnu wrote:if EveMon is not updated, Eve-Online is not fully updated.
Everyone knows EveMon just as much a part of the game as the eve client IMHO.
That's why CCP should build their own skilltraining / EVE-monitoring app. EveMon provides a great improvement in playing the game. It is mandatory. CCP shouldn't let some not-controllable third party handle a crucial part of the game. |
Dei
Mamaleek
3
|
Posted - 2012.12.15 17:24:00 -
[44] - Quote
That's the opposite of what they should do.
- 3rd party developers are free for CCP
- CCP developers can focus on the game, not mini applications outside of the game
- If CCP developers replicate Evemon, not only have they pissed off the creator, no-one else is going to develop anything knowing that at any time CCP will make an 'official' application
- CCP don't have the resources to support all apps in existance, so why should EveMon be the only one?
- CCP stated they would support 3rd party developers with the SSD
- CREST is apparently somewhere around that would help EveMon
|
Cyerus
Galactic Dominion
Eternal Strife
64
|
Posted - 2012.12.15 18:09:00 -
[45] - Quote
Version 1.0.2 is still missing the slot-layout of ships.
dgmAttributeTypes (or something) seems to be missing the information. |
Dei
Mamaleek
3
|
Posted - 2012.12.15 19:43:00 -
[46] - Quote
In ramTypeRequirements, the data interfaces for invention do not have a damageperjob of 0. They should have a damageperjob of 0. |
Steve Ronuken
Fuzzwork Enterprises
851
|
Posted - 2012.12.15 19:49:00 -
[47] - Quote
Dei wrote:In ramTypeRequirements, the data interfaces for invention do not have a damageperjob of 0. They should have a damageperjob of 0.
This isn't a new thing. Ever since I first imported the dump it's been wrong on interfaces. (Just for reference. would still be nice to have sorted)
FuzzWork Enterprises
http://www.fuzzwork.co.uk/
Blueprint calculator, invention chance calculator, isk/m3 Ore chart-á and other 'useful' utilities.As well as mysql and CSV/XLS conversions of the Static Data Extract. |
Dei
Mamaleek
3
|
Posted - 2012.12.15 21:27:00 -
[48] - Quote
Yeah, was doing work on it today and wondered what people did, and apparently you're all just manually fixing the data you strange people, you. |
Golden Gnu
EVE University
Ivy League
71
|
Posted - 2012.12.16 09:41:00 -
[49] - Quote
@mramo
Yes, because CCP is known to create epic user interfaces - it's their key sell point AFAIK?
Do not fix what is not broken. EveMon is great, no fix is needed!
@Dei
Everyone fixing it sounds like a giant waste of time. 
Join force FFS!  And let the lovely people that convert the SDE to MySQL release it when it's done, please...?
Download is the meaning of life, upload is the meaning of intelligent life
http://eve.nikr.net - home of jEveAssets |
Harry Pearce
The Intel Project
1
|
Posted - 2012.12.16 16:45:00 -
[50] - Quote
As a warning to anyone else bitten by this, not only are ship slots missing from the newest dump, but the eveIcons table has also disappeared. Whilst the iconID column had already been removed from invTypes, the table was present in the initial retribution dump.
This means you need to be aware that trying to join on the iconID column in (at least) chrFactions, crtCertificates, dgmAttributeTypes, invCategories, invGroups, invMarketGroups and invMetaGroups won't work until you've recreated an eveIcons table suited to your needs from the YAML.
There are also some market groups[1] that have hasTypes = 1, but no corresponding rows in invTypes, and one[2] with hasTypes = 0, but no subgroups - possible worth pruning from your personal dumps if you're generating a market tree!
[1] SELECT invMarketGroups.marketGroupID, marketGroupName FROM invMarketGroups LEFT JOIN invTypes ON invMarketGroups.marketGroupID = invTypes.marketGroupID WHERE typeID IS NULL AND hasTypes != 0
[2] SELECT mg.marketGroupID, mg.marketGroupName FROM invMarketGroups mg LEFT JOIN invMarketGroups mg2 ON mg.marketGroupID = mg2.parentGroupID WHERE mg.hasTypes = 0 AND mg2.marketGroupID IS NULL |
|
|
Raven Seldon
Black Thorne Corporation
Black Thorne Alliance
0
|
Posted - 2012.12.17 09:27:00 -
[51] - Quote
A bit weird it takes this long.
Will the service be discontinued? |
Nik W
Critical Mass Inc.
Nexus Fleet
4
|
Posted - 2012.12.17 23:04:00 -
[52] - Quote
Silence is bad
Can we get an update on the status of this? |
|
CCP Prism X
C C P
C C P Alliance
1015
|
Posted - 2012.12.18 09:29:00 -
[53] - Quote
I'm not ignoring this, I'm just busy with other things. My plan is to do my utmost to get everything I need to have done, and fix the SDE as well, before I go on vacation. Please don't hate me, I'm delicate like a lotus flower in bloom.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Callean Drevus
Icosahedron Crafts and Shipping
Silent Infinity
131
|
Posted - 2012.12.18 10:43:00 -
[54] - Quote
We don't hate you, we are just wondering why this is seemingly so low on CCP's priority list :(
Are new features / meetings really more important than fixing this? I mean, I know they are generally more fun to do than data exporting, but we (including the one whose task it is) will all become very happy when this is done.
It may have been asked before, but why is there not a standard program / script that does this whole dump every time an expansion is launched. I mean, the data structure cannot change THAT much every cycle... and this is not the first time we've had these issues. More like, it happens almost every time an expansion is launched...
Developer/Creator of EVE Marketeer
|
|
CCP Prism X
C C P
C C P Alliance
1015
|
Posted - 2012.12.18 12:42:00 -
[55] - Quote
Can't speak for the company, but this is low on my priority list because I'm actually not responsible for this and would rather finish up stuff first for which I will be held accountable. Not an ideal situation I know, but them's the breaks.
As to why this breaks all the time:
Multiple reasons. Bad authoring + bad integration of static data between branches + bad code in the generation scripts that has never been bad before because it needed the bad authoring + bad integration to actually become an issue, was the first issue.
So I had to make code changes to shield you from broken relations, empty record names and such. And when you change code **** breaks.
And thus I'm really responsible for this. And that's why I'll be making another dump, fixed according to the issues I see here. I was going to wait a few days to sort out all the issues, as I understood from the first few posts that the issues weren't that critical anymore. Just some missing eveUnits records. Apparently that is not the case.
But other stuff comes up. I need to respond to that as well. It's no fun when what you need gets ranked on a low priority but it has to happen somewhere. Otherwise you're not prioritizing. But I was planning on doing this today, that might not work out anymore as I've lost a lot of time between the morning and noon due to unforeseen issues.
Today or tomorrow. Sorry for the delay. If someone would make a conscice bulletpointed list of all the remaining issues, I might consider hugging him next FanFest.
But it's not like it's a huge issue for me to do it myself either. But then you don't get to feel like you helped!
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Sidrat Flush
Eve Industrial Corp
27
|
Posted - 2012.12.18 12:54:00 -
[56] - Quote
We appreciate your "free"? Time and I realise it's a labor of love mixed with a dash of hate to get the SDE to us, I want to understand how the future YAML formatted SDE will be easier to update and release, assuming it will be of course.
What about CREST does it the SDE form part of it or is crest a separate entity?
The new home of the Eve Industrial Organiser is here. Enjoy the first in a series, EIO:Refinery now
http://www.eve-files.com/media/corp/Sidrat/
Read about it http://eveindustrialorganiser.blogspot.co.uk/?m=0 |
|
CCP Prism X
C C P
C C P Alliance
1015
|
Posted - 2012.12.18 13:02:00 -
[57] - Quote
I'm not much involved with the YAML files. I suspect they'll just create more issues, at least for a fair bit. They already have.
I see numerous posts here about people wondering why invTypes doesn't have an iconID, and that's because typeID->iconID and typeID->graphicID relations have been moved into YAML files. So we now have type information split between the restore data and the YAML data.
However, YAML data should be less prone to bullshit incidents such as the data we ship to you not being the 100% current data of the release branch. As it stands we have a code versioning system and a seperate data versioning system, which means the data can be out of synch with the code. Of course we've put a lot of effort into both avoiding that and handling such incidents gracefully so we don't notice this, f.e. all the unnamed types went out to TQ and broke nothing but broke lots of third party apps.
And CREST has nothing to do with the SDE. Just like the API has nothing to do with the SDE. Although obviously any application that uses the API (Legacy or, in the future, CREST) could benefit from having data from the SDE stored locally.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Callean Drevus
Icosahedron Crafts and Shipping
Silent Infinity
131
|
Posted - 2012.12.18 13:10:00 -
[58] - Quote
CCP Prism X wrote:Can't speak for the company.... But then you don't get to feel like you helped!
My apologies for sounding so bitchy. It's nothing personal . Thanks for both the explanation and taking the time to do this. Bad code is a *****.
As to priorities, I understand why it is not high on YOUR priority list I'm just not sure why it's not higher on CCP's priority list.
I'll download the current dump and try to see if I find any issues.
Developer/Creator of EVE Marketeer
|
Muscaat
EVE Markets
16
|
Posted - 2012.12.18 13:55:00 -
[59] - Quote
CCP Prism X wrote:...awesomeness
CCP Prism X, best Prism X.
Thanks for your continued efforts to help us out  |
|
CCP Prism X
C C P
C C P Alliance
1016
|
Posted - 2012.12.18 15:34:00 -
[60] - Quote
We've got a new SDE up guys, get it while it's hot!
IMPORTANT NOTICE:
READ THIS OR YOU'LL GET ANGRY WHEN I MAKE FUN OF YOU FOR NOT HAVING READ THIS!
Now that I got your attention, reading this will also make you angry.
There is no iconID or graphicID in invTypes anymore. You will never find it there again. It has been moved into the types.YAML file we ship with the SDE zip file.
If you're using a third party conversion that does not include this data in one form or another that's your converter doing something wrong.
If you are doing a conversion and want this data back into a relationial database, you're going to have to get it from the YAML file.
Changelog:
invTypes:
Problem: table is missing the iconID column.
Solution: Get them, along with the graphicIDs, from types.YAML. They will not be found in the official backup.
eveUnits:
Problem: unitID 103, 122, 129 is missing.
Solution: These units do not have a display name, do include units without a display name.
dogmaTypeAttributes, dogmaAttributeTypes:
Problem: Missing slot attributes
Solution: Fixing eve units to include the units will retain the slot data rathre than sanitizing it out as a broken relation.
invMarketGroups, invTypes
Problem: Missing records.
Solution: These are unreleased DUST records which do not belong in the SDE.
ramTypeRequirements
Problem: Data Interfaces for invention have a damage per job of 1.
Solution: This is how the data is authored and always has been. Checked the history. I can't fix things that are by design.
Interfaces are special cased in our code. Maybe one day I get to rewrite the clusterintimacy that the S&I backend is.
invMarketGroups
Problem: Market groups exist with hasTypes = 0, but still have types. Market groups exist with hasTypes = 1 but have no types.
Solution: Check these relations and sanitize out bad data. I have no clue when GD will actually fix the data but at least I can fix your SDE
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
|
|
Steve Ronuken
Fuzzwork Enterprises
858
|
Posted - 2012.12.18 18:04:00 -
[61] - Quote
dgmEffects and dgmTypeEffects appear to be truncated from what they had on the first export for retribution.
FuzzWork Enterprises
http://www.fuzzwork.co.uk/
Blueprint calculator, invention chance calculator, isk/m3 Ore chart-á and other 'useful' utilities.As well as mysql and CSV/XLS conversions of the Static Data Extract. |
Desmont McCallock
230
|
Posted - 2012.12.18 20:28:00 -
[62] - Quote
Supplemental to Steve's post.
dgmEffects -> Inferno SDE: 2934 lines, Retribution SDE: 101 lines
dgmTypeEffects -> Inferno SDE: 34184 lines, Retribution SDE: 7332 lines
Edit: This issue doesn't affect us much in EVEMon data files generation, as we use the dgmTypeEffects table only to determine the slot usage, I can say that a new EVEMon version with updated data files is on its way. |
Quantix BlackStar
The Praxis Initiative
Gentlemen's Agreement
3
|
Posted - 2012.12.18 23:06:00 -
[63] - Quote
Desmont McCallock wrote:Supplemental to Steve's post.
dgmEffects -> Inferno SDE: 2934 lines, Retribution SDE: 101 lines
dgmTypeEffects -> Inferno SDE: 34184 lines, Retribution SDE: 7332 lines
Edit: This issue doesn't affect us much in EVEMon data files generation, as we use the dgmTypeEffects table only to determine the slot usage, I can say that a new EVEMon version with updated data files is on its way.
EveHQ uses these tables in its Item Browser and Fitting modules. In my ad-hoc testing that I've been doing with fits today, I have not run into a noticeable issue, but it would be nice to have some explanations of the removals if only for clarification.
EveHQ is a multi-faceted character application for EVE Online. Includes pilot monitoring, skill queue planning, ship fitting, industry and more. Get the latest version at http://evehq.net
|
Cielle Tische
Who Armed the Carebears
0
|
Posted - 2012.12.18 23:24:00 -
[64] - Quote
EVEIDB uses these tables for item display.
Tables used in Inferno:
[AGT]
agtagents
agtagenttypes
agtresearchagents
[CHR]
chrancestries
chrattributes
chrbloodlines
chrfactions
chrraces
[CRP]
crpactivities
crpnpccorporationdivisions
crpnpccorporationresearchfields
crpnpccorporations
crpnpccorporationtrades
crpnpcdivisions
[CRT]
crtcategories
crtcertificates
crtclasses
crtrecommendations
crtrelationships
[DMG]
dgmattributecategories
dgmattributetypes
dgmeffects
dgmtypeattributes
dgmtypeeffects
[INV]
invblueprinttypes
invcategories
invcontrabandtypes
invcontroltowerresourcepurposes
invcontroltowerresources
invflags
invgroups
invitems
invmarketgroups
invmetagroups
invmetatypes
invnames
invpositions
invtypematerials
invtypereactions
invtypes
invuniquenames
[MAP]
mapcelestialstatistics
mapconstellationjumps
mapconstellations
mapdenormalize
mapjumps
maplandmarks
maplocationscenes
maplocationwormholeclasses
mapregionjumps
mapregions
mapsolarsystemjumps
mapsolarsystems
mapuniverse
[PLN]
planetinteraction
planetschematics
planetschematicspinmap
planetschematicstypemap
[RAM]
ramactivities
ramassemblylines
ramassemblylinestations
ramassemblylinetypedetailpercategory
ramassemblylinetypedetailpergroup
ramassemblylinetypes
raminstallationtypecontents
ramtyperequirements
[OTHER]
eveunits
eveicons
Those are the SDE tables we use to generate pages. Also, we use MySQL, instead of MSSQL, and we avoid YAML like it's cursed with instant death. Guess I get to figure out how to convert the YAML Icon tables into MySQL from here on out instead of them just being left where they were >.>
Developer of www.eveidb.com - Relational EVE database!
[Still under construction -- and stupidly imported the Ret1.0.2 SDE, please disregard missing data!] |
Steve Ronuken
Fuzzwork Enterprises
858
|
Posted - 2012.12.18 23:30:00 -
[65] - Quote
Cielle Tische wrote:EVEIDB uses these tables for item display.
Tables used in Inferno:
[AGT]
agtagents
agtagenttypes
agtresearchagents
[CHR]
chrancestries
chrattributes
chrbloodlines
chrfactions
chrraces
[CRP]
crpactivities
crpnpccorporationdivisions
crpnpccorporationresearchfields
crpnpccorporations
crpnpccorporationtrades
crpnpcdivisions
[CRT]
crtcategories
crtcertificates
crtclasses
crtrecommendations
crtrelationships
[DMG]
dgmattributecategories
dgmattributetypes
dgmeffects
dgmtypeattributes
dgmtypeeffects
[INV]
invblueprinttypes
invcategories
invcontrabandtypes
invcontroltowerresourcepurposes
invcontroltowerresources
invflags
invgroups
invitems
invmarketgroups
invmetagroups
invmetatypes
invnames
invpositions
invtypematerials
invtypereactions
invtypes
invuniquenames
[MAP]
mapcelestialstatistics
mapconstellationjumps
mapconstellations
mapdenormalize
mapjumps
maplandmarks
maplocationscenes
maplocationwormholeclasses
mapregionjumps
mapregions
mapsolarsystemjumps
mapsolarsystems
mapuniverse
[PLN]
planetinteraction
planetschematics
planetschematicspinmap
planetschematicstypemap
[RAM]
ramactivities
ramassemblylines
ramassemblylinestations
ramassemblylinetypedetailpercategory
ramassemblylinetypedetailpergroup
ramassemblylinetypes
raminstallationtypecontents
ramtyperequirements
[OTHER]
eveunits
eveicons
Those are the SDE tables we use to generate pages. Also, we use MySQL, instead of MSSQL, and we avoid YAML like it's cursed with instant death. Guess I get to figure out how to convert the YAML Icon tables into MySQL from here on out instead of them just being left where they were >.>
http://sumanrs.wordpress.com/2012/02/23/yaml-to-mysql-yaml2sql-script-in-php/ may be of interest to you.
(mysql version available from another post of mine)
FuzzWork Enterprises
http://www.fuzzwork.co.uk/
Blueprint calculator, invention chance calculator, isk/m3 Ore chart-á and other 'useful' utilities.As well as mysql and CSV/XLS conversions of the Static Data Extract. |
Dei
Mamaleek
3
|
Posted - 2012.12.18 23:54:00 -
[66] - Quote
It's not too bad that it's in YAML. It is strange that the official way of converting the MSSQL backup is to do one table per expansion...
There are a crap load of YAML to XXX converters available on their main website, so it shouldn't be too hard to integrate it into your program/app. I do wish CCP would either convert the entire lot to YAML or continue with a DB backup until it's all done though. |
|
CCP Prism X
C C P
C C P Alliance
1017
|
Posted - 2012.12.19 09:40:00 -
[67] - Quote
Fixing stools!
I'm going on vacation after tomorrow, Thursday the 20th of December. That means if there are more issues after tomorrow they will not get fixed until 2013. I'm going carefully over all the joins I added to ensure that no further data is lost from the first update to remove the nameless records. Hopefully there will be no further issues with that specific change.
But if there's anything other broken, the window to report it is getting slimmer by the minute.
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
|
CCP Prism X
C C P
C C P Alliance
1019
|
Posted - 2012.12.19 11:26:00 -
[68] - Quote
Yet another SDE Deployment: http://content.eveonline.com/data/Retribution_1.0.7_463858_db.zip
@CCP_PrismX
EVE Database Developer and Expert Ranter
Member of Team Pony Express |
|
Dei
Mamaleek
3
|
Posted - 2012.12.19 11:35:00 -
[69] - Quote
Did anyone tell you how awesome you are? |
Steve Ronuken
Fuzzwork Enterprises
859
|
Posted - 2012.12.19 13:46:00 -
[70] - Quote
https://forums.eveonline.com/default.aspx?g=posts&t=184575
In case anyone wants converted versions
FuzzWork Enterprises
http://www.fuzzwork.co.uk/
Blueprint calculator, invention chance calculator, isk/m3 Ore chart-á and other 'useful' utilities.As well as mysql and CSV/XLS conversions of the Static Data Extract. |
|
|
Desmont McCallock
230
|
Posted - 2012.12.19 14:34:00 -
[71] - Quote
SDE looks OK now. Concurring, CCP Prism X all time favorite CCP.
@CCP PrismX
Still your #2 fan. (For those who don't know, Karbowiak is #1). |
Lors Dornick
Kallisti Industries
Solar Assault Fleet
357
|
Posted - 2012.12.19 15:59:00 -
[72] - Quote
Dei wrote:Did anyone tell you how awesome you are?
Shush, don't tell him!
Keep it a secret and hope that he'll work even harder ;)
CCP Eterne: Silly player, ALL devs are evil.
|
Jercy Fravowitz
School of Applied Knowledge
Caldari State
4
|
Posted - 2012.12.19 17:34:00 -
[73] - Quote
Desmont McCallock wrote:SDE looks OK now. Concurring, CCP Prism X all time favorite CCP.
- you call that OK? pah. it is totally missing the entry for 500021 in chrFactions. dont you people have real databases with proper FKs?!
- i kinda agree. he is definetly among the top 5 devs. though he was cuter when he was even younger.
- time to hide somewhere.
- oh, yeah.
- my favorite part of this thread is the part with the feedback about the lack of evemon updates, and the suggested workaround/fix. that would sure work. *chuckles*
|
Desmont McCallock
230
|
Posted - 2012.12.19 18:35:00 -
[74] - Quote
Jercy Fravowitz wrote:Desmont McCallock wrote:SDE looks OK now. Concurring, CCP Prism X all time favorite CCP.
- you call that OK? pah. it is totally missing the entry for 500021 in chrFactions. dont you people have real databases with proper FKs?!
- i kinda agree. he is definetly among the top 5 devs. though he was cuter when he was even younger.
- time to hide somewhere.
- oh, yeah.
- my favorite part of this thread is the part with the feedback about the lack of evemon updates, and the suggested workaround/fix. that would sure work. *chuckles*
If only I didn't knew your sense of humor....  |
Snus Mumriken
EVE University
Ivy League
16
|
Posted - 2012.12.19 23:19:00 -
[75] - Quote
PrismX, can you please keep the old naming scheme with the tranquility branch change ID instead of this other number? (463858, the build number or something, don't know exactly.)
Thanks!
I'm on Twitter. |
Cielle Tische
Who Armed the Carebears
0
|
Posted - 2012.12.20 05:17:00 -
[76] - Quote
Thank god for all the fixes. However, either it didn't import correctly in my DB or something, but the new ships (velator, corax, etc) don't have blueprint information. Everything else seems to be working.
Developer of www.eveidb.com - Relational EVE database!
[Still under construction -- and stupidly imported the Ret1.0.2 SDE, please disregard missing data!] |
Desmont McCallock
230
|
Posted - 2012.12.20 05:55:00 -
[77] - Quote
Cielle Tische wrote:Thank god for all the fixes. However, either it didn't import correctly in my DB or something, but the new ships (velator, corax, etc) don't have blueprint information. Everything else seems to be working.
I can see those blueprints in SDE. |
Cielle Tische
Who Armed the Carebears
0
|
Posted - 2012.12.20 07:04:00 -
[78] - Quote
Desmont McCallock wrote:Cielle Tische wrote:Thank god for all the fixes. However, either it didn't import correctly in my DB or something, but the new ships (velator, corax, etc) don't have blueprint information. Everything else seems to be working. I can see those blueprints in SDE.
I'll try reimporting then. And I meant the Venture, not Velator.... My friend just started playing and he was in a velator. Must have been looking at his ship when I typed that xD
This is my example: http://www.eveidb.com/item?i=32881 (Venture) It finds the blueprint, but the material data is missing.
Developer of www.eveidb.com - Relational EVE database!
[Still under construction -- and stupidly imported the Ret1.0.2 SDE, please disregard missing data!] |
Desmont McCallock
230
|
Posted - 2012.12.20 07:55:00 -
[79] - Quote
According to EVEMon everything is there. Scan the forum on how to get the materials for a BP as it involves more than one table. |
Cielle Tische
Who Armed the Carebears
0
|
Posted - 2012.12.20 08:29:00 -
[80] - Quote
Desmont McCallock wrote:According to EVEMon everything is there. Scan the forum on how to get the materials for a BP as it involves more than one table.
It's just the new ones. All the others work perfectly, for example: http://www.eveidb.com/item?i=24691 (Hyperion)
I reimported the data and still nothing. Also imported into a blank database to see if it was being tainted by old data, and still nada.
---Ahhhahaha. Got it. Forgot to convert the table into the saferead tab. /derp Should be able to fix this fairly quickly.
Developer of www.eveidb.com - Relational EVE database!
[Still under construction -- and stupidly imported the Ret1.0.2 SDE, please disregard missing data!] |
|
|
|
|
| |
|
| Pages: 1 2 3 :: [one page] |