Pages: 1 [2] 3 :: one page |
|
Author |
Thread Statistics | Show CCP posts - 16 post(s) |
Dei
Mamaleek
3
|
Posted - 2012.12.07 20:13:00 -
[31] - Quote
Just out of interest. How much does data structure and data change in the days before a release? |
Dread Nanana
Action Super Dupper Test Corp
7
|
Posted - 2012.12.08 16:38:00 -
[32] - Quote
Sidrat Flush wrote:When the tables are converted to yaml will errors be easier or harder to spot?
Yes, YAML - a structure known for its strong consistency checks and resilience to mistakes
The straight MSSQL dump is much superior, at least if it had any constraints maybe. Without foreign key constraints, we get what we get.
|
Dei
Mamaleek
3
|
Posted - 2012.12.08 16:59:00 -
[33] - Quote
Not sure why there aren't any integrity scripts. It sounds like this is a fairly manual process to get us the metadata. Is the actual datastructure that horrible to work with? |
Sidrat Flush
Low Sec Pharmacies
21
|
Posted - 2012.12.08 17:12:00 -
[34] - Quote
Dei wrote:Not sure why there aren't any integrity scripts. It sounds like this is a fairly manual process to get us the metadata. Is the actual datastructure that horrible to work with?
There's data they don't want us to have only thong comes to mind is the NPC rat information although it used to be included.
I'm sure there's other stuff just because would be interesting to get more information though.
So YAML and I haven't used it will be worse to work with, is a yaml sheet easy to convert to an access table or better yet a straight to excel pivot table?
I am not my skill points but http://eveboard.com/pilot/Sidrat_Flush |
Desmont McCallock
228
|
Posted - 2012.12.10 16:51:00 -
[35] - Quote
IDs 363369, 363370, 363371 are missing from invMarketGroups related to IDs 350916, 363355, 363356, 363357, 363358, 363359 in invTypes. All those are DUST 514 related data though. |
Zifrian
Licentia Ex Vereor Intrepid Crossing
442
|
Posted - 2012.12.11 20:52:00 -
[36] - Quote
So a week later and still no fixed data? Maximze your Industry Potential! - Get EVE Isk per Hour! |
Desmont McCallock
228
|
Posted - 2012.12.13 18:12:00 -
[37] - Quote
No news, is bad news. (Also bumping thread) |
Dei
Mamaleek
3
|
Posted - 2012.12.13 20:32:00 -
[38] - Quote
PrismX this morning said he was waiting on more reports on what was wrong. :S |
Desmont McCallock
228
|
Posted - 2012.12.14 05:42:00 -
[39] - Quote
He already stated that in his last post. On one hand he's justified. On the other hand I'm about to explode by the criticism I receive why I haven't updated EVEMon yet. I'm between a rock and a hard place. |
darius mclever
44
|
Posted - 2012.12.14 12:15:00 -
[40] - Quote
+1 on releasing a dump now
and then maybe using the waiting time for writing a script that validates that the dump is consistent. |
|
Zifrian
Licentia Ex Vereor Intrepid Crossing
444
|
Posted - 2012.12.14 12:56:00 -
[41] - Quote
Desmont McCallock wrote:He already stated that in his last post. On one hand he's justified. On the other hand I'm about to explode by the criticism I receive why I haven't updated EVEMon yet. I'm between a rock and a hard place. Not to mention that be said he's not the data dump guy anymore either and he's given us more than him. Maximze your Industry Potential! - Get EVE Isk per Hour! |
Golden Gnu
EVE University Ivy League
70
|
Posted - 2012.12.14 22:55:00 -
[42] - Quote
I hope they learn a lesson: (PrismX + API/SDE) or fail (J/K ) Anyway, hope they improve the process for next release. One should think it could be tested and automated pretty easy (one could of course be wrong)
if EveMon is not updated, Eve-Online is not fully updated. Everyone knows EveMon just as much a part of the game as the eve client IMHO.
Also, I'm sorry if I'm being rude... Download is the meaning of life, upload is the meaning of intelligent life http://eve.nikr.net - home of jEveAssets |
mramo
Pator Tech School Minmatar Republic
1
|
Posted - 2012.12.15 16:59:00 -
[43] - Quote
Golden Gnu wrote:if EveMon is not updated, Eve-Online is not fully updated. Everyone knows EveMon just as much a part of the game as the eve client IMHO.
That's why CCP should build their own skilltraining / EVE-monitoring app. EveMon provides a great improvement in playing the game. It is mandatory. CCP shouldn't let some not-controllable third party handle a crucial part of the game. |
Dei
Mamaleek
3
|
Posted - 2012.12.15 17:24:00 -
[44] - Quote
That's the opposite of what they should do.
- 3rd party developers are free for CCP
- CCP developers can focus on the game, not mini applications outside of the game
- If CCP developers replicate Evemon, not only have they pissed off the creator, no-one else is going to develop anything knowing that at any time CCP will make an 'official' application
- CCP don't have the resources to support all apps in existance, so why should EveMon be the only one?
- CCP stated they would support 3rd party developers with the SSD
- CREST is apparently somewhere around that would help EveMon
|
Cyerus
Galactic Dominion Eternal Strife
64
|
Posted - 2012.12.15 18:09:00 -
[45] - Quote
Version 1.0.2 is still missing the slot-layout of ships. dgmAttributeTypes (or something) seems to be missing the information. |
Dei
Mamaleek
3
|
Posted - 2012.12.15 19:43:00 -
[46] - Quote
In ramTypeRequirements, the data interfaces for invention do not have a damageperjob of 0. They should have a damageperjob of 0. |
Steve Ronuken
Fuzzwork Enterprises
851
|
Posted - 2012.12.15 19:49:00 -
[47] - Quote
Dei wrote:In ramTypeRequirements, the data interfaces for invention do not have a damageperjob of 0. They should have a damageperjob of 0.
This isn't a new thing. Ever since I first imported the dump it's been wrong on interfaces. (Just for reference. would still be nice to have sorted) FuzzWork Enterprises http://www.fuzzwork.co.uk/ Blueprint calculator, invention chance calculator, isk/m3 Ore chart-á and other 'useful' utilities.As well as mysql and CSV/XLS conversions of the Static Data Extract. |
Dei
Mamaleek
3
|
Posted - 2012.12.15 21:27:00 -
[48] - Quote
Yeah, was doing work on it today and wondered what people did, and apparently you're all just manually fixing the data you strange people, you. |
Golden Gnu
EVE University Ivy League
71
|
Posted - 2012.12.16 09:41:00 -
[49] - Quote
@mramo Yes, because CCP is known to create epic user interfaces - it's their key sell point AFAIK? Do not fix what is not broken. EveMon is great, no fix is needed!
@Dei Everyone fixing it sounds like a giant waste of time. Join force FFS! And let the lovely people that convert the SDE to MySQL release it when it's done, please...? Download is the meaning of life, upload is the meaning of intelligent life http://eve.nikr.net - home of jEveAssets |
Harry Pearce
The Intel Project
1
|
Posted - 2012.12.16 16:45:00 -
[50] - Quote
As a warning to anyone else bitten by this, not only are ship slots missing from the newest dump, but the eveIcons table has also disappeared. Whilst the iconID column had already been removed from invTypes, the table was present in the initial retribution dump. This means you need to be aware that trying to join on the iconID column in (at least) chrFactions, crtCertificates, dgmAttributeTypes, invCategories, invGroups, invMarketGroups and invMetaGroups won't work until you've recreated an eveIcons table suited to your needs from the YAML.
There are also some market groups[1] that have hasTypes = 1, but no corresponding rows in invTypes, and one[2] with hasTypes = 0, but no subgroups - possible worth pruning from your personal dumps if you're generating a market tree!
[1] SELECT invMarketGroups.marketGroupID, marketGroupName FROM invMarketGroups LEFT JOIN invTypes ON invMarketGroups.marketGroupID = invTypes.marketGroupID WHERE typeID IS NULL AND hasTypes != 0
[2] SELECT mg.marketGroupID, mg.marketGroupName FROM invMarketGroups mg LEFT JOIN invMarketGroups mg2 ON mg.marketGroupID = mg2.parentGroupID WHERE mg.hasTypes = 0 AND mg2.marketGroupID IS NULL |
|
Raven Seldon
Black Thorne Corporation Black Thorne Alliance
0
|
Posted - 2012.12.17 09:27:00 -
[51] - Quote
A bit weird it takes this long. Will the service be discontinued? |
Nik W
Critical Mass Inc. Nexus Fleet
4
|
Posted - 2012.12.17 23:04:00 -
[52] - Quote
Silence is bad
Can we get an update on the status of this? |
|
CCP Prism X
C C P C C P Alliance
1015
|
Posted - 2012.12.18 09:29:00 -
[53] - Quote
I'm not ignoring this, I'm just busy with other things. My plan is to do my utmost to get everything I need to have done, and fix the SDE as well, before I go on vacation. Please don't hate me, I'm delicate like a lotus flower in bloom. @CCP_PrismX EVE Database Developer and Expert Ranter Member of Team Pony Express |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
131
|
Posted - 2012.12.18 10:43:00 -
[54] - Quote
We don't hate you, we are just wondering why this is seemingly so low on CCP's priority list :(
Are new features / meetings really more important than fixing this? I mean, I know they are generally more fun to do than data exporting, but we (including the one whose task it is) will all become very happy when this is done.
It may have been asked before, but why is there not a standard program / script that does this whole dump every time an expansion is launched. I mean, the data structure cannot change THAT much every cycle... and this is not the first time we've had these issues. More like, it happens almost every time an expansion is launched... Developer/Creator of EVE Marketeer
|
|
CCP Prism X
C C P C C P Alliance
1015
|
Posted - 2012.12.18 12:42:00 -
[55] - Quote
Can't speak for the company, but this is low on my priority list because I'm actually not responsible for this and would rather finish up stuff first for which I will be held accountable. Not an ideal situation I know, but them's the breaks.
As to why this breaks all the time: Multiple reasons. Bad authoring + bad integration of static data between branches + bad code in the generation scripts that has never been bad before because it needed the bad authoring + bad integration to actually become an issue, was the first issue.
So I had to make code changes to shield you from broken relations, empty record names and such. And when you change code **** breaks.
And thus I'm really responsible for this. And that's why I'll be making another dump, fixed according to the issues I see here. I was going to wait a few days to sort out all the issues, as I understood from the first few posts that the issues weren't that critical anymore. Just some missing eveUnits records. Apparently that is not the case.
But other stuff comes up. I need to respond to that as well. It's no fun when what you need gets ranked on a low priority but it has to happen somewhere. Otherwise you're not prioritizing. But I was planning on doing this today, that might not work out anymore as I've lost a lot of time between the morning and noon due to unforeseen issues.
Today or tomorrow. Sorry for the delay. If someone would make a conscice bulletpointed list of all the remaining issues, I might consider hugging him next FanFest. But it's not like it's a huge issue for me to do it myself either. But then you don't get to feel like you helped! @CCP_PrismX EVE Database Developer and Expert Ranter Member of Team Pony Express |
|
Sidrat Flush
Eve Industrial Corp
27
|
Posted - 2012.12.18 12:54:00 -
[56] - Quote
We appreciate your "free"? Time and I realise it's a labor of love mixed with a dash of hate to get the SDE to us, I want to understand how the future YAML formatted SDE will be easier to update and release, assuming it will be of course.
What about CREST does it the SDE form part of it or is crest a separate entity?
The new home of the Eve Industrial Organiser is here. Enjoy the first in a series, EIO:Refinery now http://www.eve-files.com/media/corp/Sidrat/ Read about it http://eveindustrialorganiser.blogspot.co.uk/?m=0 |
|
CCP Prism X
C C P C C P Alliance
1015
|
Posted - 2012.12.18 13:02:00 -
[57] - Quote
I'm not much involved with the YAML files. I suspect they'll just create more issues, at least for a fair bit. They already have.
I see numerous posts here about people wondering why invTypes doesn't have an iconID, and that's because typeID->iconID and typeID->graphicID relations have been moved into YAML files. So we now have type information split between the restore data and the YAML data.
However, YAML data should be less prone to bullshit incidents such as the data we ship to you not being the 100% current data of the release branch. As it stands we have a code versioning system and a seperate data versioning system, which means the data can be out of synch with the code. Of course we've put a lot of effort into both avoiding that and handling such incidents gracefully so we don't notice this, f.e. all the unnamed types went out to TQ and broke nothing but broke lots of third party apps.
And CREST has nothing to do with the SDE. Just like the API has nothing to do with the SDE. Although obviously any application that uses the API (Legacy or, in the future, CREST) could benefit from having data from the SDE stored locally. @CCP_PrismX EVE Database Developer and Expert Ranter Member of Team Pony Express |
|
Callean Drevus
Icosahedron Crafts and Shipping Silent Infinity
131
|
Posted - 2012.12.18 13:10:00 -
[58] - Quote
CCP Prism X wrote:Can't speak for the company.... But then you don't get to feel like you helped!
My apologies for sounding so bitchy. It's nothing personal . Thanks for both the explanation and taking the time to do this. Bad code is a *****.
As to priorities, I understand why it is not high on YOUR priority list I'm just not sure why it's not higher on CCP's priority list.
I'll download the current dump and try to see if I find any issues. Developer/Creator of EVE Marketeer
|
Muscaat
EVE Markets
16
|
Posted - 2012.12.18 13:55:00 -
[59] - Quote
CCP Prism X wrote:...awesomeness
CCP Prism X, best Prism X.
Thanks for your continued efforts to help us out |
|
CCP Prism X
C C P C C P Alliance
1016
|
Posted - 2012.12.18 15:34:00 -
[60] - Quote
We've got a new SDE up guys, get it while it's hot!
IMPORTANT NOTICE: READ THIS OR YOU'LL GET ANGRY WHEN I MAKE FUN OF YOU FOR NOT HAVING READ THIS! Now that I got your attention, reading this will also make you angry. There is no iconID or graphicID in invTypes anymore. You will never find it there again. It has been moved into the types.YAML file we ship with the SDE zip file. If you're using a third party conversion that does not include this data in one form or another that's your converter doing something wrong. If you are doing a conversion and want this data back into a relationial database, you're going to have to get it from the YAML file.
Changelog: invTypes: Problem: table is missing the iconID column. Solution: Get them, along with the graphicIDs, from types.YAML. They will not be found in the official backup.
eveUnits: Problem: unitID 103, 122, 129 is missing. Solution: These units do not have a display name, do include units without a display name.
dogmaTypeAttributes, dogmaAttributeTypes: Problem: Missing slot attributes Solution: Fixing eve units to include the units will retain the slot data rathre than sanitizing it out as a broken relation.
invMarketGroups, invTypes Problem: Missing records. Solution: These are unreleased DUST records which do not belong in the SDE.
ramTypeRequirements Problem: Data Interfaces for invention have a damage per job of 1. Solution: This is how the data is authored and always has been. Checked the history. I can't fix things that are by design. Interfaces are special cased in our code. Maybe one day I get to rewrite the clusterintimacy that the S&I backend is.
invMarketGroups Problem: Market groups exist with hasTypes = 0, but still have types. Market groups exist with hasTypes = 1 but have no types. Solution: Check these relations and sanitize out bad data. I have no clue when GD will actually fix the data but at least I can fix your SDE @CCP_PrismX EVE Database Developer and Expert Ranter Member of Team Pony Express |
|
|
|
|
|
Pages: 1 [2] 3 :: one page |
First page | Previous page | Next page | Last page |