| Author |
Thread Statistics | Show CCP posts - 17 post(s) |
|

Valar

|
Posted - 2006.05.17 17:42:00 -
[1]
A lock issue in the database was causing the inventory system to time out on most actions. We are trying to find the cause of the deadlock, and will start up again in a moment. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.17 17:52:00 -
[2]
Coming up again now. We haven't found what caused this yet, but if it happens again we know what to look for.
This is a side effect of the upgrade to SQL Server 2005, either an issue in SQL Server 2005 or something in our code not working well with the new SQL server. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.17 18:09:00 -
[3]
Ok, we haven't found the cause for the deadlock yet, however, a new feature in SQL server 2005 might make the lock that caused this issue to be unnecessary and if that change passes testing, it will be deployed with the next patch or as a hotfix. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.17 18:14:00 -
[4]
Originally by: Raven Aure
Having just done two SQL 2005 installs (one for a bespoke trade system and the other for an SAP deployment), I'm interested.... what mode are you running SQL in - 2005 native or 2000 Backwards Compatibility?
Anything I should know about?!
The TQ db is configured for compatabilty level 90. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.18 01:19:00 -
[5]
Originally by: Nelson Vandermark
Originally by: Valar Ok, we haven't found the cause for the deadlock yet, however, a new feature in SQL server 2005 might make the lock that caused this issue to be unnecessary and if that change passes testing, it will be deployed with the next patch or as a hotfix.
Valar,
Have you guys discussed the removal of redundant records from the databases?
We are constantly deleting trashed items, old logs and other stuff thats not being used anymore. Without doing that we would make the itemID integer column explode and the database size would increase very fast. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.18 02:39:00 -
[6]
Originally by: Luigi Thirty
Originally by: Valar We are constantly deleting trashed items, old logs and other stuff thats not being used anymore. Without doing that we would make the itemID integer column explode and the database size would increase very fast.
Just something I'm curious about. How high is the itemID value right now? Millions?
We have already reached the max value of the itemID, so before we started reusing itemIDs we had around 2.100.000.000 items in the table. At any given time we have at least tens of millions of free itemIDs and many of the daily downtimes are used to delete items from the junkyards to add free itemIDs. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.18 10:35:00 -
[7]
We can't change the itemID column to bigint because that would increase the size of the items table at least 3 times and all its indexes by at least two and we can't use negtive numbers because of how the inventory system is implemented. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.18 23:04:00 -
[8]
Originally by: Inspiration
Assumping the primary key of the item table is the 'itemID' field and it is also the unique clustered index (common practice) of that table, a move to bigint will 'widen' the table by 4 bytes. The same is true for every non-clustered index that the table has, as the itemID is also stored in those (even if not part of the index fields).
A tripping of the table size, on top of the wider indexes is kind of shocking and to me only plausible under serveral conditions.
This is the thing:
Quote:
1) The table has a LOT of self-references relative to the number of fields (they would all widen)
and to add to that, there are indexes on the self-reference fields ownerID and locationID.
Of course we *could* go to bigint if the need arises but it would require us to invest in more solid state SANs or require that we move tables from the RAMSAN and that would result in a performance hit. So while we can keep the itemID as a signed 32-bit integer with minimum effort, we are going that way.  ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
|

Valar

|
Posted - 2006.05.19 02:05:00 -
[9]
Originally by: Nelson Vandermark
Originally by: Valar
We are constantly deleting trashed items, old logs and other stuff thats not being used anymore. Without doing that we would make the itemID integer column explode and the database size would increase very fast.
Oh I see well you guys certain know your stuff thatĘs for sure, although is there a possible of widening the range on objects which could be classed as redundant records?
You would only need to look at some of the oldest corporations in game to see this information and it would be objects that reside under corporation tab, I think every thing under that tab records everything, I was think about all the corporations which are not active and where created 3 years ago, thatĘs a lot of records which never get accessed and there wouldn't be any point in keep the data, unless it holds some historical importance.
If a corporation has 0 members, the corporation itself is moved to a location called something like "EVE Corporation Graveyard" and all items owned by the corp are deleted. So a deleted corporation takes up 1 record in the database and that record *has* to be there for f.ex. employment history. Other data that old corporations have f.ex wallet transaction logs, member role auditing and other stuff like that is deleted when we do regular cleanups. Characters that are deleted lose all their items, are moved to a corporation called "Doomheim", moved to a location called "EVE Character Graveyard" and only keep their skills.
I recently checked how many itemIDs we would gain by deleting all items owned by characters that are on accounts that haven't been active for X many months.
By deleting ALL items on ALL characters on accounts that were not active at the time of the check(including accounts that went inactive the day before), we would only get around 30mil free itemIDs. Thats around the same number of items as we delete in two downtimes, so its not worth the grief it would give returning customers.
There is some redundant data, but we try to keep it at a minimum. ------ Valar Database admin - Server operations team CCP Games How to write a good bugreport |
|
| |
|