Shard down for emergency maintenance [back up]

[Red Squirrel]

There seems to be an issue with the database and I am currently investigating.

It seems something happened to the items database table and all items have been wiped. At this point I am not sure what is the extent of data loss. I am going to be looking at backups.




FINAL UPDATE[5:42AM ET]: I have been monitoring the shard, and also the test and dev environments for good measure. Everything seems ok now. Unfortunately can't figure out what would have caused this, so we'll just have to hope there's some odd explaination and that it won't happen again. As a precaution I have modified the backup job to backup the database multiple times a day.

UPDATE[11:09PM ET]: I have not figured out what caused this but consider the shard up and you can resume normal playing. I will keep monitoring it to see if it happens again. I'm going to call this one a freak accident and hope it does not happen again...

UPDATE [9:53PM ET]: We have confirmed that this backup is proper. So the shard will be reverting to Apr 20 06:00:0. I may reboot it and do other maintenance but as far as your stuff is concerned the loss is only going to be about a day.

UPDATE [9:29PM ET]: Shard may appear up at times, but consider it down. I am trying different backups and still playing around. It is subject to revert back. I still cannot explain what happened but I do appear to have a backup from Thu Apr 20 06:00:08 so if it works I will go with that one.


Archived topic from AOV, old topic ID:6689, old post ID:38988