Server Downtime and Data Loss

[MJC7890]

Now would be a good time to fix them dang dice... lol... thanks for keep CC rolling!!

[jtrydz]

I know this has frustrated a lot of people, but I just wanted to say thank you. I moved yesterday and it was a terrible move - bad strategy and awful rolling... I think I lost 6 guys taking a single army at one point. But today I got to do it all over again! I learned from my mistakes and had better troop placement, and it only took me 2 armies to kill that guy this time.

Thanks Conquer Club!!!

[Allsoran]

Yeah i won a game and then it was back on, fortunately won it again. Carry on ;o)

[Arama86n]

No big deal. I'm more worried about all the money I spent at the pub last night.
Life will go on, even for the whiners

[liedson]

Lost position in all games, but no problem, let´s play.

[bigg chief]

lmao

[OliverFA]

That event was sponsorized by Symantec Veritas Netbackup and IBM Tivoli Storage Manager.

In other order of things, congratulations for your quick work in solving the issue. I am happy with the new pace of updates even if they were the cause of this data loss (which seems not to be).

[Buffaloed]

You guys might actually think about making this a feature - kind of like a mulligan in golf. I get to replay my stupid moves and try to make them smarter. Seriously - having been in the IT business for 20+ years, bad things happen sometimes and you do the best you can. I think you provide an OUTSTANDING service for very little cost, and applaud you.

[TheMissionary]

Seems to only have reflected clan games. Every one my dice results were different and had the next moves planned. Nice twist to the competition CC

[Boniman]

In the late evening of Oct 3 (CC Time) one of our core system data tables suffered data loss and could not be recovered.

Unfortunately all data from the period between the backup and the event has been lost, including moves made, games started, chat, private messages and forum posts.

Sincerely,
A weary Admin crew.

Yes its ok, but people who paid premium membership in the last 24 hours also lost that ??
Could you please tell me who i should contact to get the premium membership back..

/Lasse

Premium restored, the team is working really hard

Shit happens. Quit whining cause u are a privileged premium member. Servers crash all the time, deal with it

[TrafalgarLaw01]

I've got like 30 game running, in most of ten it's not significant the move back in time, in like 6-8game i've lost positión, even 1 iwas certainly going to win in a six player game and now, not a chance, and perhaps i got the benefit 1-2 games, but well i have to play again, as much as i'm pissed off, i know it's not the end of the world

[Cornishson73]

It's only a game, certainly not about to hold one little glitch against you, keep up the good work guys!

[Crazyirishman]

I might be mad if I had any idea what actually happened or what is affected. Otherwise I'm clueless and this forced me to go outside which wasn't bad for it being such a beautiful in Amman

[CHECK-M8]

All games in progress need to be deleted. That is the only fair way to do it. There are too many games that will have a totally different outcome, that is a result of players finding out what other players had, or other players intentions, and then it is backed up and they are given a chance to use that information against them.

I had two games going where the game was ruined, one where I had 238 areas in USA 2.1, and now it has backed up and all 6 players are all attacking me because they now know how strong I am. I went from having 238 to now having less than 80. In another game (Hive) I set up a perfect "move next to last and then first next round" to regain an area that kept me in the game. Now I have no chance.

Both games, ALL games, that have been effected by the backup need to be deleted IMO. That is the only fair way to do it. Or at the very least don't subtract any losses in any games being played during this time. Give out the winners their scores but don't subtract the losses, that I could live with also.

[eddie2]

nothing needs deleted went back 24 hours so if you made a move make the same one if there are others who played ask them to do the same one.
problem solved this does not really affect the site. and is a glitch that can easily happen on a site this big.

[drunkmonkey]

The random outcome of my rolls was lost at a random point, and the new random results are different! It's an outrage!

[poptartpsycho18]

I didn't like the drop in that game anyway.

[hushovd]

Mwah...give me a 1000 points bonus and i pretent nothing happend

Shit happend...in 5 years i play here its the first time...

[patrickaa317]

In other order of things, congratulations for your quick work in solving the issue. I am happy with the new pace of updates even if they were the cause of this data loss (which seems not to be).

Just an FYI - It was confirmed earlier in the thread that the data loss and release updates are unrelated.

Also, big thanks for communication and details around this. You can easily tell who understands IT risks and who doesn't by their comments. Anyone that knows how this all works, is quite appreciative because it's stuff we can relate to on a daily basis.

On a side note, this is going to cause me to back all my stuff up at work and home again, been a while since i did it last....

[TheProwler]

In the late evening of Oct 3 (CC Time) one of our core system data tables suffered data loss and could not be recovered.

I find this interesting...

Surely you have (lots of) storage redundancy...you should be able to recover from hardware failure without reverting to a backup.

Was it bad code? Did you implement a change that wasn't properly tested? Is your system documentation lacking and your developer(s) getting overwhelmed?

I'm just curious. Downtime is something that might happen when a disaster occurs. But having to go to a backup? Shit, somebody fucked up badly.

The only efficient solution was to roll back our entire database to the most recent backup, which happened to be approximately 24 hours before.

I think the word that is screaming at me in that sentence is "efficient".

Because I've designed a number of systems with 100+ tables...and if one the of the "core" tables somehow "suffered data loss", I would expect to be able to recover the vital information from those tables based on their child and parent tables data, and other related tables. Whatever table was lost, you should be able to re-build it with data other tables.

I know there might be some information loss like exact time of turns, but that wouldn't be a big deal. You could look at the physical order to the rows and estimate the time of turns. Obviously I have to speak in general terms because I don't know shit about your design or what table was lost. But you can go to a backup for everything up to the last backup, and then "fix" the data for the time since the last backup.


I guess without going on and on, I think you chose the word "efficient" because you know that there was a better solution with respect to recovering all the turns, but you were either too fuckin' lazy to do the work to take the site down and fix the problem properly, or because you don't understand the data well enough to fix the problem in an acceptable amount of time.


All these pats on the back that people are giving you shouldn't fool you; reverting to a backup is called "Failing".

[casper]

nothing needs deleted went back 24 hours so if you made a move make the same one if there are others who played ask them to do the same one.
problem solved this does not really affect the site. and is a glitch that can easily happen on a site this big.

I don't think players are going to make the same moves.... It's already too late in my case. I won a foggy escalating game last night. This morning it had been reverted and the player before me got to go again, now knowing where everybody was. He took the opportunity to eliminate me. It sucks. I lost probably over 120 points and Colonel ranking because of it. Yes it's just a game but I'm still pissed off.

[casper]

I guess without going on and on, I think you chose the word "efficient" because you know that there was a better solution with respect to recovering all the turns, but you were either too fuckin' lazy to do the work to take the site down and fix the problem properly, or because you don't understand the data well enough to fix the problem in an acceptable amount of time.


All these pats on the back that people are giving you shouldn't fool you; reverting to a backup is called "Failing".

Well said. I don't know why everyone is kissing the owner's ass for a colossal blunder.

[bartmancol]

Hey ho - just another chance to role another load of of shit dice again cheers CC

[berlin1945]

the other folks are getting to move twice. they will no doubt undo their mistakes incurred on the last turn. but what happened on their last turn was good for me. this isn't fair. this isn't conquer club.

[sm1ddy]

I've got a player in a large game who was dead and is now back in the game, he was kicked out for missing too many turns and now we all have to wait on him again. He has a large force right up against me and I will be LIVID if he now logs in and attacks me.