Why we were down

Why we were down

Dear users, we received quite a lot of messages of users that were concerned regarding the IMAP support removal for free accounts.

As may know, we suffered a long, major outage affecting all our services.

This is the consequence of series of bugs concerning major operational systems.

At the beginning of the year, we planned a data migration between our former infrastructure provider and a new one. As a consequence, we decided to lighten operational services in order to simplify the migration scheme; this was a terribly wrong idea as backup and data replication systems were concerned too.

As data migration should have been done quickly, this was not seen as a huge issue at the time but it should have been.

Due to series of constraints and change of priorities, data migration was never fully performed; but, we should have, at least, put critical services online again.

The 9 of September we suffered a hardware issue on servers’ managing our whole email dataset. As a consequence, we had to move drives to a new, empty box; losing in the process precious filesystem data hosted in the BBU backed cache.

We were never able to recover any useful data from the concerned system as we store data in a not-recoverable pattern without filesystem data structure.

As a consequence, you will suffer a big gap of what is available in your Inbox, we have no words and are sincerely and deeply sorry about the whole situation.

We are open to requests for any additional comments and answers.