Influxdata, a cloud-based time-series database provider, has apologised to customers after it deleted database instances in Sydney and Belgium.
The company decided to discontinue services hosted in AWS Sydney and GCP Belgium, but said that despite of many customer notifications over “several months … some users were caught by surprise and lost the data they stored in those regions”.
An apology post, by founder and CTO Paul Dix, said the two closed regions “did not get enough demand to justify the continuation of those regional services.”
Users on the Australian instance are in the worst situation: “It appears at this time that for AWS Sydney users, the data is no longer available.”
The company is seeing if it is able to restore the last 100 days of activity for users on the Belguim GCP instance.
Having decided to close the services on June 30, Dix said, attempts to contact affected customers included email, calls to numbers held for those customers, notices on the instances’ home pages, along with helping customers who did respond to migrate to other regions.
However, as the company has since found out, when it took down the instances, some customers remained – and some of those have lost their data permanently.
One aspect of the shutdown stands out: Influxdata skipped the step of disabling the service without deleting anything, to see if anyone was affected – known a “scream test”.
The company said it would do that from now on, if it has to ever discontinue a service again.
“We would conduct a scream test by shutting down the services for one hour or one day to give users who did not register the notifications a chance to notice that their workloads were not running, and then turn the service back on for a short time period to give those users one last chance to migrate their data," the post stated.
The company has also promised to retain customer data for 30 days after any future shutdown, and to "redouble efforts to contact users who have not reduced their reads or writes within the 30 or 45 days before the end-of-life date for the region."
