What happened

Half of Aura the Professional environment on GCP in ‘europe-west1’ region experienced availability issues for their instances.

As a result of an Aura component roll out, the Aura database ingress layer for two Professional tier environments: ‘europe-west1’ (GCP) and ‘eastus’ (Azure) were not automatically updating to reflect Neo4j cluster topology changes.

We initially called out only the ‘europe-west1' (GCP) affected environment (we provided a mitigation: use of an unaffected environment in that same region) but missed out 'eastus’ (Azure).

The recovery was to re-establish the connection at the database ingress level by restarting the database ingress pods in order to refresh the Neo4j cluster topology.

How the service was affected

There was an impact for some Aura Professional tier customers who had database instances in the affected environments, this would have been seen as intermittent unavailable during the duration of the roll out.

What we are doing now

We conducted a root cause analysis and we have identified a known issue with the underlying third party component used to implement the database ingress.  

Actions we are taking based on this incident: 

• Correction - we have implemented and rolled out the recommended fix for the known issue on the third party component.
• Detection - Introduce logging improvements of existing logs to include items to inform when similar events are occurring and then detecting and alerting accordingly.