Database instance write performance impacted

Incident Report for Neo4j Aura

Postmortem

What happened

Between August 22 and August 28, 2025, a performance issue affected some of our database services following a recent update. Our team quickly responded and discovered that the problem was due to a bug in the latest update, which caused certain memory settings to be incorrectly configured. We promptly applied the previous version of those settings to stabilize the affected databases. By August 28, all databases were successfully transferred to a new, stable version, and the incident was fully resolved.

The issue was caused by a misconfiguration in the system's data processing module. Specifically, an incorrect parameter setting in the data pipeline led to a bottleneck, which slowed down the processing speed. This misconfiguration affected the way data was being queued and processed, resulting in delays. Our technical team has identified the root cause and implemented a fix to ensure that the data pipeline operates efficiently, preventing future occurrences of this issue.

How the service was affected

Some customers reported a performance change on their instance. 

What we are doing now

Neo4j remains committed to providing reliable service and is implementing additional safeguards to prevent similar incidents in the future.

New mitigations being deployed:

  • Enhancing our monitoring systems to detect similar issues faster
  • Implementing additional automated checks to prevent service disruptions
  • Improving our service resilience to handle similar scenarios
  • Introducing dynamic configuration management to ensure seamless updates
  • Enabling multiple configuration defaults to better manage database priorities
  • Improving notification systems for faster response to potential issues
  • Enhancing our tools to streamline database management and transfers
Posted Oct 06, 2025 - 13:32 UTC

Resolved

The full fix for the solution has now been deployed across the estate and this issue is now resolved.
Posted Aug 27, 2025 - 16:46 UTC

Update

The full fix for the solution is continuing to be deployed across the estate and are continuing to monitor the situation.
Posted Aug 27, 2025 - 11:16 UTC

Update

We are continuing to monitor for any further issues.
Posted Aug 27, 2025 - 06:36 UTC

Update

The full fix for the solution is starting to be deployed across the estate and are continuing to monitor the situation.
Posted Aug 26, 2025 - 23:12 UTC

Update

The full fix for the solution is starting to be deployed across the estate and are continuing to monitor the situation.
Posted Aug 26, 2025 - 20:13 UTC

Update

The full fix for the solution is starting to be deployed across the estate and are continuing to monitor the situation.
Posted Aug 26, 2025 - 17:17 UTC

Update

We are continuing to monitor the situation. The full fix for the solution is currently undergoing final testing and will start deployment once testing is completed.
Posted Aug 26, 2025 - 14:48 UTC

Update

We are continuing to monitor the situation whilst a full fix for the solution is prepared.
Posted Aug 26, 2025 - 10:53 UTC

Update

We are continuing to investigate this issue.
Posted Aug 26, 2025 - 04:06 UTC

Update

We continue to work towards the issue resolution.
Posted Aug 25, 2025 - 21:44 UTC

Update

We continue to work towards the issue resolution.
Posted Aug 25, 2025 - 17:40 UTC

Monitoring

The issue impact has been refined and currently could impact AuraDB Virtual Dedicated Cloud and Business Critical instances only. We have identified the cause and the faulty component has been reverted for the majority of instances.

We will continue to monitor the situation, while the full fix is prepared.
Posted Aug 25, 2025 - 15:34 UTC

Update

We are continuing to investigate this issue.
Posted Aug 25, 2025 - 13:47 UTC

Update

We are aware of an issue that could result in a degradation in write performance for all Aura instance. We have identified the cause and are preparing a fix.
Posted Aug 25, 2025 - 13:36 UTC

Investigating

We are currently investigating this issue.
Posted Aug 25, 2025 - 13:34 UTC
This incident affected: AuraDB Virtual Dedicated Cloud (*.databases.neo4j.io) (AuraDB Virtual Dedicated Cloud on AWS (*.databases.neo4j.io), AuraDB Virtual Dedicated Cloud on Azure (*.databases.neo4j.io), AuraDB Virtual Dedicated Cloud on GCP (*.databases.neo4j.io)), AuraDB Professional (*.databases.neo4j.io) (AuraDB Professional on AWS (*.databases.neo4j.io), AuraDB Professional on Azure (*.databases.neo4j.io), AuraDB Professional on GCP (*.databases.neo4j.io)), AuraDS (*.databases.neo4j.io) (AuraDS on AWS (*.databases.neo4j.io), AuraDS on Azure (*.databases.neo4j.io), AuraDS on GCP (*.databases.neo4j.io)), AuraDS Enterprise (*.databases.neo4j.io) (AuraDS Enterprise on AWS (*.databases.neo4j.io), AuraDS Enterprise on Azure (*.databases.neo4j.io), AuraDS Enterprise on GCP (*.databases.neo4j.io)), AuraDB Business Critical (*.databases.neo4j.io) (AuraDB Business Critical (*.databases.neo4j.io) on AWS, AuraDB Business Critical (*.databases.neo4j.io) on Azure, AuraDB Business Critical (*.databases.neo4j.io) on GCP), Aura Graph Analytics (Aura Graph Analytics on AWS, Aura Graph Analytics on Azure, Aura Graph Analytics on GCP), and AuraDB Free (*.databases.neo4j.io).