We rolled out a new release (version 5.25) of the Neo4j database on 2024-10-25 starting with AuraDB Free tier. On 2024-10-28 at 10:30 UTC we progressed the roll-out to AuraDB Professional, and then next to AuraDB Business Critical and AuraDB Virtual Dedicated Cloud tiers. As we maintain a 48 hours delay before we roll-out to all instances marked as production, those instances marked as production were not affected and were still running the previous version (5.24.2) of the database.
The new release version 5.25 contained a change aimed at handling the newer GQL syntax for the CASE expression to support the “simple CASE” mode. The implementation affected the optimization phase of such queries. The recently modified handling of the WITH clause also leveraged an optimization, and the two clashed when both involved in the same query, causing the Cypher statements to fail.
The existing test suite for Cypher covered the individual keywords but not the combination of these two exact cypher keywords in this particular sequence that would trigger both optimization phases.
Customers running Cypher statements that comprise a CASE (simple) inside a WITH clause saw their queries fail and produce the error “Neo.ClientError.Statement.SyntaxError”.
Customers reported issues on some specific queries without any changes on their side.
We took the decision to halt further progress of the roll-out of the release until we had a confirmed fix for this issue.
As we identified both the pattern triggering the error and its root cause, we communicated a workaround of adjusting the queries that use a “simple CASE” inside a WITH clause and modify them to use a “generic CASE”.
A fix was prepared, but to prevent other issues, we ensured it completed the full Cypher test suite before being rolled out (2024-10-29 at 13:35 UTC) to all our customers.
We don’t believe that the issues our customers experienced are acceptable for a DBaaS. We have carried out a thorough analysis of what went wrong in this situation. The actions that we are carrying out to ensure that nothing like this can happen again fall into multiple areas:
Communication: We will be reminding customers to have development instances and designate their production instance to better isolate them from possible issues during roll-out.