In the SearchStax Managed Search service, a 503 Updates are Disabled Error means that Zookeeper has blocked updates because a Solr node has failed.
2023-05-31 17:05:20,059 ERROR - received unexpected error status from server: 503 ({"responseHeader":{"status":503,"QTime":588},"error":{"msg":"Cannot talk to ZooKeeper - Updates are disabled.","code":503}}
In one instance, a node failed after running for fifteen minutes at a load of 100. (The maximum sustainable load for this node was 8.)
The swap memory was completely utilized, causing the node to become unresponsive.
When a system is this overloaded, we often find that too many commits are clogging the system.
In this instance, SearchStax engineers had to restart the nodes one-by-one to resynchronize the indexers. Updates were suspended temporarily, but query service was not disrupted.
Questions?
Do not hesitate to contact the SearchStax Support Desk.