SearchStax Help Center

The SearchStax Managed Search service backup feature requires abundant free disk space in order to succeed. Backups may fail intermittently when the size of the index(es) is more than a third of the disk space allocated to each node of the cluster. As the index size approaches half of the disk space, backups will fail consistently. See Why do Backups Fail?

But how can you tell how much disk space is available? In early versions of Solr, this was pretty mysterious. From Solr 6.0 onward, however, you can pull this information out of the Solr Dashboard, the Managed Search Dashboard, and the Monitoring graphs. You just have to know where to look for it.

Allocated Disk Space

To determine the amount of disk space allocated to each node of your deployment, look on the Deployment Details page of the Managed Search Dashboard. The HW Specs column shows the original amount of disk space on the node. In this case it is 8.0 GB.

Note, however, that it is possible to add disk capacity to a deployment (see the section near the end of this page), but that the added space is not reflected in this display.

Disk Usage

To determine the amount of disk space in use on a node, open up the Solr Dashboard by clicking on the HTTP Solr Endpoint of the deployment. (You might need to make adjustments to your IP Filter and Solr Basic Auth security settings.)

Navigate to the Cloud > Nodes display. The data table reveals the Disk Usage of each node in the cluster. In this case, 3.5 GB of the disk is in use.

In most deployments, this number is a good proxy for the size of the index(es), but it also includes some disk space devoted to configurations, logs, and JAR files.

Disk Space from Monitoring Graphs

The graph of Disk Usage reveals some more detail. This is a graph from a newly-created NDN1 node. Initially, the disk usage was 0.43 GB.

We then restored a 13-million-record collection from a backup. (This was the same index as shown in previous examples.) Total disk usage grew to 3.95 GB. (The graph shows the upper limit of available disk space as a dashed line at 7.75 GB.)

Note that 3.95 GB – 0.43 GB = 3.52 GB, which is in good agreement with the estimate we obtained from the Solr Dashboard.

Will the Backup Succeed/Fail?

Here is a heuristic for predicting backup success/failure. If 1.5 times the occupied disk space is greater than the remaining disk space (allocated – occupied), then the backup attempt will fail. For example,

1.5 x 3.5 GB > (8.0 GB - 3.5 GB)

so the backup attempt failed.

This failure resulted in automatic email to the users of the account, notifying them that the system will need more disk space.

The failure threshold varies from one deployment to another, and even from one node to another. This is a rule of thumb only.

Adding Disk Space

It is possible to add disk space to the nodes of a deployment by making a request of SearchStax Support. We can add disk space in blocks of 50GB for $25/node/month.

For instance, we added 50 GB to the example deployment shown above. We could see the added memory in the graph:

The bump in the graph is the temporary disk usage increase due to the next backup attempt. The system needed almost 10 GB for this event. This one succeeded. The final backup file was just a little larger than the occupied disk space.