When Sophie alerts with "Events are being discarded due to the retention settings. Please refer to the Data Management guide." This either means one of the following:
- Old events are accidentally being streamed into Sophie. Please make sure you are streaming the correct data.
- Retention settings within Sophie do not meet the dates of the events being streamed. administrators need to configure the retention in accordance with their data’s dates.
Configure Incident and Detection History Retention Settings
Incidents & Anomaly-Detection history usually take much less space than raw data, the default retention of Incidents & Anomaly-Detection history is 120 days.
You can review how much space is being used for storing incidents, and reconfigure the retention, under Settings --> Storage:
To review the size, expand the “Database table sizes” section.
To change a setting, click on the number itself, then enter the new number:
Careful! the change takes effect immediately – once you approve the new number older incidents are deleted.
Managing Anomaly-Detection Models Retention
It is very important to choose the right setting, as the anomaly detection models retention cannot be modified without deleting all existing data.
This setting is controlled via the YAML configuration file on the server.
Connect to the server, then open the configuration file:
sudo vi /opt/loom/.conf/loom_config.yaml
In this file, under the
graphite section, change
retentions to the desired value.
The retentions format is
Frequencies and histories are specified using the following suffixes:
s – second
m – minute
h – hour
d – day
w – week
y - year
For example, 1m:21d will store a data-point every minute, and keep the data-points for 21 days.