1.4.0 Release Notesedit

The changes listed below have been released for the first time in Loud ML 1.4.0.

Note

Please raise issues on Github if you experience compatibility issues on your system.

Overviewedit

1.4 is a significant update in several areas, including a security update.

If you’re using release 1.3.x, please migrate to 1.4.0 to avoid sensitive information being leaked via the REST API. Now, onto new and improved functionality.

New awesome functions include:

  • Forecast: in 1.3, you could already forecast stationary data (ie, where the patterns are centered around a mean value). Now, in 1.4 you can also forecast non-stationary data that deviates from its mean. This makes Loud ML a good fit for predictive maintenance applications!
  • New data connectors natively and seamlessly supported out-of-the-box include:

  • Live anomaly detection. We’re thrilled to roll out significant enhancements in this area. Read on to learn more.

Improvementsedit

Several major enhancements to live anomaly detection are included in Loud ML 1.4:

  • #36: all output predictions are automatically tagged with the same tag(s) used in the model definition; for example, if your model contains a match_all section with two conditions matching key port=123 and dc=ams1 then output predictions written to the prediction_%model_name measurement will automatically contain the two tags: port=123 and dc=ams1.
  • Lower and upper: All output predictions written to the measurement prediction_%model_name will automatically include a lower_%feature and upper_%feature fields that represent the floor and ceiling value for predicted values, providing a 95% confidence interval.
  • Scoring: all output predictions written to the measurement prediction_%model_name automatically include a score field in range [0, 100], and is_anomaly (True|False) tag. This will allow users to plot the score value, tracking the anomalies over time and integrating easily with third-party alerting applications.
  • Automated thresholding: if you set min_threshold=0, and max_threshold=0, Loud ML will automatically calculate the optimal values during training, so if you don`t know which threshold values to use, leave it at zero! Note: you can override these values later on using the REST API.
  • Automated annotations: for each anomaly, annotations compatible with InfluxData Chronograf database will automatically be inserted in the database if you want them to be. Note: this feature is only compatible with InfluxDB at the time of writing. Learn more in the documentation.

Data model improvements:

  • You can now use default: previous in your data model definition. This tells Loud ML to fill missing values with previous values when the values don`t exist in the TSDB for a given time interval.
  • You can now use transform: diff in your data model definition. This allows Loud ML to forecast non-stationary data; ie, data that isn`t centered around its mean.

New configuration options:

  • server.mse_rtol option was added to config.yml file. The relative tolerance is used to compare MSE values during anomaly detection. The error tolerance function is similar to NumPy function: absolute(a - b) <= (atol + rtol * absolute(b))

Bug fixesedit

The 1.4 release introduces important bug fixes:

  • Important security fix: no longer leak password or other sensitive information via the REST API. If you are using 1.3.x release, please migrate to 1.4.0!
  • Enhanced support and bug fixes for Elasticsearch 6.x
  • Several bug and stability fixes in the loudmld API running background jobs.
  • Fixed bug #17: loudmld not listening on TCP port if /var/lib/loudml contains unexpected content

Breaking changesedit

Few breaking changes have been made:

  • Since doc_type was removed in Elasticsearch 6.x (see removal of types) Loud ML is now writing predicted data points to a new index named according to each model’s name.
  • loudmld _forecast API now accepts a bg parameter. If bg=true, it will run an asynchronous job in the background. On the contrary, if bg option is missing or false, it will run synchronously and return the forecast result in JSON format.

Acknowledgementsedit

Thanks to our community for downloading the packages, and for their excellent feedback!

Thanks also to Anais Dotis-Georgiou from InfluxData for her well-written tutorial on how to implement Loud ML with InfluxDB.

As always, thanks to the hard-working and dedicated team who have made Loud ML a success.

Please let us know on Github about the new features you’d like to see in the next releases!