👀Live monitoring of ML and software metrics on one platform
In cases where machine learning is discovered, it is important to constantly monitor data and structures. Even the ML model itself has remained the same, the nature of the data can change, which can significantly affect user experience. There are many software monitoring platforms on the market today, where various system and business metrics are collected to reflect the most important platform monitoring data and generate a platform. For example, Grafana, Datadog, Graphite, etc.
There are also tools for monitoring ML machine learning systems like Neptune, Amazon SageMaker Model Monitor, Censius, and other MLOps environments. But it is possible to prevent monitoring the operation of a machine learning system with classical software engineering monitoring on the same platform. This is achieved with New Relic, a telemetry platform for remote monitoring of mobile and web applications that allows you to collect, receive and be notified of all telemetry data from any source in one place. Thanks to a large number of open source tools, New Relic can work with data sources and sinks.
Sending data from the ML system to New Relic is implemented using the ml-performance-monitoring Python library with Quick Source, which is available on GitHub (https://github.com/newrelic-experimental/ml-performance-monitoring).
https://towardsdatascience.com/monitor-easy-mlops-model-monitoring-with-new-relic-ef2a9b611bd1