Open Source MLOps for Timeseries Forecasts showcase

Does anyone still remember the time when AI wasn’t just about large language models and GPTs? I’ve always had an interest in observability, machine learning, and DevOps, and all these come together perfectly in my Machine Learning Operations (MLOps) time series data project.

I’ve been collecting data from various APIs and applications for years: public parking information, personal health data, UPS battery status, revenue performance, to name a few. When observing simple metrics over time, you can deduce all kinds of trends, detect and alert on anomalies. For example, in the image below, you can see a continuously updating, naive forecast of the available parking places in Amsterdam P+R Noord.

After storing the data in time series databases, graphing the data in pretty dashboards, and running various forecast models, I have been researching the ideal open-source machine learning pipeline. Such a pipeline should be able to continuously create predictions on any time series in a flexible and scalable way, so that I could also easily replace the forecasts with heavier AI workloads, such as large language model embedding generation, LLM training, or anything in between.

The most recent setup now uses @InfluxDB for storing the time series, @Darts to create the forecasts, @Argo workflows to consistently run the pipelines, @Kubernetes clusters for scalable compute, and @Grafana for visualisation. The technical writeup was recently featured in the @TechMiners newsletter and includes references to earlier blogs with more details on the different moving parts.

While there are awesome MLOps SaaS platforms out there, it can be very refreshing to realise you can achieve similar results with an open-source software stack. Definitely don’t miss out on the results of years of experimentation that have taken place before this last piece could be written!

And as @Mitko Vasilev always says at the end of his AI-related posts: Make sure you own your AI. AI in the cloud is not aligned with you; it’s aligned with the company that owns it. (Thanks, Mitko!)

https://www.linkedin.com/feed/update/urn:li:share:7253017432158613504/