Shoutout: This work was made possibly by Circonus -- the monitoring system with full histogram support.
Gathering all kinds of telemetry data is key to operating reliable distributed systems at scale. Once you have set-up your monitoring systems and recorded all relevant data, the challenge becomes to make sense of it and extract valuable information, like:
Statistics is the art of extracting information from data. In this tutorial, we address the basic statistical knowledge that helps you at your daily work as a system operator. We will cover probabilistic models, summarizing distributions with mean values, quantiles, and histograms and their relations. Also advanced topics like time series forecasting and scalability analysis will be touched.
The tutorial focuses on practical aspects and will give you hands on knowledge of how to handle, import, analyze, and visualize telemetry data with UNIX command line tools, gnuplot, and the iPython toolkit.
If you have access to a machine with docker installed, you can boostrap an interactive working environment with a single command:
$ ./docker.sh [...] # # Data Science 4 Effective Operations # # starting jupyter notebook&lab ... done # # Notebook: # * local url: http://0.0.0.0:9999/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ # * public url: http://188.8.131.52:9999/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ # # Lab: # * local url: http://0.0.0.0:9998/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ # * public url: http://184.108.40.206:9998/?token=F2AlHtJBvHIqoLFEVfbMnUVFkcpFlJuZ
Sign-up to the mailing list, to get notified about upcoming Statistics for Engieners events.
This workshop has been held in at a number of events in slightly different forms.
See the corresponding subfolders for the presented content.
If you want to be informed about upcoming events consider watch out for the following hashtag on Twitter: #StatsForEngineers
Event: 2016-06-29 Monitorama, Portland USA
A writeup of the material was published in print by the CACM and the ACM Queue magazine.
ACM Queue 14/1: https://queue.acm.org/detail.cfm?id=2903468
CACM Vol. 59 No. 7 (paywalled): https://cacm.acm.org/magazines/2016/7/204029-statistics-for-engineers/abstract