Spark Performance Monitoring Tools – A List of Options

Spark performance monitoring tools

Which Spark performance monitoring tools are available to monitor the performance of your Spark cluster?  In this tutorial, we’ll find out.  But, before we address this question, I assume you already know Spark includes monitoring through the Spark UI?  And, in addition, you know Spark includes support for monitoring and performance debugging through the Spark […]

Spark Performance Monitoring with History Server

Spark Tutorial Perf Metrics with History Server

In this Apache Spark tutorial, we will explore the performance monitoring benefits when using the Spark History server.  This Spark tutorial will review a simple Spark application without the History server and then revisit the same Spark app with the History server.  We will explore all the necessary steps to configure Spark History server for […]

Spark Performance Monitoring with Metrics, Graphite and Grafana

spark-performance-monitor-with-graphite

Spark is distributed with the Metrics Java library which can greatly enhance your abilities to diagnose issues with your Spark jobs.  In this tutorial, we’ll cover how to configure Metrics to report to a Graphite backend and view the results with Grafana for Spark Performance Monitoring purposes. Spark Performance Monitoring Background If you already know […]