Monitor your Apache Spark job definition

Using the Spark job definition item's inline monitoring, you can track the following:

• Monitor the progress and status of a running Spark job definition.
• View the status and duration of previous Spark job definition runs.

You can get this information from the Recent Runs contextual menu in the workspace or by browsing the Spark job definition activities in the Monitor pane.

The Spark job definition inline monitoring feature allows you to view Spark job definition submission and run status in real-time. You can also view the Spark job definition's past runs and configurations and navigate to the Spark application detail page to view more details.

You can access the job runs associated with specific Spark job definition items by using the Recent runs contextual menu on the workspace homepage.

To view all the Spark applications related to a Spark job definition, go to the Monitor pane. Sort or filter the Item Type column to view all the run activities associated with the Spark job definitions.

The next step after viewing the details of an Apache Spark application is to view Spark job progress below the Notebook cell. You can refer to

• Spark application detail monitoring