Troubleshooting

Cluster generates errors on startup

To get information on a running transaction, use the spark-discovery-yt utility:

spark-discovery-yt <cluster-name> --discovery-path my_discovery_path

A task list for this transaction always includes a master, a worker, and the Spark History Server. If the master is non-functional, so are the other components. If everything is running properly, the worker log will include data on the accepted task and the start of task execution.

If a task produces an error at startup, you need to:

• Open the task log via Master Web UI. It has links to the driver and the executor logs.
• Open Worker UI via Master Web UI and review the application startup sequence.
• View the Event Log.