But the first one has been working for me for a couple of years. Depending on your version of IntelliJ, you may not have the second option as I mentioned in the screencast. Scala Breakpoints in IntelliJ Debugger Example I’ll mention it in the resource section below as well. If you are entirely new to using IntelliJ for building Scala based Spark apps, you might wish to check out my previous tutorial on Scala Spark IntelliJ.
#Spark intellij jar code#
But, feel free to translate the following to your own code right away as well. Pull the aforementioned repo if you want to take a slow approach to debug Scala in Spark. The ultimate goal will be to use your own code and environment. I know, I know, I caught you off guard their right? See Resources section below.Īnd here’s a shocker, another requirement is IntelliJ. Portions of this tutorial require SBT and I’ve provided a sample project available for you to pull from Github below. We’ll go through a few examples in this Scala Spark Debugging tutorial, but first, let’s get the requirements out of the way. Breakpoints are stored in IntelliJ (not in your application’s code).A debugger is provided in IntelliJ and is responsible for stopping at breakpoints and displaying the current program state.A breakpoint is a marker that you can set to specify where execution should pause when you are running your application.What’s the difference between Breakpoints and Debuggers? SCala Breakpoints and Debuggersįirst, let’s cover some background. It is assumed that you are already familiar with the concepts and value of debugging your Spark Scala code, but we’ll quickly review a few key concepts before diving into the screencast examples. We’ll break the topic of debugging Scala based Spark programs into two sections:Īs you’ll see in this tutorial there a few different options to choose from which depend on your Scala debug needs as well as if you are wishing to debug Spark in standalone mode vs debugging Spark job running on your cluster. These instructions and screencasts will hopefully allow you to start debugging Spark apps in IntelliJ and help me remember in the future. We’ll go through a few examples and utilize the occasional help of SBT. So, in this tutorial, let’s cover debugging Scala based Spark programs in IntelliJ tutorial. Debugging with Scala code was easy, but when I moved to Spark things didn’t work as expected. Have you struggled to configure debugging in IntelliJ for your Spark programs? Yeah, me too.
0 kommentar(er)
