In an effort to get better at working with Kinja’s backend codebase (which is largely in Scala), a group of us at FMG have been taking the Functional Programming Principles in Scala course on Coursera. So far I’m having a great time!
In the process of ramping up, I learned about an interesting feature in IntelliJ IDEA: Scala Worksheets! I’m pretty sure Scala IDE for eclipse implemented them first and that JetBrains implemented it later. Either way, they look like this:
As far as I can tell, they work like this:
- Worksheets are named with the .sc extension, rather than .scala
- Worksheets consist of a single object literal with some number of statements in the body
- You can run a whole worksheet by clicking the green button
- When you run a worksheet, the result of each line in the object block gets rendered on the right hand panel
In the Coursera course, Odersky uses them when writing simple code snippets to demonstrate new concepts. In terms of the user experience, it has a more casual feel and a quicker turnaround time as compared to a traditional full-on sbt project that you would get with a REPL, combined with the ability to go back and edit things you’ve ran as with a script file.
Being a data engineer, I of course immediately noticed the similarities to Jupyter, which we use to prototype and share analyses with other engineers and analysts. Moreover, as someone who googles open source projects as a hobby, I came up with an idea for setting up a Scala notebook on my desktop, and not too much later I had it working! I’m honestly pretty proud.
Instead of Jupyter, I decided to try out nteract. nteract is a Jupyter-compatible notebook written in electron. The upshot of this is that I get a notebook app that doesn’t require me to run a webserver, plus the ability to use arbitrary Jupyter kernels. You can read more about nteract on their blog!
Jupyter’s kernels are what allow it to run arbitrary languages other than python. As long as a kernel’s been written for it, you can use it (and there are many). Common alternate environments for Jupyter include R, Julia and pyspark. Today we’re installing a Jupyter kernel for Scala.
As it turns out, there’s actually a pretty decent (so far) Scala kernel for Jupyter. It comes with support for many of Ammonite’s features (Ammonite being a collection of projects around using Scala in a REPL or as a shell), as well as (not tested by me, yet) support for inline plot generation. Sick!
Getting all of this to work is shockingly easy. My system, a macbook, already has java installed, Scala installed with homebrew, and python installed via anaconda. You can likely follow this process with a few modifications on Linux. On Windows, you might have some problems. If you try either of these platforms, leave a comment! With that, the process is as follows:
- Install nteract. I used the dmg.
- Follow the directions for installing the python kernel. I’m using anaconda, so I used the anaconda install directions.
- Following jupyter-scala’s Quick Start directions, download this shell script. What this script does is download coursier to a tmpfile, which I’ve been describing to people as “npx but for Scala”, uses it to run a pure scala installer, and then delete the tmpfile. Dope! I edited the shell script to use Scala 2.12, though this likely works with Scala 2.11 as well.
- Now you should be able to fire up nteract, click “Language” in the top bar, and see both your anaconda python install and Scala!
So far it’s great! I get most of the advantages of the worksheet, but now I can run and re-run individual cells rather than the whole file. You do end up trading out the IDE’s features for Ammonite REPL features, but as someone who’s not used to using tools like IDEA I don’t have very strong opinions on this.