In this paper we introduce our system “Kale” that enables Jupyter Notebooks to seamlessly interface with HPC workflows, leveraging distributed computational resources for iterative human-in-the-loop scientific exploration. We discuss the motivation for Jupyter in the context of data-intensive and high-performance computing, and how one can introduce interactivity in this space. Our system, Kale, enables Jupyter notebooks to communicate with backend jobs via a service that can be used to control, monitor and communicate with these tasks. We use iPyWidgets to introduce graphical controls for Kale tasks within the notebook, and enable seamless integration with existing workflow managers. Finally, we discuss a deep learning use case at NERSC where Kale can provide interactive feedback and control to the user.
