Gateways 2018 has ended
Wednesday, September 26 • 2:50pm - 3:10pm
Kale: A System for Enabling Human-in-the-loop Interactivity in HPC Workflows

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

In this paper we introduce our system “Kale” that enables Jupyter Notebooks to seamlessly interface with HPC workflows, leveraging distributed computational resources for iterative human-in-the-loop scientific exploration. We discuss the motivation for Jupyter in the context of data-intensive and high-performance computing, and how one can introduce interactivity in this space. Our system, Kale, enables Jupyter notebooks to communicate with backend jobs via a service that can be used to control, monitor and communicate with these tasks. We use iPyWidgets to introduce graphical controls for Kale tasks within the notebook, and enable seamless integration with existing workflow managers. Finally, we discuss a deep learning use case at NERSC where Kale can provide interactive feedback and control to the user.

Wednesday September 26, 2018 2:50pm - 3:10pm CDT
Lil Tex Auditorium, Commons Conference Center 10100 Burnet Road, Bldg 137, Austin, TX 78758