Jupyter Notebooks

(Andy Wootton) #1

I’ve seen a few comments about Jupyter ‘computational notebooks’. I’ve done a bit of reading and thought about writing a ‘What I Know Is’ mini-essay but I decided an open question might be more helpful, so: “what about Jupyter then; what’s that good for?”

(Marc Cooper) #2

I used Jupyter recently for exploring a bunch of AI stuff in python. It’s useful to be able to split a task into sub-tasks in Jupyter then only run the stuff you update – handy when there’s bunch of computation, for example. Embed pics is a bonus.

I’m no expert in either Jupyter nor python, but I found the environment easy to explore, learn, and use.

I use wolfram quite a bit, and I’d love them to implement some of Jupyter’s ideas.

(Marc Cooper) #3

I just reinstalled Sagemath and discovered that they now use Jupyter.

(Andy Wootton) #4

Thanks @auxbus.

I’ve been playing with a graph library in Clojure, so using a REPL from inside emacs. I’ve been writing quite a lot of notes to myself as comments. When I read about Jupyter, it seemed a similar idea but with better formatting and no hard line-breaks. Am I missing any other advantages? At the moment, the extra benefits don’t seem worse the faff. I seem to need Python, a web server and 1 of a selection of Clojure add-ons.

I’ve not found a clear description of the use-case envisaged by the developers. Everyone seems to assume its obvious. Is it a lab notebook for software experiments; a deliberate intertwingling of code, data and documentation?

(Andy Wootton) #5

Is the following correct?:

A notebook exists as a JSON file but is served up as a web service by a notebook server. The notebook server is written in Python so requires a Python environment.

A kernel runs the code. The default is the iPython kernel, clearly for Python. Other kernels can be added for other languages, which make a connection to a different environment.

(Marc Cooper) #6

It’s basically for taking notes with embedded computation. Hence, good for experimenting with AI stuff step by step. In the case of Sagemath, doing big sums and stuff. e.g. you can plot graphs (of a range of types).

re: installation: Yes, it’s python app; though that should be available on most machines (no idea about Windows). It runs its own webserver, so no added installs required.

One cool thing about Sagemath is that you can access the computation engine via LaTex:

\documentclass[a4paper, 12pt, oneside]{memoir}


Using Sage\TeX, you can use Sage to compute things and put them into \LaTeX{}

For example, there are $\sage{number_of_partitions(1269)}$ integer partitions 
of $1269$. You don't need to compute the number yourself, or even cut and 
paste it from somewhere. \bigskip

Here's some Sage code:

 f(x) = exp(x) * sin(2*x)

The second derivative of $f$ is

  \frac{\mathrm{d}^{2}}{\mathrm{d}x^{2}} \sage{f(x)} =
  \sage{diff(f, x, 2)(x)}.

Here's a plot of $f$ from $-1$ to $1$:

\sageplot[width=.95\textwidth]{plot(f, -1, 1)}


I presume it’s something like this you want but with clojure instead of maths.

(Marc Cooper) #7

That sound about right. The notebook (file) is a blob of JSON with images embedded, if you have any in the notebook.

There’s a list of kernels: https://github.com/jupyter/jupyter/wiki/Jupyter-kernels There are a few Clojure references.

The elixir kernel is pretty good :slight_smile:

(Andy Wootton) #8

I’ve just discovered the http://nbviewer.jupyter.org/ for reading other people’s notebook ‘source data’ without installing.

I’m wondering whether to install from the Ubuntu repositories rather than the recommended Anaconda that might tread all over my default Python install.

(Andy Wootton) #9

When you say “taking notes”, I think ‘while listening to a talk’, which is when I use Freemind, because I can quickly use structure to reduce typing, by showing relations between blocks of words, so I can get ideas down as quickly as people talk. Do you mean “making notes”, as in ‘writing an essay’, when you have more time to do the tagging to embed structure?

(Marc Cooper) #10

I use mindmapping tools in conversation and meetings too. I just found an archive of a project from 2006 with a Freemind file which opened in MindNode where I interviewed folk in a company in just that way. I’m surprised more folk don’t do it. It certainly beats pen and paper in so many ways.

I have quite a few tools and methods for note-taking. I’ve lost count :smiley:

Mindmaps; timelines; text with or without images and with or without remote real time processing – about half a dozen tools I use regularly e.g. Scrivener, TexPad, Quiver, Typora, Sublime, …; designs (e.g. Balsamiq); “scratch spaces” (e.g. Scapple, yEd); other: Sagemath, soulver, monodraw, Jupyter.

Basically, I use whatever’s necessary to capture whatever I find interesting and provides easy data exchange and, preferably, data input – macOS is incredibly powerful at the latter, and I’m surprised it isn’t talked about more.

My preferences for “writing an essay” is to “compile” to LaTeX, so I look for tools able to export in that format too.

(Andy Wootton) #11

I’ve worked out why my intuition has been telling me I ought to be investigate notebooks. I’ve been trying to computationally build documents out of component objects, as we would software. Jupyter notebooks are almost the inverse. They are a document structure state, containing component objects, including data describing computation. Sadly, they don’t appear to offer component re-use. I can see now that the JSON data could be change-tracked, which was another concern I had about them.

I’ve never understood why we used different tools to manage code and other text objects.

(Marc Cooper) #12

I showed above how Jupyter content can be “reused” in LaTeX, for example.

However, I don’t understand what you are after. It seems to me to be unreasonable to expect one tool to cater for all domains. The interchange protocol we have are, basically, mime-types and a loose collection of exchange protocols.

(Andy Wootton) #13

I wasn’t looking for a tool. I was being disturbed by something in the space I’m trying to make a solution in, doing something completely differently, that I didn’t understand. I had to make sure I wasn’t wasting my time duplicating something that appears to have good traction and was ‘good enough’ for my purposes.

I’m interested in it too, for what it’s good at. I like small pieces, loosely joined and data with embedded metadata too. Most of the tools I use have flat text files that I can keep the master of in the Cloud, so it appears to fit well into my mobile tool box.

I would like to have one sharp tool for each job though and to use them in customised tool-chains, to meet my needs. I need to see how Jupyter interfaces with REPLs. I shoved jupyter-notebook on from the Ubuntu repos but had an unresolved reference to python-ipywidgets which I added. It appears to start up but I ran out of play time.

(Marc Cooper) #14

This intro video is pretty good at going through the basics at a decent speed.

If you want something better, I guess it’s time to roll up your sleeves and write it.

(Andy Wootton) #15

Thanks, I think that’s the one I watched part of, but I dropped out when it started telling me how to install. It’s how I knew to type jupyter-notebook when I had installed, though I’ve been using leiningen so I missed the “-” out and thought everything had gone horribly wrong at 01:30 this morning.

(Marc Cooper) #16

JupyterLab might also interest you. This example contains two notebooks, an image, a markdown file (with bits of LaTeX) and its preview. You can add more kernels, of course.