Livebook Tweaks

July 1, 2023 · 4 min read

I've learned a few things using Livebook that took a minute to uncover, so I wanted a place to keep it all together. I'll list them as a table of contents and explain further in each section, with links to my stumbling through issues and PRs.

  1. Version management: if you use a version manager like asdf or rtx, the Livebook version will be bound to your global or local instance, depending on the directory.
  2. MacOS app bundles Elixir: the Livebook desktop app is a great case study for bundling Erlang and Elixir.
  3. Reopening Livebook: if you close the Livebook window, you can open a new one by clicking on the Menu Bar icon and selecting Open.
  4. Menu Bar quirks: when running applications like Vanilla or Bartender, the icon only shows up if I restart Vanilla.
  5. Remote file management: the docker image defaults the working directory to /data, so if you use relative pathing in your notebooks or apps for storing files, you either have to infer this or change the WORKDIR in Docker.

Version Management

I spent maybe 20-30 minutes installing Livebook 0.9.3 and running it for my collection of notebooks. I would execute the command livebook server index.livemd in the directory, which ran version 1.14.4 at the time of discovery. When I upgraded Livebook, I was outside this directory which took in the global version 1.14.5. Because rtx or asdf completely isolates versions, I wasn't aware it was executing the 0.9.2 build with 1.14.4. Changing .tool-versions by using rtx use elixir@1.14.5-otp-25 brought my local environment up to global, and the Livebook version was what I expected.

I suspect this may bite me and others consistently if we're not careful. I'm sure I've had similar issues in the past that I did not equate to this at the time.

Desktop Elixir Bundler

A ton has changed since January 2022, but I opened this issue, which spurred a subsequent pull request. This work has slowly morphed into what is being called "ElixirKit" and is in the elixirkit/ directory in the repository. ElixirKit has a different surface area than Elixir Desktop, which is somewhat of a wrapper or orchestrator for wxWidgets in Elixir.

Reopening Livebook

In the past, when this was a Dock icon only, if you closed the browser window, it was less intuitive for me to open it again. I could reopen the window by clicking the Dock icon (see this issue for history). Now the app stays resident in the Menu Bar, and we can reopen windows or view logs to see the hidden console window. The view logs option is perfect for getting the auth token again. The dock also had issues fully closing the runtime but was ironed out <3.

I use the free version of the Vanilla app. If Livebook is on the always-visible right side, the icon will display whenever I close and reopen the desktop app. If it's on the default left side, I must quit Vanilla for the icon to appear. The long-term fix of keeping it visible is enough for me, but it may bite other people that use similar menu bar applications.

This issue spells out what I saw and may be helpful. I kept trying to run the app again, not knowing that it was there in the menu bar but I just couldn't see it.

Remote File Management

I had issues understanding how to work with notebooks or public apps on Huggingface. When I used something like data = "./data/file-sorted.csv" where the notebook-relative data/ directory existed, I would get problems like ** (File.Error) could not open "./data/file-sorted.csv": no such file or directory. My fix at the time was for apps to use something like data = System.get_env("PWD") to get the current working directory. This change posed a problem locally because my relative data/ directory is in .gitignore. Using PWD would save the data in the current directory, which would not fall under .gitignore rules.

The long-term fix, which you can see here, was to include WORKDIR "/notebooks" at the end of the file. WORKDIR tells Docker the working directory so that /data is now the place for only Livebook configuration, and any relative paths will work as expected.

There are other parts to this Dockerfile, such as pulling down my notebooks and the ash_tutorial so that I could work on Huggingface for things like Whisper or other ML models. Adding git or FFmpeg as dependencies is trivial. I explicitly copy the public-apps directory from my repository so that the Huggingface repo stays pure and only cares about setting up the environment.

One of my goals is to flesh out a system where I can work on notebooks remotely and periodically synchronize changes up to the repository. That's why I've included git-sync but I haven't worked out how to leverage it. A public or private app could leverage Kino.start_child/1 to start a GenServer that watches the filesystem for changes and presents a UI to commit and push changes. I believe something like egit could do this, I would need to create the UI for it. I'd certainly take this approach over shelling out to the git CLI though that's not extraordinarily difficult either.