Nix on macOS for isolated development environments

I’ve never used Nix before last week. I’ve head about NixOS and the Nix package manager before, but I thought those things were the same, or it was required to use the NixOS to be able to take the advantages of Nix package management. What attracted me about Nix was the idea of isolated environments and the idea of non-global package management. Or the rollback in case you install something that later you regret. This sounds a lot like Docker layers, but for the whole OS.

I use macOS for both, my work computer and my personal machine. I have been using Homebrew since the first time I used a Mac computer in 2011. I know about Macports, but never used it since people tell me it is not very straight forward, and I personally don’t see many benefits over Homebrew. Even thought, Homebrew has been essential to my development workflow on the macOS. I have been growing frustrated with Homebrew. The fact that I upgrade one package or install one tool for something specific and then my whole system installation starts to upgrade is not convenient in most of the time. An installation that would have taken 5 seconds becomes a system upgrade that takes 10 minutes or more. Another issue with that is when upgrades breaks workflow. I am working on something that needs GStreamer 1.18 in one project and GStreamer 1.20 in another. I don’t know if it’s even possible to have two versions installed like that with Homebrew, what I saw before were two packages with different names but meaning the same software in different versions.

Some people, me included, would see here an opportunity to use Docker or similar solution to create isolated development environments. That works great on Linux, since you will be sharing the same running kernel with the Docker container, but on macOS that is not the case. macOS cannot run Docker natively, everything you run inside Docker actually is running in a VM. So all the system resources are pre-allocated to the VM and separated from macOS. You will never be using the full-power of your computer in that case. Another layer of complications of using Docker for an isolated environment on macOS is when you are, like me, using the M1 chip. You will be needing to compile a lot of things from scratch inside Docker containers that would be native to arm64 in order to create base systems for your workflows. All that to say that I don’t think Docker on a macOS makes the best choice for isolated development environments.

Until recently, I was sticking with Homebrew to maintain my development workflow. But a few days ago, I discovered that Nix package management can be used on macOS. This seems to be the case for a long time, but for some reason I have never heard of anyone using Nix on macOS in my bubble of friends and acquaintances. I started reading about Nix and discovered the `nix-shell` tool. Not only, I could keep my system installation clean of packages I don’t need, but I could also install some packages in a temporary shell session and not affect my global package installations. This sounded great to me, as it would fix my frustrations with Homebrew and provide environment isolation that I thought I could only get with Docker.

There are some excellent guides on how to migrate from Homebrew to Nix”'), I think it is out of scope for me to go over again in detail on what you can do. I’ve uninstalled Homebrew from my personal machine and followed the guides to install Nix package manager.

After uninstalling Homebrew completely, I ran the following command:

sh <(curl -L https://nixos.org/nix/install)

Then I checked if everything was working by running the command:

nix-shell -p nix-info --run "nix-info -m"

After this part is working, I have installed the Nix Darwin modules as described in their repository README:

nix-build https://github.com/LnL7/nix-darwin/archive/master.tar.gz -A installer
./result/bin/darwin-installer  

At this point, you can edit the file ~/.nixpkgs/darwin-configuration.nix to customize your system. That is the file where you can also include the system-wide packages you want to have installed. Here is where the fun started for me. I like to use Neovim to edit configuration files, but at this point I lost Neovim when I uninstalled Homebrew. Now I can use nix-shell to create a shell where I have Neovim and use that to edit my global Nix packages configuration to include Neovim.

nix-shell -p neovim
neovim ~/.nixpkgs/darwin-configuration.nix

It worked perfectly. I included Neovim in the global system packages:

environment.systemPackages = [
    pkgs.neovim
  ];

Then ran the command to update my system:

darwin-rebuild switch

This way I have Neovim available globally.

As of today, I have many other packages installed globally in my system. This is an illustrative list:

environment.systemPackages = [
    pkgs.gitFull
    pkgs.neovim
    pkgs.wget
    pkgs.curlFull
    pkgs.python310
    pkgs.hstr
    pkgs.gnupg
    pkgs.htop
    pkgs.jq
    pkgs.mosh
    pkgs.ripgrep
    pkgs.sshuttle
    pkgs.ffmpeg
  ];

So far, so good. But I would like now to use Nix shells to create an isolated development environment for some projects that I work on. That was one of the main reasons I wanted to try Nix in the first place.

Using Nix to bootstrap per-project development environments

Looking at the nix-shell documentation, I can create a file shell.nix in the root directory of the project I’m working on and then call:

nix-shell —run zsh

The —run zsh argument here is just because I prefer to use zsh instead of bash. I know the must be a better way of doing that. But this works for now. This command starts a new zsh session with the configuration present in shell.nix. Here is an example of shell.nix file for the gst-plugins-rs project.

let
  pkgs = import <nixpkgs> { overlays = [ (import ~/.nixpkgs/overlays/a52dex.nix) ]; };
in
  pkgs.mkShell {
    buildInputs = [
      pkgs.gst_all_1.gstreamer
      pkgs.gst_all_1.gst-plugins-base
      pkgs.gst_all_1.gst-plugins-good
      pkgs.gst_all_1.gst-plugins-ugly
      pkgs.gst_all_1.gst-plugins-bad
      pkgs.gst_all_1.gst-devtools
      pkgs.darwin.apple_sdk.frameworks.Security
      pkgs.pkg-config
      pkgs.cairo
    ];
  }

This is all I need to have an isolated development environment to work on gst-plugins-rs project in my M1 macOS.

target/debug/libgstvideofx.dylib: Mach-O 64-bit dynamically linked shared library arm64

This is compiling everything to arm64, which means it is running natively. Yay!

Caveats and Workarounds

If you looked closely, the shell.nix file that I used here for the gst-plugins-rs project contained some “overlays” custom parameter in the imports. This is overriding one dependency of the pkgs.gst_all_1.gst-plugins-ugly package which could not be originally compiled to the arm64 architecture. Luckily, I have found a Pull Request that had a new package definition that fixed issues when compiling the a52dec to the M1 processor.

Conclusion

From now on, whenever I need to work in some project, I can write a shell.nix file with the system dependencies for that project and not worry that I am breaking some other project’s workflow.

Not everything is perfect, though. I had to spend considerable time to learn a bit of the Nix language, which I think is essential in if you want to consider Nix package manager. Anyway, for now I am happy with Nix and I will keep using it for the foreseeable future.

If you start using Nix on your macOS please consider donating to maintain the work on having Nix support for macOS. I’m not involved in that project, but I think it’s nice to support this work if you benefit from it and can afford it.


I'm open to hear from your experiences on things I write about. If you want to connect, you can find me at @rafaelcaricio@fosstodon.org or on other places on the internet. All posts from this site are also published to the Fediverse account @blog@caricio.com where you can receive ActivityPub updates about new posts.
The icons used by this site can be found at Hacker icons created by Freepik - Flaticon.