profile
viewpoint

Ask questions/nix will not be writable on macOS Catalina

This is not a short term bug, but it will become an issue when macOS Catalina is released this fall. macOS is now split across two volumes (system and data) with a read-only system volume. This means that /nix will no longer be writable.

Some more information can be found in the related WWDC talk and some session notes people took from a Q&A.

Summary: the system volume, which is mounted at / will become non-writable. Some directories that need to be writable are connected via firmlinks (an Apple invention) to the data volume. /nix is not among these locations, so with the release of macOS Catalina, this location is no longer an option.

I see two possible solutions:

  1. I could try and file a bug to convince Apple to pre-install /nix as a firmlink to a writable location. I think this has limited success and Nix would then depend on Apple to not drop this link in a future release.
  2. We could move Nix on macOS to a different default location. Possible locations are those that Apple chooses to pre-install as links to writable locations. Two examples are /usr/local and /opt, so we could move to /usr/local/nix or /opt/nix. I would hope that these locations are common enough so that Apple would not drop them in the future.

I wanted to raise this issue early, before it becomes a problem for users. If this issue tracker is not the right place, please feel free to move this discussion elsewhere. I would also be available for testing any potential solution, since I have access to a macOS Catalina beta.

NixOS/nix

Answer questions burke

If it helps anyone else, here's my plan of attack for our tooling to get a writable /nix. Code samples are all lazy, approximate, and simplified. The met?/meet paradigm drives most of our tooling; meet is only evaluated if met? is false, then met? is asserted to be true afterward.

1. Configure synthetic.conf(5)

The file /etc/synthetic.conf is used on macOS Catalina to create empty, immutable directories as direct children of / that can be used as mount points.

We can likely write this out right away on Mojave and have it get picked up in the Catalina upgrade, saving the reboot to activate the config.

met? { File.read('/etc/synthetic.conf').contains?('nix') }
meet { File.write('/etc/synthetic.conf', "nix\n") }

2. Reboot

The only way to activate /etc/synthetic.conf is by rebooting, which is why it's ideal to write out on Mojave before the Catalina upgrade.

met? { Dir.exist?('/nix') }
meet { system('reboot') }

3. Create a volume

We'll want to query diskutil(8) info for /nix and cache it, since it takes 180ms to generate:

def nix_disk_info
  @nix_disk_info ||= Hash[`diskutil info /nix`.scan(/\s+(.*?):\s+(.*?)$/m)]
end
def clear_ndi
  @nix_disk_info = nil
end
met? { nix_disk_info["Volume Name"] == "Nix" }
meet { system('diskutil apfs addVolume disk1 APFSX Nix -mountpoint /nix'); clear_ndi }

4. Enable ownership

Nix definitely requires full UNIX permissions:

met? { nix_disk_info["Owners"] == "Enabled" }
meet { system('diskutil enableOwnership /nix'); clear_ndi }

5. Give the user ownership

We're not allowed to change ownership of .Spotlight-V100, .Trashes, and .fseventsd, and should silence those messages when we:

met? { File.stat('/nix').uid == Process.uid }
meet { system("chown -R #{Process.uid} /nix") }

6. Amend /etc/fstab

In order for /nix to mount on boot:

met? { File.read('/etc/fstab').contains?('LABEL=Nix') }
meet { File.open('/etc/fstab', 'a') { |f| f.puts ('LABEL=Nix /nix apfs rw') } }

7. Enable FileVault

I haven't bothered to figure out yet whether we can just pass it on stdin or what but we need to generate a password and provide it to diskutil (twice) here. Maybe just SecureRandom.hex(32) to generate.

met? { nix_disk_info['FileVault'] == 'Yes' }
meet { system('diskutil apfs enableFileVault /nix -user disk -stdinpassphrase', stdin_data: passphrase) ; clear_ndi }

8. Save the password in the Keychain

If we don't do this step, the user will be prompted for a password when they reboot.

The state we're trying to replicate is:

Screen Shot 2019-09-26 at 16 25 48 Screen Shot 2019-09-26 at 16 25 51

met? do
  uuid = nix_disk_info['Volume UUID']
  `security find-generic-password -l Nix -a "#{uuid}" >/dev/null 2>&1`
  $!.success?
end
meet do
  system(<<~SH)
    security add-generic-password \
      -l "Nix" \
      -a "#{uuid}" \
      -s "#{uuid}" \
      -D "Encrypted Volume Password" \
      -w "#{generated_password}" \
      -T "/System/Library/CoreServices/APFSUserAgent" \
      -T "/System/Library/CoreServices/CSUserAgent"
  SH
end

I'm unclear on why the one created by macOS's modal has two copies of APFSUserAgent; I was only able to find one path to the binary on my machine. The user will also be prompted on reboot to allow APFSUserAgent permission to this key, which seems fine because I seem to recall that granting this from the command line is c̣͔͑͆om͊p͍̳̠̂͋͂l̩͔̤et̰̻͛̈́e͍͓̬ͯ̇̋ m̦̓adn̂es̯̪͚s.

EDIT: It works

useful!

Related questions

Nix daemon fails to start on macOS 10.15 Catalina hot 1
Stream error in the HTTP/2 framing layer hot 1
locked from update - nix hot 1
Nix single user install on Linux wants 'nixbld' group and workers, else - it fails hot 1
Error: cloning builder process: Operation not permitted installing Nix 2.2.1 in (Arch) Linux hot 1
Single-user install should not use /run/user/<pid> for build directories. hot 1
nix-channel segfault: Fatal error: glibc detected an invalid stdio handle hot 1
nix-channel segfault: Fatal error: glibc detected an invalid stdio handle hot 1
ARMv7 Build Failure: undefined reference to __atomic_fetch_add_8 - nix hot 1
source:https://uonfu.com/
Github User Rank List