Setup NixOS on Windows 11 with HyperV

Today I spent some time setting up NixOS on Windows 11 with Hyper-V and it turned out pretty good.

As I’ve worked over the years I’ve come to the conclusion that you really want a fairly Locked Down machine to work on connected to a powerful one. I recently built a rather beefy computer to replace my 6 year old Desktop and because it’s running an Intel 12900k I figured the best thing is just keeping Windows as the main host there.

An aside: I Really Didn’t Want to use Windows like this. I’ve LONG wanted to simply run NixOS (or even Proxmox) as the Host and occasionally use Windows. Maybe some day I’ll manage to get that working, but I’ve spent many full weekends trying to get it to work. I am simply too exhausted trying to force things to work.¹ I really would rather spend that time relaxing playing games and watching shows with my girlfriend.

Anyways, back to how I spent my entire saturday morning: I wanted a NixOS VM with a bunch of CPU (20 cores) and RAM (16GB/64GB; so far) to act as a dedicated builder for WSL and my M1 MBA. I am probably going to setup another NixOS VM as my actual “remote” NixOS desktop and this one is simply going to stay a builder for any of my lighter machines.

To be honest I think the M1 is pretty fast already, but it doesn’t hurt to run this on the beefy desktop instead.

Setting all this up was pretty straight forward, but I’m gonna skip some steps to help me get this post finished².

Configuring the Virtual Machine Link to heading

First, you’ll need to enable HyperV. If you’ve enabled WSL you’ll have some things enabled, but you need to enable everything under the Windows Features selection window. Namely this will give us the tool which lets us setup, configure, and manage our machines.

HyperV is designed primarily for running Windows so it requires some effort to get things running.

The main thing being that after you setup the machine to go back to reconfigure it to disable Secure Boot. You’re already booted so this isn’t the end of the world. I do think (from vague memory) you can set your own Secure Boot keys, but maybe not.

The next thing I would suggest is Dynamic Memory. NixOS has the kernel support for this already, but some distros do not. A lot of HyperV’s features only work in particular versions of mainstream distros so… you’ll need to be prepared to read some sys and dmesg logs if you want to run a VM with any fancy features. I will note that Dynamic Memory in my experience reports extremely strange usage in htop and free because for some reason the system managing the dynamic memory doesn’t tell the kernel that this memory isn’t being used anymore. I ended up with processes using up a total of like 300MB but free reporting 80-90% actual utilization (NOT caching). I don’t have a solution for this, but if you look in your sys logs you’ll see the system responsible for managing the dynamic memory is running so… I guess it’s okay. This is just apparently a known issue on Hyper-V for Linux.

tl;dr: You might want to not use Dynamic Memory, but for NixOS it’ll be… fine probably (I am using it anyways).

Wrt. to installing NixOS it’s pretty straight forward as any install with NixOS is. I’ve thought that the best way to configure NixOS is actually to just get it installed with the right partitions, file system (I used ext4 for this one), and users configured (+ssh keys).

Then I build the system in WSL or on a remote server and deploy it to the VM using deploy-rs which allows specifying remote servers to deploy NixOS configurations to in a flake.

I really love using deploy-rs for both this as well as configuring temporary machines in AWS with my exact configuration using simply the latest ami built by NixOS’s CI pipelines.

The one caveat is that configuring NixOS initially uses Channels, the legacy system, and you’ll need to enable experimental features when you install NixOS as well as upgrade your version of Nix to 2.4 (which isn’t the default yet) so you can properly deploy your changes to the server.

Another recommendation is ssh’ing in to this server once it’s rebooted and setup a git bare repo which you can push the configuration you used to deploy via deploy-rs so you can rebuild it inside that machine instead. It’s a lot faster when all you want to do is add an SSH key, change some nix configuration, or etc. versus doing a whole big deploy using deploy-rs.