Today I spent some time setting up NixOS on Windows 11 with Hyper-V and it turned out pretty good.
As I’ve worked over the years I’ve come to the conclusion that you really want a fairly Locked Down machine to work on connected to a powerful one. I recently built a rather beefy computer to replace my 6 year old Desktop and because it’s running an Intel 12900k I figured the best thing is just keeping Windows as the main host there.
An aside: I Really Didn’t Want to use Windows like this. I’ve LONG wanted to simply run NixOS (or even Proxmox) as the Host and occasionally use Windows. Maybe some day I’ll manage to get that working, but I’ve spent many full weekends trying to get it to work. I am simply too exhausted trying to force things to work.1 I really would rather spend that time relaxing playing games and watching shows with my girlfriend.
Anyways, back to how I spent my entire saturday morning: I wanted a NixOS VM with a bunch of CPU (20 cores) and RAM (16GB/64GB; so far) to act as a dedicated builder for WSL and my M1 MBA. I am probably going to setup another NixOS VM as my actual “remote” NixOS desktop and this one is simply going to stay a builder for any of my lighter machines.
To be honest I think the M1 is pretty fast already, but it doesn’t hurt to run this on the beefy desktop instead.
Setting all this up was pretty straight forward, but I’m gonna skip some steps to help me get this post finished2.
Configuring the Virtual Machine
First, you’ll need to enable HyperV. If you’ve enabled WSL you’ll have some things enabled, but you need to enable everything under the Windows Features selection window. Namely this will give us the tool which lets us setup, configure, and manage our machines.
HyperV is designed primarily for running Windows so it requires some effort to get things running.
The main thing being that after you setup the machine to go back to reconfigure it to disable Secure Boot. You’re already booted so this isn’t the end of the world. I do think (from vague memory) you can set your own Secure Boot keys, but maybe not.
The next thing I would suggest is Dynamic Memory. NixOS has the
kernel support for this already, but some distros do not. A lot
of HyperV’s features only work in particular versions of
mainstream distros so… you’ll need to be prepared
to read some sys and dmesg logs if you want to run a VM with any
fancy features. I will note that Dynamic Memory in my experience
reports extremely strange usage in
free because for some reason the system managing
the dynamic memory doesn’t tell the kernel that this
memory isn’t being used anymore. I ended up with processes
using up a total of like 300MB but
80-90% actual utilization (NOT caching). I don’t
have a solution for this, but if you look in your sys logs
you’ll see the system responsible for managing the dynamic
memory is running so… I guess it’s okay. This is
just apparently a known issue on Hyper-V for Linux.
tl;dr: You might want to not use Dynamic Memory, but for NixOS it’ll be… fine probably (I am using it anyways).
Wrt. to installing NixOS it’s pretty straight forward as any install with NixOS is. I’ve thought that the best way to configure NixOS is actually to just get it installed with the right partitions, file system (I used ext4 for this one), and users configured (+ssh keys).
Then I build the system in WSL or on a remote server and deploy it to the VM using deploy-rs which allows specifying remote servers to deploy NixOS configurations to in a flake.
I really love using
deploy-rs for both this as well
as configuring temporary machines in AWS with my exact
configuration using simply the latest ami built by NixOS’s
The one caveat is that configuring NixOS initially uses Channels, the legacy system, and you’ll need to enable experimental features when you install NixOS as well as upgrade your version of Nix to 2.4 (which isn’t the default yet) so you can properly deploy your changes to the server.
Another recommendation is ssh’ing in to this server once
it’s rebooted and setup a git bare repo which you can push
the configuration you used to deploy via
deploy-rs so you can rebuild it inside that machine
instead. It’s a lot faster when all you want to do is add
an SSH key, change some nix configuration, or etc. versus doing
a whole big deploy using
I also actually looked up the state of the art. Nvidia now supports doing a single Pass Through to Windows with their drivers (AMAZING! Not sarcastic), but now it becomes this ordeal of running the GPU in weird states and you need two GPUs to make everything work right. The Integrated GPU or my old 980 would probably work, but like I said I’m exhausted. What I really want is to setup the Looking Glass stuff, but there is a bunch of Audio problems there. I mostly wanted Windows for games. I hear Linux is Getting Better thanks to Valve+Steam Deck, and I’m far more comfortable doing stuff that Linus was, but the REAL issue is getting Windows to Properly work with the GPU passthrough. ↩︎
An exercise for the reader… ↩︎