intel AMT - the good parts
A while ago, I decided I should make a real k8s cluster at home, so
got a few "HP EliteDesk 800 G2" mini PCs from a computer recycler.
They're pretty good - small, low power, and if you don't get the 85W
model, the power brick is quite small. I've been running Talos on
them, and set it up to PXEboot on each start, which is convenient.
That along with my noobiness with physical k8s also means I've ended
reinstalling them a lloooot, which is mostly fine - I flip a config
line in the TFTP config (uncomment APPEND talos.experimental.wipe=system in /srv/tftp/pxelinux.cfg/default)
and reboot and they reinstall.
Sometimes they get stuck though in the BIOS, though, I think if PXEboot is too slow the BIOS fails it, then finds the disks are indeed not bootable and waits at a prompt for what to do.
I dimly remembered there was a thing called "Intel AMT", and so had a look in to that - the the tldr is that you can just run (MeshCommander)[https://meshcommander.com] (a big ball of JavaScript) and get a KVM in to tell the BIOS to try again!
Setting it up wasn't so hard:
- reboot each machine in to setup mode and enable AMT over the network
- set a password that's complex enough for it to not complain
- put that password and the IPs in to MeshCommander
I did notice that the KVM functionality mostly only worked shortly after boot, though, which was annoying if I didn't catch the failure and login to fix it right away. It eventually it occurred to me it might be powering down the graphics chip-set or something, and that does seem to be the case, and as far as I can tell there's no way to power it up again without a reboot.
Fortunately, other people have this problem, so I could get some 'dummy DVI' (and Display Port) adaptors that electrically look like a display enough to keep the chip-set awake, and voila, it Just Works - a reliable-enough built in IP KVM.