Now that I soldered the transistors in as close as I could to the board, most of the noise is gone. I previously had them in sockets and the legs must have been picking things up.
Further reading here, the development blog thread:
https://bjornjuhl.com/forum/viewtopic.php-f=4&t=5456.html