>There is a part near the beginning where he derives the diatonic scale from first principles.

I skimmed through the whole article and only found some brief mention of Helmholtz, which no clear route from his research to the scale.

As a better derivation, I recommend this paper by William Sethares:

https://sethares.engr.wisc.edu/paperspdf/consonance.pdf

Or the informal summary version:

https://sethares.engr.wisc.edu/consemi.html

But note that Sethares's work assumes the Plomp and Levelt curve is correct, which is only approximately true. In practice there is also "higher order beating", which causes additional dissonance at ratios close to small integer ratios. This means the most accurate model of dissonance combines Sethares's approach with ratio complexity as in just intonation. This is backed by listening tests:

https://www.nature.com/articles/s41467-024-45812-z

The best informal overview and demonstration for this concept I know of is this Youtube video:

https://www.youtube.com/watch?v=iqxfE2nfGhY

Note also that there's a strong cultural/training component to dissonance perception. To minimize this you should ignore the preferences of musicians and only ask untrained listeners.