I just felt like we never got to the heart of the matter of why the models work and how to approach developing them, it was all about learning a bag of tricks.
Meanwhile, math and CS being a lot more axiomatic by nature, they also made a lot more sense to me.
That being said, that specificity of physics, the unbridgeable gap between reality and the models we build to describe it, in retrospect, is what makes it more interesting to me today (it's not just a "closed" system in the sense that math is — of course the relationship between math and physics is itself fascinating but that's yet another topic), but I still feel like I haven't found the right pedagogical approach to make it fit my mindset.
Maths (and especially compsci!) are constructions by and for humans.
Is it any wonder it is as you describe? It would be odd if it was any other way.
Math and CS are mostly human-made, so most of the theorems/proofs/axioms are either straightforward or elegant—there are infinitely many possible axioms with no objective way to choose between them, so people generally choose to work with the ones that are the easiest for humans to reason about. You certainly could define a complicated set of axioms with dozens of special exceptions, but unless there are some external reasons why these axioms are important, nobody will want to work with them.
Conversely, physics exists to model the real world, so unlike math and CS, physics doesn't have the privilege of being able to choose the most convenient/elegant/simplest axioms to work with. Given the constraints of the real-world data, physicists will still choose the most elegant possible model, but sometimes a wacky model is the only way to accurately model the universe.
Nobody is really happy about this though, so physics textbook authors love to make their equations/derivations look simple/obvious/elegant, but this is often completely misleading, since often the rules of the universe are so weird that nobody would ever guess them without having ran the experiments first. But textbooks tend to downplay actual experiments in favour of post-hoc explanations, which tend to make the readers think that they're missing something.
> Physics is an endless source of frustration to me. It feels like a mix of random tricks, most of which I don’t understand.
Your feelings are correct, since physics really is mostly a set of random rules that nobody truly understands. But the important thing is that these random rules correctly model nearly everything in the universe to within a few hundredths of a percent, so they're not completely arbitrary.
> Are there good resources to learn it?
The annoying/inconvenient answer is to do lots of lab work. This is actually fairly accessible though, since a measuring tape, a scale, and a slow motion camera (present on any modern phone) is all that you need for most kinematics/mechanics experiments, and a multimeter, a 9V battery, some resistors, and some magnets are enough for most electromagnetics experiments.
Not sure if it'll help you with gaining an intuitive understanding, but at least it'll be interesting!
General advice take a look at the references in works you've recently read and look for lower level topics that interest you, after repeating a few times you'll find your way to physics or chemistry and you can use the above as reference works. The best resource is the one you actually use. If https://www.youtube.com/learning works better for you then use it.
The standard text to build understanding in physics is University Physics by Sears & Zemansky.
It's worth remembering you're quite far from the ground in physics, and it's mostly taught with "neat" cases that give insight into physics. I.e. the thought experiment to show why kinetic energy must scale quadratically with velocity is carefully designed to show that conclusion. You shouldn't expect to have come up with it off the cuff.