theres actually a very important reason you want it to be an actual embedded dsl or tiny programming language!
The reason why llms can code at all is the hugeeeee amount of RL based on the loop of 1 "write code", 2 get compile time or runtime errors,3 fix it and iterate. Data file formats dont have that feedback loop so models will fall off the rails faster. Writing code that fits a latent adhoc schema just wont work as well, or will require burning a lot more context.
from that perspective, it could just be an EDSL little library in the host language, or it could be a friggin little custom language with an interpreter and good error messages.
> dct validate <file.yml>
Or any of the other commands that read the file will quickly fail with any syntax or SQL issues. It’s not just an open schema.
That lack of flexibility is actually a feature for one of the bigger problems we're solving: lineage, or knowing where a number came from. If the chart layer can't transform data, the logic stays in SQL, where you can version, test and audit it.
It's also a choice about who this is for. The data community already works in SQL, Jinja and YAML every day, so there's no new language to learn.