The more you know about what you want to do ahead of time, the more optimally you can reorder your sequence of steps to give a better-than-naive solution. It makes me think about all software in terms of these abstract computation graphs and makes me wonder what else we can optimize automatically.
Of course, we do absolutely do need a formal model here, so we know what edits are possible, the same way db engines have relational algebra as their backing model. But this whole thing makes me feel like manual software optimization is soon to fall to AI. And I'm thinking that includes good-old-fashion AI first, not second, to LLM's. But I'm sure LLMs would be useful here too, especially for the formalization.
I've been waiting for a literal decade for this, for the same reasons as the grand-parent poster. I literally had a chapter of my NSF CAREER proposal on this (failed, woof, but the reviewers were wrong! this rocks). The potential here is absurd. Eg, novel query optimizers and novel DB indexes could be created that, when connected to the right charting tools, would automatically emit efficient graphics and query results. Very, very cool work, thank you.
I think good-old fashioned AI is the key here! Lot of the Lean proofs are discharged by a proof-search procedure called "grind". And if "grind" fails, an LLM can read the logs of the failed proof-search and figure out if there are any missing theorems needed.
I'm the same user name @gmail.com if you ever have any questions. It's been a while but I'd be happy to try to page things back in for a good cause.
We will probably see more such things as the consequences of the end of the free performance lunch play out. Hardware and software will specialize more, plenty of interesting work to do.
serialization/fercode.py, codegen.py and argumentscgreader_t.cpp. It's in Python for easier cross builds, so that there is no need to build the tooling for the host platform.
I had the idea for this project years ago while writing Web Browser Engineering with Chris Harrelson (see https://browser.engineering/). Then a few years ago I made a first attempt at this project with Yuvaraj (https://droidkid.github.io/), but for various reasons we never got very far. I restarted the project with Bhargav (https://bhargavkk.com/) about a year ago, and focused much more seriously on the semantics of Skia itself, which made progress much more rapid. Still, I was, frankly, shocked by how good the results are.
(The library is called goldy, and until I spend some time on it the readme and docs are sadly LLM generated)
Goldy has the high level structure of the shader graph and exchanges with the outside world (host memory, surfaces, etc) which is turned into a backend specific graph IR (different backends have different rules for command list retention and other properties). But my library also embeds Slang and uses Slang IR to inject and introspect on the user written shaders. I can then at runtime turn it into CUDA graphs (slang can turn any shader into a CUDA kernel) but with the added benefit that the runtime can manipulate the kernels and graph as more information is available. For example if a host upload that usually is scheduled between kernels isn't done on a specific graph submission, the graph partitioner can fuse kernels and remove a fence.
I plan to take advantage of the compiler and runtime introspection to implement some interesting features like shader coroutines without sacrificing GPU residency.