Trade-off: datafusion allows you to do fine grained storage integration (spill to disk as a part of the algorithm).
The icebug/ladybug way is coarse grained. But it allows you to run cypher instead of writing datafusion operators.
~10 years ago, we helped create apache arrow, helped create GPU data frames, and been running for the last decade the open source pygraphistry and now gfql cpu+gpu property graph engine for this. Likewise, Nvidia has been doing great with cuGraph (OSS) for GPU algs around this.
It is great you are finding success with this direction, but "shoulders of giants" merit credit - 100+ people.
I didn't write the graph algorithms in networkit. Somebody else did.
But if you're looking for 100-200 graph algorithms running efficiently on columnar memory, I haven't found a working implementation that's permissively open source.
Happy to work with cugraph or anyone else who builds on top of Parquet/Arrow ecosystem.
https://arrow.apache.org/docs/r/authors.html
CUDA and the ecosystem around it is more complicated.
I'll use it. I agree that it advanced the state of the art and helped fund some of the truly OSS projects.
Don't feel the need to bring it up on a HN comment. Neither does the parent article by Sem.
In fact, it's in the vendor's interest to transparently route the algorithms in Icebug to the GPU in a compatible way instead of writing their own APIs.