It's the transportation "mode shifting" difficulty. Per the AI, the term of art is "Pure Transfer Penalty". It's the "ick" when doing bike => bus => bike instead of "only bike" or "only car".

Mode switching has a cost. Usually std::sort is good enough compared to picking the prime optimal algorithm for your expected shape. Just call the function and get on with your day.

I think both your arguments are true. It all depends on the velocity of the capability growth and the fact that opportunity cost is expensive.

Once we get out of this hypergriwth phase the very same AI companies that now are giving you llms will provide a service that employed a rich mixture of optimized models that will reduce the operational costs to achieve the required results

I'm a little confused: LLMs were invented in 2018.

Is the middle option asking LLM to generate a classic ML model? Or generate tons of them and pick the best?