upvote
> But I won't follow you as far as "expecting developers to have expertise in how and where their software runs is unreasonable".

In my professional experience, management tends to treat this as a dichotomy rather than a range. And its lead to situations where we're choosing to route every decision through a small number of 'experts' and expecting that to scale, or have people make very bad decisions about things like indices, data layout, etc.

I align with you on this. If you are using some sort of system, there is a sort-of minimum bar of expertise one should acquire. When using analytical platforms, having a basic understanding of how data is stored and the levers it gives you to optimize data processing should be standard.

Some examples:

MySQL - I would expect developers to be able to come up with a list of columns to index and why, understand why UUIDv4 is a bad choice for primary keys, know how to write performant SQL by hand and understand when their ORM is going to do the wrong thing, etc.

Redshift - Understand dist and sort keys and have some mental models on how data shuffles and is pruned based on each choice.

Iceberg + Spark/Trino - Understand that data tends to be stored/loaded as batch files. Understand how deletes work. Understand partitions and sort as options. Understand generally how bloom filters work and when you should decide to place a bloom filter. Understand why UUIDv7 is better than UUIDv4 when its part of a partition key.

reply
Apologies my comment was very tongue and cheek. I understand what you're getting at with regards to dedicated DB style teams being a potential bottleneck and not effective for every organization, as for example most orgs get by fine with a simple RDS instance with minimal management and optimization.

What I'm calling out practically is a de-specialization in roles and the trade-offs it imposes.

For example, the "jack of all trades master of one" that was already propagating under the guise of "Full stack engineer" or "DevOps" is further exasperated when those consolidated roles are also delegating critical knowledge acquisition to LLMs.

Further, as we consolidate roles and delegate knowledge acquisition to llms, we're also paying people less, giving less job security, lowering the general quality of life not just for developers, but for operations staff and users, and generally promoting total consolidation within big tech.

reply
What's worked really well for us is not having the DBAs as gate keepers, but rather having them as guardians and experts for devs to rely on.

Our DBAs are the first line of defense when a DB starts to struggle. They crack the database open, see what's going on, and troubleshoot and sometimes fix the problems. If it's just a simple stored proc change or table change, our DBAs are empowered to go in and fix them. If there's a code change needed, they work with the devs to fix the code.

But by and large the devs still write most of the queries and tables. We as devs get DB training, but we also have a very open door policy with the DBAs so the devs can lean on their expertise and ask advice before sending stuff out. Our devs get a much better sense of "This seems like it could be expensive, let's talk to the DBAs and see what they'd suggest".

It's worked great.

I think any company would benefit from this sort of subject matter expert employment.

reply
No apology needed; I understand what you're getting at, and I broadly agree. It's a spectrum between "extremely easy-to-hire people that operate in such a narrow niche that they're an operational liability with limited capabilities" and "expect everyone to be an expert at every level of the stack". The right point on that spectrum is different depending on context, but I do think that a majority of software shops would be better served by moving their required skillset more towards the generalist end of that spectrum, because the default is often far too niche (driven by poor tradeoffs and short-termism in service of growth/hiring, usually).

I wanna re-emphasize that this is not a new problem. It's not because of DevOps culture or cloud complexity or scale or whatever. Very limited-specialty people were always operational liabilities and had limited positive impact on feature delivery once you accounted for the help they needed to do anything that spanned multiple levels of the stack. There are just more engineers working on more systems with tighter timeline expectations now, so it seems like the complexity incumbent on the engineering role went up in general. It didn't (it went up in some situations and down in some situations), we just started noticing operational pain more often.

I definitely do agree that there's widespread ignorance of the velocity and difficulty-of-work tradeoffs that arise from requiring a wider range of specialties from engineers, and a similarly widespread failure to adjust compensation and timeline expectations accordingly.

reply
I usually see full-stack and DevOps roles as being a creation of the startup and early growth phases of a company. In those phases, the market and product definition is changing rapidly. Whatever you lose in reduced technical depth of these engineers, you gain in reduced communication costs, faster ability to make changes, and additional context and ability to spot efficiencies because they understand the full stack. And these companies are usually default-dead anyway, so if they screw things up and cause an outage, it's no worse than if they build the wrong product and have no users.

I've never seen a DevOps role in Big Tech (I've ping-ponged back and forth between the startup and FANG worlds a few times), and usually a full-stack engineer is someone very senior who designed a lot of the product when it was young and stuck with it for a decade. Instead there's usually a clear SRE/SWE split, and hyper-specialization, enough that someone might actually become the expert on one particular screen of a mobile app.

If you're frustrated with the de-specialization of roles, it's entirely possible you're working at companies at the wrong stage of their lifecycle. Move to a big company and the struggle is to avoid becoming pigeon-holed into one very specific task (and then laid off when the structure of the product changes so that task is no longer necessary). Such a career change can be challenging at first because your career can get specialized to "working for startups and early growth companies", and then big companies won't even look at your resume. But if you try enough places and say in the interview that you're looking for a place where you can really specialize and develop expertise, you can often find a manager desperate enough to take a chance.

LLMs are another issue. I'm worried that they're going to make the economy very unstable because the whole reason a firm exists is to capture institutional knowledge and the LLM captures that institutional knowledge in the model, where a.) it's not specific to a differentiated business and b.) it's subject to model-collapse issues. Could easily see the whole technological world collapse through lack of expertise, but if that happens it's a bigger problem that you can only fix by developing the expertise yourself.

reply
> Such a career change can be challenging at first because your career can get specialized to "working for startups and early growth companies", and then big companies won't even look at your resume.

How would you suggest to approach such a career change successfully?

reply