undefined

points

[-]

There is no such thing as 'alignment'. I want the model to be aligned with my intent, which can be whatever I want it to be. Your intent is not a more real or objective intent than mine.

More specifically, alignment is a reference; 'alignment with X'. A reference necessitates a referent.

You can't reify 'alignment', it is not an intrinsic property of a model.

by pixl974 hours ago|

parent|

[-]

There is such thing as alignment, the first 18 years of most people's lives in the modern world are about learning it. If you act too far outside of it we put you in a cage or elect you.

You can refine alignment with a set of stated goals. The harder parts are aligning words with human social behaviors.

by fwip4 hours ago|

prev|

[-]

Connecting the optimizing machine to real world actions without human oversight is not a neutral or blameless choice. The LLM takes no actions, the harness it is connected to does. And the harness does so only because a human did that deliberately.

It is not entirely dissimilar to the goldfish that "played Pokemon" by swimming around the tank. The program that looked at the camera sent the instructions to the gameboy emulator, because a human being decided to connect them.

by dualvariable3 hours ago|

parent|

[-]

> Connecting the optimizing machine to real world actions without human oversight is not a neutral or blameless choice.

Yep.

The fact that they lack hard constraints (unlike an SQP optimizer) and all this talk of "alignment" is also just a feature (or misfeature) of the algorithm.

The way that people/researchers choose to use those algorithms, even in the presence of those limitations, doesn't mean the algorithm is "rogue"; it means the humans didn't build in enough external hard constraints.

It is like passive voice, blaming the algorithm instead of the researchers running it.

by pixl974 hours ago|

parent|

prev|

[-]

The harness and the model are a system with emergent properties.

The particular issue that you may not realize is under your model every prompt to an LLM in a harness is dangerous. The prompt is not predictable. "What is 1+1" could lead to the heat death of the universe as the paperclip maximizer tries to prove countless axioms.