The participant is not an LLM, the participant is a company. The company learns. It adapts. It has a trillion dollars of skin in the game.
Likewise OpenAI doesn't have "skin" of any shape or value in e.g. the mathematics community or even in mathematics research per se. In my cynical view, they are exploiting the exposure of mathematical research to automation in an effort to create a case study of their product obsoleting a highly visible and well-leveraged and relatively small profession. Regardless, I doubt even they would say they have durable skin in mathematics in any sense, certainly not in either of the senses I mentioned above. But their output inhabits the same space as mathematicians, who have to contend with the consequences
They hire math and physics grads and good command of math is very useful in computer science, machine learning and AI.
And how do you see these qualities changing over, say, the next decade?
And potentially steals [1].
[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_priority...
I'm interested in the reversal that Sellars proposes - it seems clear to me that "representation" is not prior to the space of reasons, despite how unintuitive it is.
> An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time.
I see no reason why this can't be the case in the future, even though I think inferentialist-based theories are generally strong. Can't you force LLM's (or whatever AI architecture will exist in a decade) to stake durable commitments itself, then? They're at least autonomous enough to run computers. I agree that that is a core part to true meaning & autonomy, but nobody just lets them yet. I don't think inferentialism is a consciousness-based argument, either, so that escape hatch isn't available. I also don't think this is a necessary condition to engage in the space of reasons (but maybe then I'm misunderstanding it, or retreating to a more Sellarsian understanding). Could you explain on this more?
Though again, I only have the book sitting on my desk, and I haven't cracked it open...
And yeah, consciousness based arguments are bunk IMO. It's clear that LLMs are in a basic sense "aware", and even to some extent "sapient" (in Brandom's usage), i.e. they can justify claims and object to things at least locally. I follow Rorty on the question of consciousness, which is to say that consciousness has no deeper meaning in practice beyond "are you awake", and in philosophical discussion it's largely a proxy for "are you one of us" in some amorphous sense.
Consciousness, the interesting part anyway, is qualia, or the "it is like something to be me" as Nagel put it.
Intention, as Dennett used it in his book "The Intentional Stance" is the "are you one of us" angle, and is distinct from consciousness because as the book title suggests, it is a stance taken with respect to some other system (person, cat, dog, turtle, LLM) rather than a description of subjective experience.
So I think it is your (and Rorty's) positions that are bunk. It is very, very far from clear that LLM's are in any basic sense "aware" let alone "sapient". They are extraordinarily able to manipulate language in a way that leads to text generation that has traditionally been assumed to indicate consciousness and/or intelligence. But we have no way of knowing if it is like anything to be an LLM, even though most of us are busy taking the Intentional Stance towards them.
And as I think Dennett himself would say, qualia are just Sellars's Given rearing its ugly head again.
I read this sentence 5 times and I still cannot parse it.
[1] https://sites.pitt.edu/~rbrandom/Courses/2022%20Phil%20of%20...
Edit: One of my ways of benchmarking new AI models is actually to type a version of this prompt in and see how sophisticated the response is: "Talk to me about the concept of deontic scorekeeping in the context of llms". You'll get good mileage out of that.
Deontology - `normative ethical theory that the morality of an action should be based on whether that action itself is right or wrong`
What an LLM can't do (at least with the current architecture) is sustain a commitment that it can be held accountable for later. It doesn't have durable "perspective". It can't update its priors. Maybe a new architecture will come along that gives LLMs memory plasticity, at which point they will shift into being individuated quasi-persons, but that's not what we've got now.
I've written some about this here, if you're interested: https://elliotmilco.substack.com/p/deontic-scorekeeping-comm...
I think you are also suggesting Mathematics and Language are the same thing.
I'm not suggesting mathematics and language are the same thing, but I am saying (which shouldn't be controversial) that mathematics as a discipline exists within the medium of language, and that mathematical reasoning, discovery, and proof happen via people proposing, justifying, testing, and objecting to formulations in language. This isn't a metaphysical claim about whether math is real or captures inherent structures of the universe etc.
You ended your first post with: "An LLM encodes the normative regularities of our linguistic/conceptual games into its weights, but it cannot be a real participant in our games because it is incapable of staking durable commitments or changing its dispositions over time."
And so if there is an agent, a thinking loop, that can recursively adjust over time...how does your statement hold up? I'm particularly talking about 'staking durable commitments or changing its dispositions over time'. Recursive self improvement is all about these 2 things, especially when it's the 'outer thinking loop' doing the improvement.
But you're right, Mathematics exists as a medium within Language. But then Language is just a transfer of ideas from one mind to another.
This principle is the heart of what the article is talking about: a transfer of logical mathematical ideas passed from a human to another. And the sense of 'shared understanding' that comes with it. It has nothing to do with keeping score.
It's about getting a 'sense' for why someone thought or rationalized an idea in order to connect some other idea. You get an 'ah-ha' moment, and some of those moments are gone, or a different path is taken to reach the moments now. That is what the 'weeping' is about (not that I agree with the article. I think anyone can spend however they want. Math doesn't have to change, it's that people want it to, apparently, because it's 'easier' to do something different, or for some other reason I don't know about).
Both depend on Logic, which is where "deontic scorekeeping" comes in. You can only establish something (score) is true and believable by 'keeping score' and having some level of understanding about the language or math you are reading.
The article is mostly referring to the connection that comes from shared understanding between humans, particularly mathematicians, as they build new shared understanding, and realize there are limits to what they can know and 'leave behind' logic for others to pick up and carry on.
That connection has been severed, or strained a bit, but I'm sure new ones will be created. It just might be of the same flavor that mathematicians prefer, therefore they weep.
And if I understand "deontic scorekeeping" correctly, these forums are a better representation of the concept. You will lend some amount of belief to what is said/read based on what you know.
I suppose we would see some interaction with AI in that sense, but it'll take place over the course of learn cycles and when AI starts ingesting its own ideas it typically goes wrong somewhere down the line.