AFAIK this is not secure. The SSH protocol doesn’t require the server to know the client’s public key for the connection to succeed. Instead, the client actively sends its public key to the server while authenticating, and the server just decides whether to allow a connection or not. OpenSSH will typically match the client’s key against a fixed list of keys, but nothing stops a rogue server from allowing connections from any client key. In fact, OpenSSH itself can be configured to do this using AuthorizedKeysCommand.
replyIt does need to know the public key. The AuthorizedKeysCommand does not provide a yes/no answer, it has to respond with a list of full public keys (or certificates) that are authorized to connect for a given user id. The SSH server then uses the public key to challenge the client to demonstrate it has the private key.
replyHowever, an easy attack in the same ballpark, is to accept the connection without any password or public key auth, and then accept agent forwarding, and ask that agent connection to authorize a connection to a target server, with the user's keys. Never forward your agent connection to an untrusted host. Though -- I imagine this pattern is common when setting up a new host -- trust the first connection, and forward your agent so you can pull resources (like git repos) from the new host to set it up ...