You can still have it run normal scripting for the tasks, no need to put every sword swing thru AI. The option where AI just modifies that own scripting over time is also viable

Depends. If the models are smart enough to write enough solid helper scripts, so that the model only does the big picture thinking, not guiding every step .. then they will get powerful and capable quickly. I am also playing a bit with minecraft AI right now. Fun to experiment.