upvote
Yes and no, sol hits a point where reframing its working context becomes hard. It sticks to what you harness very well, but changes become harder and harder.

E.g. ask it to make contract for a spec in code and then ask it to violate that contract. Overall an excellent model, just need to stop and put it back into we are harnessing or specing not building for a few turns not just try to pivot it off with one prompt.

reply
This is the same as when using LLMs as chatbots to ask questions.

If you ask another question in the same context it has all the information from that context and will be difficult to stray from anything in that context, e.g. if the LLM has gone down the wrong path or you are doing something slightly differently then it is difficult to steer the LLM away from the old context. This is why I tend to start a new context whenever I ask a question even if related to a previous question/answer. It can also be useful to do if/when the LLM gets stuck as a way of resetting it.

Note: this is probably why sub-agents are useful/work as they have a new context history.

reply