upvote
That is a good optimization, but you still need to figure out the starting state at a given location, and for quite a lot of languages that is not solvable in the local case.

E.g. if you're looking at a page of Ruby, you can not in the general case know if it's inside or outside a quoted string, as the quote character can be any arbitrary character (and I really mean any. "% x " excluding the double-quotes is a valid Ruby quoted string where the quote character is space but you can e.g. pick a unicode codepoint you're not going to use anywhere else in the file)

So unless you parse from the start of the file, you're left with fuzzy matching, and that can be made good enough the vast majority of time.

reply