if you enable history substring search in zsh you can get that feature anywhere in the command and not just the start. it's very useful if you remember some snippet of the command like it had the word debug in it etc. in fish that is built in and on by default.

though one function that i recently added will narrow your history down based on a space separated set of filters which i found useful

    function hgrep
        set -l cmd history
        for pattern in $argv
            set cmd "$cmd | grep -- "(string escape "$pattern")
        end
        eval $cmd | cat
    end

use `hgrep ruby debug foo` would filter for a line that contained all the words ruby, debug and foo.

This is a readline feature. You can enable it in bash (and other shells?) with:

bind '"\e[A": history-search-backward'

bind '"\e[B": history-search-forward'

Clearly I am missing some setting. On zsh on macOS and I tried what you just said and it doesn't work like this

    bindkey '^[[A' history-search-backward
    bindkey '^[[B' history-search-forward