Okular handles cbz that contain jxl with no issue. (IIUC both archive format and image format support is provided via a pluggable extension system but I don't recall the details because my setup has "just worked" for a very long time now.)

Also FYI you can use mupdf to read cbz archives although I don't personally recommend it for that usecase.