I'm not convinced he's done the calibration, at least in his demo video. It's pretty bad, imo.
replyI think he's just running it on the raw unrectified images.
For anyone wanting to play with this, OpenCV has a number of tutorials on this using stuff like stereoBM (which also uses SAD if I'm remembering my algos).
The first step is calibration -- intrinsics (basically figuring out the equations that can cancel out the lens distortion) and extrinsics (how far apart are the two cameras in XYZ, tilt, etc).
can this be done manually by setting constants till the images look rectified? last time i played with opencv for this it didnt do a very good job at automatically calculating the k's. im hoping an algo comes out that lets me do disparity without calibration one day :)
replyYou can use structure from motion for this. Walk around and capture some video. It’s the same way that drone photogrammetry works, ie bundle adjustment. It’ll give you the intrinsics at least.
replyBut with a high res sensor, even flex in the PCB will invalidate the calibration. Only takes a few microns for a pixel to be in the wrong place.
While I wouldn't say opencv makes it simple they do provide functions like findChessboardCorners, CharucoDetector, stereoCalibrate - even code to generate printable calibration patterns.
replyAnd these days, you can get an LLM to stitch those functions together
lots of “world models” are pretty good at “stereo with zero calibration”
replybut if you want to calibrate, try printing out a board https://calib.io/pages/camera-calibration-pattern-generator
Can also help if you can remove IR filter + add an IR noise projector
reply