Reliable extrinsic calibration is critical to fuse camera and inertial measurement unit (IMU), which are usually used in stereo visual-inertial odometry (VIO), however, it is difficult to obtain extrinsic parameters in practice. This article proposes a stereo VIO with the capability of calibrating the unknown extrinsic parameters online. The initial values of IMU-camera and camera–camera transformations are estimated during the initial process, which fully leverages commonly observed features between stereo cameras. The pose and velocity are estimated and the extrinsic parameters are jointly refined. In addition, the accelerometer and gyroscope biases, and gravity direction are taken into account. The proposed VIO scheme has been demonstrated to be effective without prior knowledge of extrinsic parameters.