An adaptive multi-direction image fusion algorithm is proposed to solve the problem that one fusion algorithm is not suitable for many kinds of image stitching, and there will be significant seam-line at the boundary of the overlap region after image fusion. According to the positional relation of the two images in the coordinate system after projection deformation, we divide the image stitching into two types. We design different image fusion algorithm against the characteristics of each type of stitching, and seam-line be in all boundary direction is eliminated. The algorithm can estimate the type of image stitching rapidly and accurately and perform the corresponding multi-direction image fusion and eventually eliminate the seam-line. Adding brightness adjustment before stitching will make the final result achieve consistency in brightness if the two images for stitching have a considerable difference in brightness.