The fusion of infrared and visible images is hard due to their different modalities. Different from existing methods using the integer-order gradient, we design an optimization model to fuse infrared and visible images using fractional-order gradient information. In this way, the complementary information of the source images can be better preserved. In order to better highlight the target and retain effective details, we use the results of the optimization model as pre-fusion images to guide the final image fusion. For better highlighting the target, we use MDLatLRR to extract the base layer of the pre-fusion image and use it as the base layer of the fused image. In addition, for getting more effective details, we use the pre-fusion image to calculate the weight map, which is used as the tradeoff parameter of the norm optimization problem to get the fused detail layers. Experimental results show that our method can highlight the target better while maintaining effective details. Compared with the current state-of-the-art image fusion methods, our method shows better fusion performance in both subjective and objective evaluation.