For autonomous navigation, a mobile robot is required to move toward a destination while avoiding obstacles. In this paper, we present a motion planner based on CNN. In terms of obstacle avoidance, since a position of a dynamic obstacle changes with time, it is important for the robot to plan the avoidance motions in consideration of the time series variation in the images. For this purpose, an LSTM block is applied to the CNN. The policy of the motion planner represented by CNN with LSTM is trained through imitation learning. In this regard, however, it is difficult for the robot to recognize unknown objects as obstacles. For obstacle recognition, a perception process is further provided between the image inputs and CNN with LSTM in the motion planner. In the navigation experiments, we show that the robot based on the proposed motion planner is able to move toward a destination autonomously while avoiding a standing and walking person, respectively.