This paper presents the design of the image-based control algorithm for interactive Earth observation. The image-based control algorithm is obtained from the modelling of the satellite pose in space. It is shown that the image-based control algorithm can be designed for two types of satellite attitude control problems: direction tracking and oriented-direction tracking. The reference target is not limited to the camera centre, but can be given anywhere in the image. The image-based control algorithm requires in-image tracking of one or two points on the Earth's surface (depending on the type of the controller). To achieve robust image-based tracking, the general framework for tracking points on the Earth's surface, which is assumed to be locally flat, is presented. The method is based on geometric local image features that are invariant to several image transformations and change in some environmental conditions. The presented methods are experimentally validated in simulation environment.