Hand tracking is a common task in a gesture recognition system. Many techniques have been introduced to make successful hand tracking. In hand tracking system, most of previous works tracked the hand position using attached marker on hands. Several researchers have used a color image for skin color detection. However, using marker based need to attach marker on hands or wear gloves to make hand can be detected. When using color information, there is a need to extract many different skin colors. Furthermore, the lighting and background on the situation also need to be concerned to avoid a cluttered background that can affect the detection and tracking. This paper presents the real-time hand tracking using three dimensional (3D) data. This 3D data is coming from the Kinect sensor, which is working in real-time. 3D data from Kinect sensor is depth image data which can be used to detect and track the motion of the hand. This paper proposes hand tracking method using a hand tracker algorithm released by NiTE, hand's segmentation method, hand contour detection and center of palm detection. The hand's segmentation method consists of the ROI of the hand's area and background subtraction. The propose hand tracking algorithm is rotation invariant, since it can detect and track various rotations of hand. Additionally, it also can remove unwanted object (noise) that also moving parallelly with the hand's position.