Moving objects or more generally foreground objects are the simplest objects in the field of computer vision after the pixel. Indeed, a moving object can be defined by 4 integers only, either two pairs of coordinates or a pair of coordinates and the size. In fixed camera scenes, moving objects (or blobs) can be extracted quite easily but the methods to produce them are not able to tell if a blob corresponds to remaining background noise, a single target or if there is an occlusion between many target which are too close together thus creating a single blob resulting from the fusion of all targets. In this paper we propose an novel method to refine moving object detection results in order to get as many blobs as targets on the scene by using a tracking system for additional information. Knowing if a blob is at proximity of a tracker allows us to remove noise blobs, keep the rest and handle occlusions when there are more than one tracker on a blob. The results show that the refinement is an efficient tool to sort good blobs from noise blobs and accurate enough to perform a tracking based on moving objects. The tracking process is a resolution free system able to reach speed such as 20 000fps even for UHDTV sequences. The refinement process itself is in real time, running at more than 2000fps in difficult situations. Different tests are presented to show the efficiency of the noise removal and the reality of the independence of the refinement tracking system from the resolution of the videos.