Object tracking for video and camera image capturing becomes more popular in recent scientific and application domain. Many Scientists in Image Processing try to find an efficient and accurate way to track the objects' moving in real time video applications. Particle Swarm Optimization is one of the most efficient and potential approaches developed to provide linear and non-linear object tracking by identifying special patterns for selected object, and achieving efficient algorithm to calculate random object moving. Compare to classic Kalman Filter, Particle Filter, or other improved PSO, this paper is aim to find a more efficient and precious pathway using categorized particle movement with dynamic inertia weight value in PSO to build a tracking procedure in order to provide better tracking speed and quality result in different types of video records and different visions of object moving. The proposed Categorized PSO based target tracking scheme can achieve better quality when compared to Particle Filter and typical Particle Swarm Optimization methods in our tracking experiments. It has been developed in C++ environment and tested against videos to demonstrate its excellent tracking ability in object retrieval and high accurate in tracking objects movement within other similar object's environment.