The foraging problem, which evolved from ants finding food and delivering them to the nest, is for a swarm of robots to transport objects from a source to a destination. To avoid collision of robots or congestions, object transportation is generally conducted in a pipelining manner. Existing solutions towards the foraging problem either have performance problems or cannot tolerate failures. We present a self-stabilizing solution for the swarm robotic system to form a pipeline structure to transport objects and achieve the foraging task. In this paper, we first define the stable state for the swarm, which includes two requirements: (1) the robots operate in non-overlapping regions, i.e., transport objects in a pipeline structure and (2) the transportation rate of the system is optimal. Then, we introduce our self-stabilizing algorithm for the foraging problem and prove its convergence and the correctness of its convergence properties. Due to the self-stabilization nature, our solution is decentralized and fault tolerant. The swarm can achieve the foraging task as long as there is at least one working robot. Due to the definition of the stable state, our swarm system, when converged, can achieve optimal performance. We conduct simulations to evaluate the effectiveness of our algorithm, and the experimental results show that from any state, our algorithm can converge very quickly to reach the stable state and provide optimal performance for object transportation.