Collective object manipulation is one of the frequently observed behaviors in natural swarm systems which inspired many researchers to study and implement such methods via robotic systems. Adding obstacle avoidance capability to the agents involved in the manipulation task creates a ground for parallel object manipulation and collective construction in cluttered environments. In this paper, the obstacle avoidance problem is studied in a group of agents with limited vision range as they are manipulating an object with an arbitrary shape. In order to decrease the energy consumption of the system and reduce the problems associated with communication, the obstacle avoidance problem is formulated as an agent-level reaction which eliminates the need for inter-agent communications or information symmetry in the system. Building upon our previously developed decentralized force control algorithm, the presented methods combines two behaviors that are defined based on manipulation and avoidance forces to avoid possible obstacles in the region. Multiple simulated scenarios and detailed studies of the system are presented in the article which suggest the effectiveness of adding a simple agent-level behavior to create a successful decentralized object manipulation while avoiding obstacles.