Based on the analysis of the interaction between a manipulator's hand and a working object, a model representing the constrained dynamics of the robot is first discussed. The constraint forces are expressed by an algebraic function of states, input generalized forces, and constraint condition, and then direct position / force controller without force sensor is proposed based on the algebraic relation. To give the grinding system the ability to adapt to any object shape being changed by the grinding, we added estimating function of the constraint condition in real time for the adaptive position / force control, which is indispensable for our method instead of not using force sensor. Evaluations through continuous shape-grinding experiment by fitting the changing constraint surface with spline functions, indicates that reliable position / force control and shape-grinding work can be achieved by this proposed controller.