This paper introduces a framework of using natural Chinese spoken instructions to navigate a robot at distance in the scenario of disaster rescues. A model of three layers of cascaded Conditional Random Fields (CRFs) is proposed to process the instructions, which outputs a series of structured commands that are called Continuous Movements (CMs) of robot. 12-type Navigation Part of Speech (NPOS) is defined and tagged through the first layer of CRFs, and the sequence of NPOS tagging is used as the main features (or observed sequence) for the other two layers of CRFs. Feature selection is critical to the degree of accuracy of the model; experiments with the text corpus we collected in our laboratory, and the results are analyzed with respect to the consideration of features selected. The overall rate of correctly understanding the instructions in our experiment is 70.79% which is still not acceptable in practical use where a confirmation dialog with the operator is needed. The demonstration of this frame is quite well in our lab experiments of navigating a Pioneer 3-AT robot.