In the covering information system with decision-making (CISD), τ-lower and upper approximation operators are introduced, and some corresponding properties are discussed. This paper also explores reductions of a covering which is based on a new concept of the degree of approximate dependency, and proposes a reduction algorithm based on the importance degree. After reduction, a decision tree is generated and rules are extracted from the decision tree. Finally, the above mentioned method is demonstrated by an example in medical diagnosis.