Independently Checkable Proofs from Decision Procedures: Issues and Progress

Abstract

In many verification applications the desired outcome is that the formula is unsatisfiable: a satisfying assignment essentially exhibits a bug and unsatisfiability implies a lack of bugs, at least for the property being verified. Most current high-performance satisfiability solvers and special-theory decision procedures are unable to provide proof of unsatisfiability. Since bugs have been discovered in many such programs long after being put into service, an uncheckable decision poses a significant problem if important economic or safety decisions are to be based upon it. This talk develops the thesis is that decision procedures can and should be designed with the ability to output an independently checkable proof. While finding a proof is hard, checking a proof can be straightforward if the proof system is simple enough. (By a “proof” we mean a real proof, with no steps omitted.) In practice, most underlying theories can produce a resolution proof. We argue that outputting such a proof does not place an undue burden on the decision procedures. We report on practical progress in this area for satisfiability solvers. Experiments have been carried out with what might be the first implementations of solver and proof checker that were developed completely independently, having only the specifications of the proof-file format as common knowledge. There is a trend toward combining high-performance satisfiability solvers with other theorem-proving methods. As the total systems become more complex, the need for “independent audits” becomes greater. Design goals for checkable proofs are proposed.