research-article

Formally Reasoning About Quality

Authors:

Shaull Almagor,

Orna KupfermanAuthors Info & Claims

Journal of the ACM (JACM), Volume 63, Issue 3

Article No.: 24, Pages 1 - 56

https://doi.org/10.1145/2875421

Published: 17 June 2016 Publication History

Abstract

In recent years, there has been a growing need and interest in formally reasoning about the quality of software and hardware systems. As opposed to traditional verification, in which one considers the question of whether a system satisfies a given specification or not, reasoning about quality addresses the question of how well the system satisfies the specification. We distinguish between two approaches to specifying quality. The first, propositional quality, extends the specification formalism with propositional quality operators, which prioritize and weight different satisfaction possibilities. The second, temporal quality, refines the “eventually” operators of the specification formalism with discounting operators, whose semantics takes into an account the delay incurred in their satisfaction.

In this article, we introduce two quantitative extensions of Linear Temporal Logic (LTL), one by propositional quality operators and one by discounting operators. In both logics, the satisfaction value of a specification is a number in [0, 1], which describes the quality of the satisfaction. We demonstrate the usefulness of both extensions and study the decidability and complexity of the decision and search problems for them as well as for extensions of LTL that combine both types of operators.

References

[1]

Eric Allender, Peter Bürgisser, Johan Kjeldgaard-Pedersen, and Peter Bro Miltersen. 2009. On the complexity of numerical analysis. SIAM Journal on Computing 38, 5, 1987--2006.

Digital Library

[2]

Shaull Almagor, Guy Avni, and Orna Kupferman. 2013a. Automatic generation of quality specifications. In Proceedings of the 25th International Conference on Computer Aided Verification. Lecture Notes in Computer Science, Vol. 8044. Springer, Berlin, 479--494.

[3]

Shaull Almagor, Udi Boker, and Orna Kupferman. 2011. What’s decidable about weighted automata? In 9th International Symposium on Automated Technology for Verification and Analysis. Lecture Notes in Computer Science, Vol. 6996. Springer, Berlin, 482--491.

Digital Library

[4]

Shaull Almagor, Udi Boker, and Orna Kupferman. 2013b. Formalizing and reasoning about quality. In Proceedings of the 40th International Colloqium on Automata, Languages, and Programming. Lecture Notes in Computer Science, Vol. 7966. Springer, Berlin, 15--27.

Digital Library

[5]

Shaull Almagor, Udi Boker, and Orna Kupferman. 2014. Discounting in LTL. In Proceedings of the 20th International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, Vol. 8413. Springer, Berlin 424--439.

[6]

Shaull Almagor, Yoram Hirshfeld, and Orna Kupferman. 2010. Promptness in omega-regular automata. In 8th International Symposium on Automated Technology for Verification and Analysis. Lecture Notes in Computer Science, Vol. 6252. Springer, Berlin, 22--36.

Digital Library

[7]

Shaull Almagor and Orna Kupferman. 2011. Max and sum semantics for alternating weighted automata. In 9th International Symposium on Automated Technology for Verification and Analysis. Lecture Notes in Computer Science, Vol. 6996. Springer, Berlin, 13--27.

Digital Library

[8]

Shaull Almagor and Orna Kupferman. 2015. High-quality synthesis against stochastic environments (Submitted).

[9]

Sanjeev Arora and Boaz Barak. 2009. Computational Complexity—A Modern Approach. Cambridge University Press, New York, NY.

Digital Library

[10]

Roderick Bloem, Krishnendu Chatterjee, Thomas A. Henzinger, and Barbara Jobstmann. 2009. Better quality in synthesis through quantitative objectives. In Proceedings of the 21st International Conference on Computer Aided Verification. Lecture Notes in Computer Science, Vol. 5643. Springer, Berlin, 140--156.

Digital Library

[11]

A. Bohy, V. Bruyère, E. Filiot, and J.-F. Raskin. 2013. Synthesis from LTL specifications with mean-payoff objectives. In Proceedings of the 19th International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Lecture Notes in Computer Science, Vol. 7795. Springer, Berlin, 169--184.

Digital Library

[12]

Mikolaj Bojańczyk and Thomas Colcombet. 2006. Bounds in ω-regularity. In Proceedings of the 21st IEEE Symposium on Logic in Computer Science. 285--296.

Digital Library

[13]

Udi Boker, Krishnendu Chatterjee, Thomas A. Henzinger, and Orna Kupferman. 2014. Temporal specifications with accumulative values. ACM Transactions on Computational Logic 15, 4, 27:1--27:25.

Digital Library

[14]

Udi Boker, Thomas A. Henzinger, and Jan Otop. 2015. The target discounted-sum problem. In 30th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS’15), Kyoto, Japan. 750--761.

Digital Library

[15]

Udi Boker, Orna Kupferman, and Adin Rosenberg. 2010. Alternation removal in Büchi automata. In Proceedings of the 37th International Colloquium on Automata, Languages, and Programming. Lecture Notes in Computer Science, Vol. 6199. Springer, Berlin, 76--87.

Digital Library

[16]

Patricia Bouyer, Nicolas Markey, and Raj Mohan Matteplackel. 2014. Averaging in LTL. In Proceedings of CONCUR 2014 - Concurrency Theory - 25th International Conference (CONCUR’14), Rome, Italy. 266--280.

[17]

Glenn Bruns and Patrice Godefroid. 2004. Model checking with multi-valued logics. In Proceedings of the 31st International Colloquium on Automata, Languages, and Programming. Lecture Notes in Computer Science, Vol. 3142. 281--293.

[18]

Pavol Černỳ, Krishnendu Chatterjee, Thomas A. Henzinger, Arjun Radhakrishna, and Rohit Singh. 2011. Quantitative synthesis for concurrent programs. In Proceedings of the 23rd International Conference on Computer Aided Verification. 243--259.

Digital Library

[19]

Krishnendu Chatterjee, Vojtech Forejt, and Dominik Wojtczak. 2013. Multi-objective discounted reward verification in graphs and MDPs. In Proceedings of the 19th International Conference on Logic for Programming, Artificial Intelligence, and Reasoning (LPAR’19), Stellenbosch, South Africa. 228--242.

[20]

Edmund M. Clarke, Orna Grumberg, and Doron Peled. 1999. Model Checking. MIT Press, Cambridge, MA.

Digital Library

[21]

Mads Dam. 1994. CTL<sup>&sstarf;</sup> and ECTL<sup>&sstarf;</sup> as fragments of the modal μ-calculus. Theoretical Computer Science 126, 77--96.

Digital Library

[22]

Luca De Alfaro, Marco Faella, Thomas A. Henzinger, Rupak Majumdar, and Mariëlle Stoelinga. 2005. Model checking discounted temporal properties. Theoretical Computer Science 345, 1, 139--170.

Digital Library

[23]

Luca De Alfaro, Marco Faella, and Mariëlle Stoelinga. 2004. Linear and branching metrics for quantitative transition systems. In Proceedings of the 31st International Colloquium on Automata, Languages, and Programming. Lecture Notes in Computer Science, Vol. 3142. 97--109.

[24]

Luca De Alfaro, Thomas A. Henzinger, and Rupak Majumdar. 2003. Discounting the future in systems theory. In Proceedings of the 30th International Colloquium on Automata, Languages, and Programming. Lecture Notes in Computer Science, Vol. 2719. 1022--1037.

Digital Library

[25]

Josée Desharnais, Vineet Gupta, Radha Jagadeesan, and Prakash Panangaden. 2004. Metrics for labelled Markov processes. Theoretical Computer Science 318, 3, 323--354.

Digital Library

[26]

Alexandre Donzé and Oded Maler. 2010. Robust satisfaction of temporal logic over real-valued signals. In Proceedings of the 8th International Conference on Formal Modeling and Analysis of Timed Systems (FORMATS’10). 92--106.

Digital Library

[27]

Alexandre Donzé, Oded Maler, Ezio Bartocci, Dejan Nickovic, Radu Grosu, and Scott Smolka. 2012. On temporal logic and signal processing. In Proceedings of the 10th International Conference on Automated Technology for Verification and Analysis (ATVA’12). Springer, Berlin, 92--106.

Digital Library

[28]

M. Droste, W. Kuich, and H. Vogler (eds.). 2009. Handbook of Weighted Automata. Springer.

Digital Library

[29]

Manfred Droste, Werner Kuich, and George Rahonis. 2008. Multi-valued MSO logics OverWords and trees. Fundammenta Informaticae 84, 3--4, 305--327.

Digital Library

[30]

Manfred Droste and George Rahonis. 2009. Weighted automata and weighted logics with discounting. Theoretical Computer Science 410, 37, 3481--3494.

Digital Library

[31]

Manfred Droste and Heiko Vogler. 2012. Weighted automata and multi-valued logics over arbitrary bounded lattices. Theoretical Computer Science 418, 14--36.

Digital Library

[32]

E. Allen Emerson and Joseph Y. Halpern. 1986. Sometimes and not never revisited: On branching versus linear time. Journal of the ACM 33, 1, 151--178.

Digital Library

[33]

E. Allen Emerson and Chin-Laung Lei. 1985. Modalities for model checking: Branching time logic strikes back. In Proc. 12th ACM Symp. on Principles of Programming Languages. 84--96.

Digital Library

[34]

E. Allen Emerson and Chin-Laung Lei. 1986. Efficient model checking in fragments of the propositional μ-calculus. In Proceedings of the 1st IEEE Symposium on Logic in Computer Science. 267--278.

[35]

Marco Faella, Axel Legay, and Mariëlle Stoelinga. 2008. Model checking quantitative linear time logic. Electronic Notes in Theoretical Computer Science 220, 3, 61--77.

Digital Library

[36]

Emmanuel Filiot, Raffaella Gentilini, and Jean-François Raskin. 2014. Finite-valued weighted automata. In 34th International Conference on Foundation of Software Technology and Theoretical Computer Science, (FSTTCS’14), New Delhi, India. 133--145.

[37]

Martin Fränzle, Michael R. Hansen, and Heinrich Ody. 2015. Discounted duration calculus. In Proceedings of the 27th Nordic Workshop on Programming Theory. RUTR-SCS16001, 75--77.

[38]

Paul Gastin and Denis Oddoux. 2001. Fast LTL to Büchi automata translation. In Proceedings of the 13th International Conference on Computer Aided Verification. Lecture Notes in Computer Science, Vol. 2102. Springer, Berlin, 53--65.

Digital Library

[39]

Y. Gurevich and L. Harrington. 1982. Trees, automata, and games. In Proceedings of the 14th ACM Symposium on Theory of Computing. ACM Press, New York, NY, 60--65.

Digital Library

[40]

Hans Hansson and Bengt Jonsson. 1994. A logic for reasoning about time and reliability. Formal Aspects of Computing 6, 102--111.

Digital Library

[41]

Gerard J. Holzmann. 2004. The Spin Model Checker: Primer and Reference Manual. Addison-Wesley, New York, NY.

Digital Library

[42]

IEEE. 1993. IEEE Standard Multivalue Logic System for VHDL Model Interoperability (Std_logic_1164).

[43]

Stephen H. Kan. 2002. Metrics and Models in Software Quality Engineering (2nd ed.). Addison-Wesley Longman Publishing Co., Inc., New York, NY.

Digital Library

[44]

Daniel Kirsten and Sylvain Lombardy. 2009. Deciding unambiguity and sequentiality of polynomially ambiguous min-plus automata. In Proceedings of the 26th International Symposium on Theoretical Aspects of Computer Science (STACS’09). Freiburg, Germany, 589--600.

[45]

Daniel Krob. 1994. The equality problem for rational series with multiplicities in the tropical semiring is undecidable. International Journal of Algebra and Computation 4, 3, 405--425.

[46]

Orna Kupferman. 2006. Sanity checks in formal verification. In Proceedings of the 17th International Conference on Concurrency Theory. Lecture Notes in Computer Science, Vol. 4137. Springer, Berlin, 37--51.

Digital Library

[47]

Orna Kupferman and Yoad Lustig. 2007. Lattice automata. In Proceedings of the 8th International Conference on Verification, Model Checking, and Abstract Interpretation. Lecture Notes in Computer Science, Vol. 4349. Springer, Berlin, 199--213.

Digital Library

[48]

Orna Kupferman, Nir Piterman, and Moshe Y. Vardi. 2006. Safraless compositional synthesis. In Proceedings of the 18th International Conference on Computer Aided Verification. Lecture Notes in Computer Science, Vol. 4144. Springer, Berlin, 31--44.

Digital Library

[49]

Orna Kupferman, Nir Piterman, and Moshe Y. Vardi. 2009. From liveness to promptness. Formal Methods in System Design 34, 2, 83--103.

Digital Library

[50]

Orna Kupferman and Moshe Y. Vardi. 1997. Synthesis with incomplete information. In 2nd International Conference on Temporal Logic. 91--106.

[51]

Orna Kupferman and Moshe Y. Vardi. 2005. Safraless decision procedures. In Proceedings of the 46th IEEE Symposium on Foundations of Computer Science. 531--540.

Digital Library

[52]

Orna Kupferman, Moshe Y. Vardi, and Pierre Wolper. 2000. An automata-theoretic approach to branching-time model checking. Journal of the ACM 47, 2, 312--360.

Digital Library

[53]

Robert P. Kurshan. 1998. FormalCheck User’s Manual. Cadence Design Systems, Inc. Columbia, MD.

[54]

Marta Z. Kwiatkowska. 2007. Quantitative verification: Models techniques and tools. In ESEC/SIGSOFT FSE. 449--458.

Digital Library

[55]

François Laroussinie and Ph Schnoebelen. 1994. A hierarchy of temporal logics with past. In Proceedings of the 11th Symposium on Theoretical Aspects of Computer Science. 47--58.

Digital Library

[56]

Orna Lichtenstein, Amir Pnueli, and Lenore Zuck. 1985. The glory of the past. In Logics of Programs. Lecture Notes in Computer Science, Vol. 193. Springer, Berlin, 196--218.

Digital Library

[57]

Eleni Mandrali. 2012. Weighted LTL with discounting. In CIAA. Lecture Notes in Computer Science, Vol. 7381. Springer, Berlin, 353--360.

Digital Library

[58]

Z. Manna and A. Pnueli. 1995. The Temporal Logic of Reactive and Concurrent Systems: Safety. Springer.

Digital Library

[59]

Marvin L. Minsky. 1967. Computation: Finite and Infinite Machines (1 ed.). Prentice Hall, Upper Saddle River, NJ.

Digital Library

[60]

Satoru Miyano and Takeshi Hayashi. 1984. Alternating finite automata on ω-words. Theoretical Computer Science 32, 321--330.

[61]

Mehryar Mohri. 1997. Finite-state transducers in language and speech processing. Computational Linguistics 23, 2, 269--311.

Digital Library

[62]

Seong-ick Moon, Kwang Hyung Lee, and Doheon Lee. 2004. Fuzzy branching temporal logic. IEEE Transactions on Systems, Man, and Cybernetics, Part B 34, 2, 1045--1055.

Digital Library

[63]

Shota Nakagawa and Ichiro Hasuo. 2015. Near-optimal scheduler synthesis for LTL with future discounting. In 10th International Symposium on Trustworthy Global Computing.

[64]

N. Piterman, A. Pnueli, and Y. Saar. 2006. Synthesis of reactive(1) designs. In Proceedings of the 7th International Conference on Verification, Model Checking, and Abstract Interpretation. Lecture Notes in Computer Science, Vol. 3855. Springer, Berlin, 364--380.

Digital Library

[65]

Amir Pnueli and Roni Rosner. 1989. On the synthesis of a reactive module. In Proceedings of the 16th ACM Symposium on Principles of Programming Languages. 179--190.

Digital Library

[66]

Claude E. Shannon. 1949. The synthesis of two terminal switching circuits. 28, 1, 59--98.

[67]

Lloyd S. Shapley. 1953. Stochastic games. In Proceedings of the National Academy of Science, Vol. 39. National Academy of Sciences, 1095.

[68]

Diomidis Spinellis. 2003. Code Reading: The Open Source Perspective. Addison-Wesley, New York, NY.

Digital Library

[69]

Wolfgang Thomas. 1990. Automata on infinite objects. Handbook of Theoretical Computer Science 2, 133--191.

Digital Library

[70]

Moshe Y. Vardi. 1996. An automata-theoretic approach to linear temporal logic. In Logics for Concurrency: Structure versus Automata. Lecture Notes in Computer Science, F. Moller and G. Birtwistle (Eds.), Vol. 1043. Springer, Berliln, 238--266.

Digital Library

[71]

Moshe Y. Vardi and Pierre Wolper. 1986. An automata-theoretic approach to automatic program verification. In Proceedings of the 1st IEEE Symposium on Logic in Computer Science. 332--344.

[72]

Moshe Y. Vardi and Pierre Wolper. 1994. Reasoning about infinite computations. Information and Computation 115, 1, 1--37.

Digital Library

Cited By

Mittelmann MMaubert BMurano APerrussel L(2025)Formal verification and synthesis of mechanisms for social choiceArtificial Intelligence10.1016/j.artint.2024.104272339:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.artint.2024.104272
Aguilera JDiéguez MFernández-Duque DMcLean B(2025)Gödel–Dummett linear temporal logicArtificial Intelligence10.1016/j.artint.2024.104236338(104236)Online publication date: Jan-2025
https://doi.org/10.1016/j.artint.2024.104236
Kupferman OShenwald N(2025)Games with Weighted Multiple ObjectivesAutomated Technology for Verification and Analysis10.1007/978-3-031-78709-6_6(110-132)Online publication date: 1-Feb-2025
https://doi.org/10.1007/978-3-031-78709-6_6
Show More Cited By

Index Terms

Formally Reasoning About Quality
1. Theory of computation
  1. Logic
  2. Models of computation
    1. Computability

Recommendations

Model Checking LTL Formulae in RAISE with FDR
IFM '09: Proceedings of the 7th International Conference on Integrated Formal Methods

The Raise Specification Language (RSL) is a modeling language which supports various specification styles. To apply model checking to RSL concurrent descriptions, we translate RSL specifications into the input language CSPM of FDR. FDR is the model ...
Validated templates for specification of complex LTL formulas

Formal verification approaches that check software correctness against formal specifications have been shown to improve program dependability. Tools such as Specification Pattern System (SPS) and Property Specification (Prospec) support the generation ...
Seven at one stroke: LTL model checking for high-level specifications in B, Z, CSP, and more

The size of formal models is steadily increasing and there is a demand from industrial users to be able to use expressive temporal query languages for validating and exploring high-level formal specifications. We present an extension of LTL, which is ...

Comments

Information & Contributors

Information

Published In

cover image Journal of the ACM

Journal of the ACM Volume 63, Issue 3

September 2016

303 pages

ISSN:0004-5411

EISSN:1557-735X

DOI:10.1145/2957788

Editor:
Éva Tardos
Cornell University

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2016

Accepted: 01 January 2016

Revised: 01 November 2015

Received: 01 November 2014

Published in JACM Volume 63, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

ERC
European Research Council under the European Union's Seventh Framework Programme

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

42
Total Citations
View Citations
507
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)3

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mittelmann MMaubert BMurano APerrussel L(2025)Formal verification and synthesis of mechanisms for social choiceArtificial Intelligence10.1016/j.artint.2024.104272339:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.artint.2024.104272
Aguilera JDiéguez MFernández-Duque DMcLean B(2025)Gödel–Dummett linear temporal logicArtificial Intelligence10.1016/j.artint.2024.104236338(104236)Online publication date: Jan-2025
https://doi.org/10.1016/j.artint.2024.104236
Kupferman OShenwald N(2025)Games with Weighted Multiple ObjectivesAutomated Technology for Verification and Analysis10.1007/978-3-031-78709-6_6(110-132)Online publication date: 1-Feb-2025
https://doi.org/10.1007/978-3-031-78709-6_6
Jamroga WMittelmann MMurano APerelli GDastani MSichman JAlechina NDignum V(2024)Playing Quantitative Games Against an Authority: On the Module Checking ProblemProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662947(926-934)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662947
Hyland DGutierrez JShankaranarayanan KWooldridge MDastani MSichman JAlechina NDignum V(2024)Rational Verification with Quantitative Probabilistic GoalsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3662941(871-879)Online publication date: 6-May-2024
https://dl.acm.org/doi/10.5555/3635637.3662941
Yu XLi YGeng S(2024)Fuzzy Linear Temporal Logic with Quality ConstraintsMathematics10.3390/math1219314812:19(3148)Online publication date: 8-Oct-2024
https://doi.org/10.3390/math12193148
Yu XLi YGeng SLi H(2024)Fuzzy Computation Tree Temporal Logic with Quality Constraints and Its Model CheckingAxioms10.3390/axioms1312083213:12(832)Online publication date: 27-Nov-2024
https://doi.org/10.3390/axioms13120832
Hyland DMittelmann MMurano APerelli GWooldridge MMarquis POrtiz MPagnucco M(2024)Incentive design for rational agentsProceedings of the 21st International Conference on Principles of Knowledge Representation and Reasoning10.24963/kr.2024/44(464-474)Online publication date: 2-Nov-2024
https://dl.acm.org/doi/10.24963/kr.2024/44
Cecconi ABarbaro LDi Ciccio CSenderovich A(2024)Measuring rule-based LTLf process specifications: A probabilistic data-driven approachInformation Systems10.1016/j.is.2023.102312120(102312)Online publication date: Feb-2024
https://doi.org/10.1016/j.is.2023.102312
Nayak SNeider DRoy RZimmermann M(2024)Robust computation tree logicInnovations in Systems and Software Engineering10.1007/s11334-024-00552-7Online publication date: 20-Mar-2024
https://doi.org/10.1007/s11334-024-00552-7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents