Skip to main content

Advertisement

SpringerLink
Log in

Auditing a database under retention policies

  • Regular Paper
  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract

Auditing the changes to a database is critical for identifying malicious behavior, maintaining data quality, and improving system performance. But an accurate audit log is an historical record of the past that can also pose a serious threat to privacy. Policies that limit data retention conflict with the goal of accurate auditing, and data owners have to carefully balance the need for policy compliance with the goal of accurate auditing. In this paper, we provide a framework for auditing the changes to a database system while respecting data retention policies. Our framework includes an historical data model that supports flexible audit queries, along with a language for retention policies that can hide individual attribute values or remove entire tuples from the history. Under retention policies, the audit history is partially incomplete. Thus, audit queries on the protected history can include imprecise results. We propose two different models (a tuple-independent model and a tuple-correlated model) for formalizing the meaning of audit queries. We implement policy application and query answering efficiently in a standard relational system and characterize the cases where accurate auditing can be achieved under retention restrictions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Antova, L., Jansen, T., Koch, C., Olteanu, D.: Fast and simple relational processing of uncertain data. In: ICDE, pp. 983–992 (2008)

  2. ARMA Internaltional: Generally Accepted Recordkeeping Principles. http://www.arma.org/GARP/

  3. Ataullah, A., Aboulnaga, A., Tompa, F.: Records retention in relational database systems. In: Proceeding of the ACM Conference on Information and Knowledge Management (CIKM), pp. 873–882 (2008)

  4. Bertino, E., Bettini, C., Samarati, P.: A temporal authorization model. In: ACM Conference on Computer and Communications Security (CCS), pp. 126–135. ACM Press, New York (1994)

  5. Biskup J.: A foundation of codd’s relational maybe-operations. ACM Trans. Database Syst. 8, 608–636 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  6. Blakeley J., Coburn N., Larson P.: Updating derived relations: detecting irrelevant and autonomously computable updates. TODS 14(3), 369–400 (1989)

    Article  MathSciNet  Google Scholar 

  7. Blakeley J.A., Larson P.A., Tompa F.W.: Efficiently updating materialized views. SIGMOD Rec. 15(2), 61–71 (1986)

    Article  Google Scholar 

  8. Chomicki, J.: Temporal query languages: a survey. In: Temporal Logic (ICTL’94), vol. 827, pp. 506–534 (1994)

  9. EMC Corporation: http://www.emc.com

  10. Fabbri, D., LeFevre, K., Zhu, Q.: PolicyReplay: misconfiguration-response queries for data breach reporting. In: Proceedings of the VLDB Endowment, vol. 3, no. (1–2), pp. 36–47 (2010)

  11. Gadia S.K.: A homogeneous relational model and query languages for temporal databases. ACM Trans. Database Syst. 13, 418–448 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gadia, S.K., Nair, S.S., Poon, Y.C.: Incomplete information in relational temporal databases. In: 18th VLDB Conference (1992)

  13. Garcia-Molina, H., Labio, W., Yang, J.: Expiring data in a warehouse. In: VLDB Conference, pp. 500–511 (1998)

  14. Grahne G.: The Problem of Incomplete Information in Relational Databases. Springer, Berlin (1991)

    Book  MATH  Google Scholar 

  15. GRM LLC: http://www.grmdocumentmanagement.com

  16. Guo S., Sun W., Weiss M.: Solving satisfiability and implication problems in database systems. ACM Trans. Database Syst. 21(2), 270–293 (1996)

    Article  Google Scholar 

  17. Hasan, R., Winslett, M.: Trustworthy vacuuming and litigation holds in long-term high-integrity records retention. In: Proceedings of the 13th International Conference on Extending Database Technology, pp. 621–632. ACM (2010)

  18. Hasan, R., Winslett, M., Mitra, S.: Efficient Audit-based Compliance for Relational Data Retention. UIUC Dept. of CS Tech Report UIUCDCS-R-2009-3044 (2009)

  19. Hochbaum D., Moreno-Centeno E.: The inequality-satisfiability problem. Oper. Res. Lett. 36(2), 229–233 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  20. Imielinski T., Lipski W.: Incomplete information in relational databases. J. ACM 31(4), 761–791 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  21. Jensen C.S., Mark L.: Queries on change in an extended relational model. IEEE TKDE 4, 192–200 (1992)

    Google Scholar 

  22. Jensen C.S., Mark L., Roussopoulos N.: Incremental implementation model for relational databases with transaction time. IEEE Trans. Knowl. Data Eng. 3, 461–473 (1991)

    Article  Google Scholar 

  23. Koubarakis M.: Database models for infinite and indefinite temporal information. Inf. Syst. 19, 141 (1994)

    Article  Google Scholar 

  24. Lageweg B., Lenstra J., Kan A.: Minimizing maximum lateness on one machine: computational experience and some applications. Stat. Neerl. 30(1), 25–41 (1976)

    Article  MATH  Google Scholar 

  25. LeFevre, K., Agrawal, R., Ercegovac, V., Ramakrishnan, R., Xu, Y., DeWitt, D.: Limiting disclosure in hippocratic databases. In: VLDB Conference, pp. 108–119 (2004)

  26. LexisNexis: Document Retention & Destruction Policies for Digital Data. http://www.lexisnexis.com/applieddiscovery/lawlibrary/whitePapers/ADI_WP_DocRetentionDestruction.pdf

  27. Lomet, D.B., Barga, R.S., Mokbel, M.F., Shegalov, G., Wang, R. Zhu, Y.: Transaction time support inside a database engine. In: ICDE, p. 35 (2006)

  28. Lu, W., Miklau, G.: AuditGuard: a system for database auditing under retention restrictions. IN: Proceedings of the VLDB Endowment vol. 1, no. 2, pp. 1484–1487 (2008)

  29. Lu, W., Miklau, G.: Auditing a database under retention restrictions. In: IEEE International Conference on Data Engineering (ICDE), pp. 42–53 (2009)

  30. Mullins, C.S.: Database Archiving for Long-term Data Retention. http://www.tdan.com/view-articles/4591 (2006)

  31. OpenText Corporation: http://www.opentext.com

  32. Perez, R.A., Moreau, L.: Securing provenance-based audits. In: International Provenance and Annotation Workshop 2010. Springer, Berlin (2010)

  33. RainStor Inc.: http://rainstor.com

  34. Rosenkrantz, D.J., Hunt, H.B.: Processing conjunctive predicates and queries. In: VLDB Conference, p. 72 (1980)

  35. SAND Technology: http://www.sand.com

  36. Sarda N.L.: Extensions to sql for historical databases. IEEE Trans. Knowl. Data Eng. 2, 220–230 (1990)

    Article  Google Scholar 

  37. Sarma, A., Benjelloun, O., Halevy, A., Widom, J.: Working models for uncertain data. In: ICDE (2006)

  38. Schneier B., Kelsey J.: Secure audit logs to support computer forensics. ACM Trans. Inf. Syst. Secur. 2(2), 159–176 (1999)

    Article  Google Scholar 

  39. Shaull, R., Shrira, L., Xu, H.: Skippy: a new snapshot indexing method for time travel in the storage manager. In: ACM SIGMOD Conference, pp. 637–648 (2008)

  40. Simons, B., Sipser, M.: On scheduling unit-length jobs with multiple release time/deadline intervals. Oper. Res. 80–88 (1984)

  41. Skyt J., Jensen C., Mark L.: A foundation for vacuuming temporal databases. Data Knowl. Eng. 44(1), 1–29 (2003)

    Article  MATH  Google Scholar 

  42. Snodgrass, R., Yao, S., Collberg, C.: Tamper detection in audit logs. In: 13th VLDB Conference, pp. 504–515 (2004)

  43. Snodgrass R.T.: The TSQL2 Temporal Query Language. Kluwer Academic Publishers, Norwell (1995)

    Book  MATH  Google Scholar 

  44. Snodgrass, R.T.: Developing time-oriented database applications in SQL. Morgan Kaufmann Publishers Inc., San Francisco (1999)

  45. Snodgrass, R.T., Collberg, C.S.: The τ-BerkeleyDB Temporal Subsystem. Published: Available at http://www.cs.arizona.edu/tau/tbdb/

  46. Stahlberg, P., Miklau, G., Levine, B.N.: Threats to privacy in the forensic analysis of database systems. In: SIGMOD Conference, pp. 91–102 (2007)

  47. Toman, D.: Expiration of historical databases. In: Symposium on Temporal Representation and Reasoning (TIME), pp. 128–135 (2001)

  48. Wang, Q., Yu, T., Li, N., Lobo, J., Bertino, E., Irwin, K., Byun, J.W.: On the correctness criteria of fine-grained access control in relational databases. In: VLDB Conference, pp. 555–566 (2007)

  49. Waters, B., Balfanz, D., Durfee, G., Smetters, D.: Building an encrypted and searchable audit log. In: NDSS, vol. 6 (2004)

  50. Wrozek, B.: Electronic Data Retention Policy (2001). http://www.sans.org/reading_room/whitepapers/backup/electronic-data-retention-policy_514

  51. ZL Technologies, Inc.: http://www.zlti.com

  52. ZyLAB: http://www.zylab.com

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Massachusetts, 140 Governors Drive, Amherst, MA, 01003, USA

    Wentian Lu, Gerome Miklau & Neil Immerman

Authors
  1. Wentian Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerome Miklau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neil Immerman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wentian Lu.

Additional information

The authors gratefully acknowledge the comments of the VLDBJ editors and the anonymous reviewers. Authors Lu and Miklau were supported by NSF CAREER Grant No. 0643681.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, W., Miklau, G. & Immerman, N. Auditing a database under retention policies. The VLDB Journal 22, 203–228 (2013). https://doi.org/10.1007/s00778-012-0282-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-012-0282-x

Keywords

Search

Navigation