Skip to main content
Springer Nature Link
Log in

Plausible Deniability of Redacted Text

  • Conference paper
  • First Online:
Computer Security. ESORICS 2024 International Workshops (ESORICS 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15263))

Included in the following conference series:

  • 59 Accesses

Abstract

Providing privacy for natural language text data remains a largely open problem, despite its great practical importance. The current state of the art is manual redaction of sensitive words such as names, addresses etc. In this paper we propose viewing a corpus of text as a probability distribution over sequences of words. A sentence is then one realization from this distribution and redacting words changes the probability distribution. We use the Renyi-divergence divergence as a measure of the distance between two redacted datasets. We show that if enough words are redacted then sensitive redacted text can be made be statistically indistinguishable from non-sensitive redacted text. This can be used to develop efficient redaction strategies, that minimise the amount of redaction while meeting a privacy target.

This work was supported by Science Foundation Ireland grant 16/IA/4610.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Adding noise to an embedding perturbs it to nearby words, the way in which words are mapped to be close together (or far apart) therefore directly affects the output of the word-level DP sanitisation process.

  2. 2.

    See Appendix https://anonymous.4open.science/r/appendix_repo-F4CC for more details.

  3. 3.

    The choice of embedding will, in general, affect the estimated divergence. This can be mitigated by calculating the divergence for many different embeddings and using the worst-case (i.e. largest) value. However, we found the impact to be relatively minor in practice, see Section-6.3, and SentenceBERT [15] to work well.

  4. 4.

    We select the range to be large enough that \(D_{\alpha }\) no longer increases as we increase \(\alpha \).

  5. 5.

    https://huggingface.co/datasets/medal.

  6. 6.

    Data can be downloaded by following the instructions in the repository https://github.com/xuqiongkai/PATR.

  7. 7.

    https://huggingface.co/datasets/amazon_reviews_multi.

  8. 8.

    https://www.kaggle.com/general/256134.

  9. 9.

    In particular, the DP analysis ignores correlations between the words in a sentence and so may greatly underestimate the information release. The impact of correlations on DP is well known and was first noted by [9].

  10. 10.

    https://anonymous.4open.science/r/appendix_repo-F4CC.

  11. 11.

    Training code can be found at: https://github.com/pytorch/examples/tree/main/word_language_model.

  12. 12.

    And one of the major deficiencies of all approaches tied to a single up front choice of embedding, such as word-level DP approaches.

  13. 13.

    https://www.sbert.net/.

  14. 14.

    The embedding vector of each word in a sentence is calculated, and the mean of these vectors is used as the sentence embedding.

References

  1. Abadi, M., et al.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318. CCS ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2976749.2978318

  2. Bosch, N., Crues, R., Shaik, N., Paquette, L.: Hello, [REDACTED]: protecting student privacy in analyses of online discussion forums. Grantee Submission (2020)

    Google Scholar 

  3. Brown, H., Lee, K., Mireshghallah, F., Shokri, R., Tramèr, F.: What does it mean for a language model to preserve privacy? In: 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 2280–2292. FAccT ’22, Association for Computing Machinery, New York, NY, USA (2022). https://doi.org/10.1145/3531146.3534642

  4. Bun, M., Steinke, T.: Concentrated differential privacy: simplifications, extensions, and lower bounds (2016)

    Google Scholar 

  5. Chen, S., et al.: A customized text sanitization mechanism with differential privacy. In: Findings of the Association for Computational Linguistics: ACL 2023, pp. 5747–5758. Association for Computational Linguistics, Toronto, Canada (2023). https://doi.org/10.18653/v1/2023.findings-acl.355

  6. Doudalis, S., Kotsogiannis, I., Haney, S., Machanavajjhala, A., Mehrotra, S.: One-sided differential privacy (2017)

    Google Scholar 

  7. Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) Theory of Cryptography, pp. 265–284. Springer, Berlin, Heidelberg (2006)

    Chapter  MATH  Google Scholar 

  8. Feyisetan, O., Balle, B., Drake, T., Diethe, T.: Privacy- and utility-preserving textual analysis via calibrated multivariate perturbations. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp. 178–186. WSDM ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3336191.3371856

  9. Kifer, D., Machanavajjhala, A.: No free lunch in data privacy. In: Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data, pp. 193–204. SIGMOD ’11, Association for Computing Machinery, New York, NY, USA (2011). https://doi.org/10.1145/1989323.1989345

  10. Mattern, J., Weggenmann, B., Kerschbaum, F.: The limits of word level differential privacy. In: Findings of the Association for Computational Linguistics: NAACL 2022, pp. 867–881. Association for Computational Linguistics, Seattle, United States (2022). https://doi.org/10.18653/v1/2022.findings-naacl.65

  11. Mironov, I.: Rényi differential privacy. In: 2017 IEEE 30th Computer Security Foundations Symposium (CSF). IEEE (2017). https://doi.org/10.1109/csf.2017.11

  12. Murugadoss, K., et al.: Building a best-in-class automated de-identification tool for electronic health records through ensemble learning. medRxiv (2021). https://doi.org/10.1101/2020.12.22.20248270

  13. Noshad, M., Moon, K.R., Sekeh, S.Y., Hero, A.O.: Direct estimation of information divergence using nearest neighbor ratios. In: 2017 IEEE International Symposium on Information Theory (ISIT). IEEE (2017). https://doi.org/10.1109/isit.2017.8006659

  14. Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014). http://www.aclweb.org/anthology/D14-1162

  15. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks (2019)

    Google Scholar 

  16. Shi, W., Shea, R., Chen, S., Zhang, C., Jia, R., Yu, Z.: Just fine-tune twice: selective differential privacy for large language models. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 6327–6340. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates (2022). https://aclanthology.org/2022.emnlp-main.425

  17. Shokri, R., Shmatikov, V.: Privacy-preserving deep learning. In: Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1310–1321. CCS ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2810103.2813687

  18. Voigt, R., Jurgens, D., Prabhakaran, V., Jurafsky, D., Tsvetkov, Y.: RtGender: a corpus for studying differential responses to gender. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan (2018). https://aclanthology.org/L18-1445

  19. Wen, Z., Lu, X.H., Reddy, S.: MeDAL: medical abbreviation disambiguation dataset for natural language understanding pretraining. In: Proceedings of the 3rd Clinical Natural Language Processing Workshop. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.clinicalnlp-1.15

  20. Yue, X., Du, M., Wang, T., Li, Y., Sun, H., Chow, S.S.M.: Differential privacy for text analytics via natural text sanitization. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 3853–3866. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.findings-acl.337

  21. Zhao, X., Li, L., Wang, Y.X.: Provably confidential language modelling. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 943–955. Association for Computational Linguistics, Seattle, United States (2022). https://doi.org/10.18653/v1/2022.naacl-main.69

Download references

Author information

Authors and Affiliations

  1. Trinity College Dublin, Dublin, Ireland

    Vaibhav Gusain & Douglas Leith

Authors
  1. Vaibhav Gusain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Douglas Leith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vaibhav Gusain .

Editor information

Editors and Affiliations

  1. Institut Polytechnique de Paris, Palaiseau, France

    Joaquin Garcia-Alfaro

  2. University of Calgary, Calgary, AB, Canada

    Ken Barker

  3. Universitat Autònoma de Barcelona, Bellaterra, Spain

    Guillermo Navarro-Arribas

  4. Universitat Autònoma de Barcelona, Bellaterra, Spain

    Cristina Pérez-Solà

  5. Chaincode Labs, New York, NY, USA

    Sergi Delgado-Segura

  6. Norwegian University of Science and Technology - NTNU, Gjøvik, Norway

    Sokratis Katsikas

  7. Polytechnique Montréal, Montreal, QC, Canada

    Frédéric Cuppens

  8. University of Piraeus, Piraeus, Greece

    Costas Lambrinoudakis

  9. Polytechnique Montréal, Montreal, QC, Canada

    Nora Cuppens-Boulahia

  10. Bydgoszcz University of Science and Technology, Bydgoszcz, Poland

    Marek Pawlicki

  11. Bydgoszcz University of Science and Technology, Bydgoszcz, Poland

    Michał Choraś

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gusain, V., Leith, D. (2025). Plausible Deniability of Redacted Text. In: Garcia-Alfaro, J., et al. Computer Security. ESORICS 2024 International Workshops. ESORICS 2024. Lecture Notes in Computer Science, vol 15263. Springer, Cham. https://doi.org/10.1007/978-3-031-82349-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-82349-7_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-82348-0

  • Online ISBN: 978-3-031-82349-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics