Skip to main content
SpringerLink
Log in

User Study of the Effectiveness of a Privacy Policy Summarization Tool

  • Conference paper
  • First Online:
Information Systems Security and Privacy (ICISSP 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1221))

Included in the following conference series:

Abstract

The complexity of privacy policies makes it difficult for users to understand its content. In order to solve this, tools exist that analyze and summarize those privacy policies, and present the results in a standardized visual format. The use of these tools can make it possible to analyze any privacy policy, that is, they have the advantage of scale, unlike processes that require manual classification. However, there is scarce research on their effectiveness and how users perceive them. In this paper, an experimental survey was conducted to evaluate whether one such tool, PrivacyGuide, could communicate risk and increase interest in the content of the privacy policy itself. The survey was conducted in Japan with Japanese participants, and considered two languages of the privacy policy, Japanese and English. The results show that interest in the privacy policy increased after viewing the privacy policy summary. On the other hand, risk communication was limited to the case of an English language privacy policy. In addition, survey participants also provided positive and negative feedback about the tool: there was interest in using the tool in a variety of scenarios, but there was also lack of trust in the results. The findings suggest that privacy policy summarization tools have potential to help users, but that there are barriers for adoption of the tool.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The explanation of the meaning of each of these privacy aspects is detailed in [20].

References

  1. Alexa: Top Sites in Japan. https://www.alexa.com/topsites/countries/JP

  2. Benjamini, Y., Hochberg, Y.: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B (Methodol.) 57(1), 289–300 (1995)

    MathSciNet  MATH  Google Scholar 

  3. Bracamonte, V., Hidano, S., Tesfay, W.B., Kiyomoto, S.: Evaluating privacy policy summarization: an experimental study among Japanese users. In: Proceedings of the 5th International Conference on Information Systems Security and Privacy, ICISSP, vol. 1, pp. 370–377. INSTICC, SciTePress (2019). https://doi.org/10.5220/0007378403700377

  4. Curran, P.J., West, S.G., Finch, J.F.: The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol. Methods 1(1), 16–29 (1996)

    Article  Google Scholar 

  5. European Parliament: Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46 (2016)

    Google Scholar 

  6. Gideon, J., Cranor, L., Egelman, S., Acquisti, A.: Power strips, prophylactics, and privacy, oh my! In: Proceedings of the Second Symposium on Usable Privacy and Security, SOUPS 2006, pp. 133–144. ACM (2006). https://doi.org/10.1145/1143120.1143137

  7. Gluck, J., et al.: How short is too short? Implications of length and framing on the effectiveness of privacy notices. In: Twelfth Symposium on Usable Privacy and Security (SOUPS 2016), pp. 321–340. USENIX Association, Denver (2016)

    Google Scholar 

  8. Harkous, H., Fawaz, K., Lebret, R., Schaub, F., Shin, K.G., Aberer, K.: Polisis: automated analysis and presentation of privacy policies using deep learning. In: 27th USENIX Security Symposium (USENIX Security 18), pp. 531–548. USENIX Association, Baltimore (2018)

    Google Scholar 

  9. Kelley, P.G., Cesca, L., Bresee, J., Cranor, L.F.: Standardizing privacy notices: an online study of the nutrition label approach. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2010, pp. 1573–1582. ACM, New York (2010). https://doi.org/10.1145/1753326.1753561

  10. Kim, D.J., Ferrin, D.L., Rao, H.R.: A trust-based consumer decision-making model in electronic commerce: the role of trust, perceived risk, and their antecedents. Decis. Support Syst. 44(2), 544–564 (2008). https://doi.org/10.1016/j.dss.2007.07.001

    Article  Google Scholar 

  11. Kizilcec, R.F.: How much information?: Effects of transparency on trust in an algorithmic interface. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI 2016, pp. 2390–2395. ACM, New York (2016). https://doi.org/10.1145/2858036.2858402

  12. Kline, R.B.: Principles and Practice of Structural Equation Modeling, 2nd edn. Guilford Press, New York (2005)

    MATH  Google Scholar 

  13. Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors 46(1), 50–80 (2004). https://doi.org/10.1518/hfes.46.1.50_30392

    Article  Google Scholar 

  14. McDonald, A.M., Cranor, L.F.: The cost of reading privacy policies. I/S: J. Law Policy Inf. Soc. 4, 543 (2008)

    Google Scholar 

  15. Proctor, R.W., Ali, M.A., Vu, K.P.L.: Examining usability of web privacy policies. Int. J. Hum.-Comput. Interact. 24(3), 307–328 (2008). https://doi.org/10.1080/10447310801937999

    Article  Google Scholar 

  16. Rosseel, Y.: Lavaan: an R package for structural equation modeling. J. Stat. Softw. 48(2), 1–36 (2012). https://doi.org/10.18637/jss.v048.i02

    Article  Google Scholar 

  17. Statistics Bureau, Ministry of Internal Affairs and Communications: Population and Households of Japan 2010. Tech. rep

    Google Scholar 

  18. Steinfeld, N.: “I agree to the terms and conditions”: (How) do users read privacy policies online? An eye-tracking experiment. Comput. Hum. Behav. 55, 992–1000 (2016). https://doi.org/10.1016/j.chb.2015.09.038

    Article  Google Scholar 

  19. Sunyaev, A., Dehling, T., Taylor, P.L., Mandl, K.D.: Availability and quality of mobile health app privacy policies. J. Am. Med. Inform. Assoc. 22(e1), e28–e33 (2014). https://doi.org/10.1136/amiajnl-2013-002605

    Article  Google Scholar 

  20. Tesfay, W.B., Hofmann, P., Nakamura, T., Kiyomoto, S., Serna, J.: PrivacyGuide: towards an implementation of the EU GDPR on internet privacy policy evaluation. In: Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics, IWSPA 2018, pp. 15–21. ACM, New York (2018). https://doi.org/10.1145/3180445.3180447

  21. ToSDR: Terms of Service; Didn’t Read(2019). https://tosdr.org/

  22. Zaeem, R.N., German, R.L., Barber, K.S.: PrivacyCheck: automatic summarization of privacy policies using data mining. ACM Trans. Internet Technol. 18(4), 53:1–53:18 (2018). https://doi.org/10.1145/3127519

    Article  Google Scholar 

  23. Zimmeck, S., Bellovin, S.M.: Privee: an architecture for automatically analyzing web privacy policies. In: 23rd USENIX Security Symposium (USENIX Security 2014), pp. 1–16. USENIX Association, San Diego (August 2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. KDDI Research, Inc., Saitama, Japan

    Vanessa Bracamonte, Seira Hidano & Shinsaku Kiyomoto

  2. Goethe University Frankfurt, Frankfurt, Germany

    Welderufael B. Tesfay

Authors
  1. Vanessa Bracamonte
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seira Hidano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Welderufael B. Tesfay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shinsaku Kiyomoto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vanessa Bracamonte .

Editor information

Editors and Affiliations

  1. IIT-CNR, Pisa, Italy

    Paolo Mori

  2. Plymouth University, Plymouth, UK

    Steven Furnell

  3. MODESTE/ESEO, Angers Cedex 2, France

    Olivier Camp

Appendix

Appendix

(See Figs. 3, 4 and Tables 2, 3).

Fig. 3.
figure 3

Experiment website registration forms. Left: registration form showing an English privacy policy. Right: registration form showing a Japanese language privacy policy.

Full size image
Fig. 4.
figure 4

PrivacyGuide result screens. Top: higher risk privacy policy result. Bottom: lower risk privacy policy result.

Full size image
Table 2. English text of the information presented in the PrivacyGuide result screen for the high and low risk privacy policies.
Full size table
Table 3. Measurement items. Items adapted from [10].
Full size table
Table 4. (continued)
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bracamonte, V., Hidano, S., Tesfay, W.B., Kiyomoto, S. (2020). User Study of the Effectiveness of a Privacy Policy Summarization Tool. In: Mori, P., Furnell, S., Camp, O. (eds) Information Systems Security and Privacy. ICISSP 2019. Communications in Computer and Information Science, vol 1221. Springer, Cham. https://doi.org/10.1007/978-3-030-49443-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49443-8_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49442-1

  • Online ISBN: 978-3-030-49443-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation