skip to main content
10.1145/3508398.3511517acmconferencesArticle/Chapter ViewAbstractPublication PagescodaspyConference Proceedingsconference-collections
research-article
Public Access

Towards Automated Content-based Photo Privacy Control in User-Centered Social Networks

Published: 15 April 2022 Publication History

Abstract

A large number of photos shared online often contain private user information, which can cause serious privacy breaches when viewed by unauthorized users. Thus, there is a need for more efficient privacy control that requires automatic detection of users' private photos. However, the automatic detection of users' private photos is a challenging task, since different users may have different privacy concerns and a generalized one-size-fits-all approach for private photo detection would not be suitable for most users. User-specific detection of private photos should, therefore, be investigated. Furthermore, for effective privacy control, the exact sensitive regions in private photos need to be pinpointed, so that sensitive content can be protected via different privacy control methods. In this paper, we propose a novel system, AutoPri, to enable automatic and user-specific content-based photo privacy control in online social networks. We collect a large dataset of 31, 566 private and public photos from real-world users and present important observations on photo privacy concerns. Our system can automatically detect private photos in a user-specific manner using a detection model based on a multimodal variational autoencoder and pinpoint sensitive regions in private photos with an explainable deep learning-based approach. Our evaluations show that AutoPri can effectively determine user-specific private photos with high accuracy (94.32%) and pinpoint exact sensitive regions in them to enable effective privacy control in user-centered online social networks.

References

[1]
2011. Teacher sacked for posting picture of herself holding glass of wine and mug of beer on Facebook . https://www.dailymail.co.uk/news/article-1354515/Teacher-sacked-posting-picture-holding-glass-wine-mug-beer-Facebook.html.
[2]
2016. Social Media Update 2016 - Pew Research Center . https://www.pewresearch.org/internet/2016/11/11/social-media-update-2016/.
[3]
2017. Bathroom Selfie Phenomenon . https://www.theguardian.com/media/shortcuts/2017/ jan/24/finished-in-there-yet-how-bathroom-selfie-became-huge.
[4]
2020 a. Face Plus Plus . http://www.faceplusplus.com .
[5]
2020 b. Facebook . https://www.facebook.com .
[6]
2020. Instagram . https://www.instagram.com/.
[7]
2020. YOLO . https://pjreddie.com/darknet/yolo/.
[8]
Ramakrishna Ayyagari, Varun Grover, and Russell Purvis. 2011. Technostress: technological antecedents and implications. MIS quarterly, Vol. 35, 4 (2011), 831--858.
[9]
Andrew Besmer and Heather Richter Lipford. 2010. Moving beyond untagging: photo privacy in a tagged world. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1563--1572.
[10]
YouTube Official Blog. 2012. Face blurring: when footage requires anonymity. Blog.(18 July 2012). Retrieved April, Vol. 13 (2012), 2017.
[11]
Alex Braunstein, Laura Granka, and Jessica Staddon. 2011. Indirect content privacy surveys: measuring privacy without asking about it. In Proceedings of the Seventh Symposium on Usable Privacy and Security. 1--14.
[12]
Kelly Erinn Caine. 2009a. Exploring everyday privacy behaviors and misclosures . Ph.,D. Dissertation. Georgia Institute of Technology.
[13]
Kelly E Caine. 2009b. Supporting privacy by preventing misclosure. In CHI'09 Extended Abstracts on Human Factors in Computing Systems. ACM, 3145--3148.
[14]
Rich Caruana and Alexandru Niculescu-Mizil. 2006. An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd international conference on Machine learning. 161--168.
[15]
Leucio Antonio Cutillo, Refik Molva, and Melek Önen. 2012. Privacy preserving picture sharing: Enforcing usage control in distributed on-line social networks. In Proceedings of the Fifth Workshop on Social Network Systems. ACM, 6.
[16]
Maeve Duggan. 2013. Photo and video sharing grow online. Pew Research Internet Project (2013).
[17]
Tom Fawcett. 2006. An introduction to ROC analysis. Pattern recognition letters, Vol. 27, 8 (2006), 861--874.
[18]
Andrea Frome, German Cheung, Ahmad Abdulkader, Marco Zennaro, Bo Wu, Alessandro Bissacco, Hartwig Adam, Hartmut Neven, and Luc Vincent. 2009. Large-scale privacy protection in google street view. In 2009 IEEE 12th international conference on computer vision. IEEE, 2373--2380.
[19]
Wenbo Guo, Dongliang Mu, Jun Xu, Purui Su, Gang Wang, and Xinyu Xing. 2018. LEMNA: Explaining Deep Learning based Security Applications. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, 364--379.
[20]
Rakibul Hasan, Eman Hassan, Yifang Li, Kelly Caine, David J Crandall, Roberto Hoyle, and Apu Kapadia. 2018. Viewer experience of obscuring scene elements in photos to enhance privacy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 47.
[21]
Rakibul Hasan, Patrick Shaffer, David Crandall, Eman T Apu Kapadia, et almbox. 2017. Cartooning for enhanced privacy in lifelogging and streaming videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops . 29--38.
[22]
Jianping He, Bin Liu, Deguang Kong, Xuan Bao, Na Wang, Hongxia Jin, and George Kesidis. 2016. Puppies: Transformation-supported personalized privacy preserving partial image sharing. In 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). IEEE, 359--370.
[23]
Geoffrey E Hinton. 1999. Products of experts. (1999).
[24]
Roberto Hoyle, Robert Templeman, Steven Armes, Denise Anthony, David Crandall, and Apu Kapadia. 2014. Privacy behaviors of lifeloggers using wearable cameras. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 571--582.
[25]
Henry Hsu and Peter A Lachenbruch. 2014. Paired t test. Wiley StatsRef: Statistics Reference Online (2014).
[26]
Hongxin Hu, Gail-Joon Ahn, and Jan Jorgensen. 2013. Multiparty access control for online social networks: model and mechanisms. IEEE Transactions on Knowledge and Data Engineering, Vol. 25, 7 (2013), 1614--1627.
[27]
Panagiotis Ilia, Iasonas Polakis, Elias Athanasopoulos, Federico Maggi, and Sotiris Ioannidis. 2015. Face/off: Preventing privacy leakage from photos in social networks. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM, 781--792.
[28]
Katarzyna Janocha and Wojciech Marian Czarnecki. 2017. On loss functions for deep neural networks in classification. arXiv preprint arXiv:1702.05659 (2017).
[29]
Sanjay Kairam, Joseph'Jofish' Kaye, John Alexis Guerra-Gomez, and David A Shamma. 2016. Snap decisions?: How users, content, and aesthetics interact to shape photo sharing behaviors. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 113--124.
[30]
Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).
[31]
Thomas R Knapp. 1990. Treating ordinal scales as interval scales: an attempt to resolve the controversy. Nursing research, Vol. 39, 2 (1990), 121--123.
[32]
Balachander Krishnamurthy and Craig E Wills. 2009. On the leakage of personally identifiable information via online social networks. In Proceedings of the 2nd ACM workshop on Online social networks. ACM, 7--12.
[33]
Yifang Li, Nishant Vishwamitra, Bart P Knijnenburg, Hongxin Hu, and Kelly Caine. 2017. Effectiveness and users' experience of obfuscation as a privacy-enhancing technology for sharing photos. Manuscript submitted for publication, Vol. 4 (2017).
[34]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740--755.
[35]
Erika McCallister, Timothy Grance, and Karen A Scarfone. 2010. Guide to protecting the confidentiality of personally identifiable information (PII). Special Publication (NIST SP)-800--122 (2010).
[36]
Andrew D Miller and W Keith Edwards. 2007. Give and take: a study of consumer photo-sharing culture and practice. In Proceedings of the SIGCHI conference on Human factors in computing systems. Acm, 347--356.
[37]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
[38]
Prajit Ramachandran, Barret Zoph, and Quoc V Le. 2017. Searching for activation functions. arXiv preprint arXiv:1710.05941 (2017).
[39]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision . 618--626.
[40]
Anna Cinzia Squicciarini, Mohamed Shehab, and Federica Paci. 2009. Collective privacy management in social networks. In Proceedings of the 18th international conference on World wide web. ACM, 521--530.
[41]
Anna Cinzia Squicciarini, Smitha Sundareswaran, Dan Lin, and Josh Wede. 2011. A3p: adaptive policy prediction for shared images over popular content sharing sites. In Proceedings of the 22nd ACM conference on Hypertext and hypermedia. ACM, 261--270.
[42]
Kimia Tajik, Akshith Gunasekaran, Rhea Dutta, Brandon Ellis, Rakesh B Bobba, Mike Rosulek, Charles V Wright, and Wu-chi Feng. 2019. Balancing Image Privacy and Usability with Thumbnail-Preserving Encryption. IACR Cryptology ePrint Archive, Vol. 2019 (2019), 295.
[43]
Shan-Tair Wang, Mei-Lin Yu, Chi-Jen Wang, and Chao-Ching Huang. 1999. Bridging the gap between the pros and cons in treating ordinal scales as interval scales from an analysis point of view. Nursing research, Vol. 48, 4 (1999), 226--229.
[44]
Mike Wu and Noah Goodman. 2018. Multimodal generative models for scalable weakly-supervised learning. In Advances in Neural Information Processing Systems. 5575--5585.
[45]
Jun Yu, Baopeng Zhang, Zhengzhong Kuang, Dan Lin, and Jianping Fan. 2017. iPrivacy: image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Transactions on Information Forensics and Security, Vol. 12, 5 (2017), 1005--1016.
[46]
S Zerr, JH Stefan Siersdorfer, and E Demidova. 2012. Picalert! data set.
[47]
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition . 2921--2929.

Cited By

View all
  • (2024)Let me quickly share it - Time Pressure when Sharing on Social MediaProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701578(280-299)Online publication date: 1-Dec-2024
  • (2024)Feminist Interaction Techniques: Social Consent Signals to Deter NCIM ScreenshotsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676380(1-14)Online publication date: 13-Oct-2024
  • (2024)DIPA2Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314397:4(1-30)Online publication date: 12-Jan-2024
  • Show More Cited By

Index Terms

  1. Towards Automated Content-based Photo Privacy Control in User-Centered Social Networks

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      CODASPY '22: Proceedings of the Twelfth ACM Conference on Data and Application Security and Privacy
      April 2022
      392 pages
      ISBN:9781450392204
      DOI:10.1145/3508398
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 April 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. deep learning
      2. privacy control
      3. social media

      Qualifiers

      • Research-article

      Funding Sources

      Conference

      CODASPY '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 149 of 789 submissions, 19%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)209
      • Downloads (Last 6 weeks)18
      Reflects downloads up to 01 Mar 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Let me quickly share it - Time Pressure when Sharing on Social MediaProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701578(280-299)Online publication date: 1-Dec-2024
      • (2024)Feminist Interaction Techniques: Social Consent Signals to Deter NCIM ScreenshotsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676380(1-14)Online publication date: 13-Oct-2024
      • (2024)DIPA2Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314397:4(1-30)Online publication date: 12-Jan-2024
      • (2024)Secure Cloud Album: Design and ImplementationFrontiers in Cyber Security10.1007/978-981-96-0151-6_13(199-216)Online publication date: 27-Dec-2024
      • (2023)Analyzing the Use of Large Language Models for Content Moderation with ChatGPT ExamplesProceedings of the 3rd International Workshop on Open Challenges in Online Social Networks10.1145/3599696.3612895(1-8)Online publication date: 4-Sep-2023
      • (2023)Relationship privacy preservation in photo sharingOnline Social Networks and Media10.1016/j.osnem.2023.10026837-38(100268)Online publication date: Sep-2023
      • (2023)Social Honeypot for Humans: Luring People Through Self-managed Instagram PagesApplied Cryptography and Network Security10.1007/978-3-031-33488-7_12(309-336)Online publication date: 19-Jun-2023
      • (2022)Learning and Preserving Relationship Privacy in Photo Sharing2022 IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT)10.1109/BDCAT56447.2022.00029(170-173)Online publication date: Dec-2022

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media