Skip to main content

Advertisement

Springer Nature Link
Log in

Privacy protection and beautification of cornea images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Thanks to the technological advances, social media has become more popular year by year, especially when it is common to upload selfies to the Internet where anyone from anywhere can have access to them, thus leading to some privacy issue. More specifically, when a selfie photo is relatively clear and bright, there could be a high probability of revealing a person’s locations and/or some associated information. In this paper, a framework is designed to automatically obtain the cornea information. First, the Haar Cascade algorithm is applied on the captured eye area and a find-tuned YOLO object detector is then used for iris localization. Next, an image calibration is performed to get a more accurate identification. Furthermore, image super resolution and denoising are applied to boost the image quality. Finally, Google Vision API is used for object detection. Experimental results indicate that certain privacy information could be obtained from a photo via aforementioned processes, especially when some person information can be identified. More specifically, for certain phones when the image capturing distance is around 50 cm, the probability to identify a person (from Google Vision API) can be as high as 60%. Although lowering image quality could help reduce the risk of privacy exposure, it could make the photo undesirably blur. To address these issues, a novel method is proposed to remove sensitive privacy information while at the same time being able to produce eye-stunning images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29

Similar content being viewed by others

References

  1. Amazon Ring light product. https://www.amazon.com/ring-light/s?k=ring+light. published by Amazon, Accessed 16 Dec 2020

  2. Boyle M, Edwards C, Greenberg S (2000) The effects of filtered video on awareness and privacy, pp 1–10

  3. Daugman J (2007) New methods in iris recognition. IEEE Trans Syst Man Cybern Part B (Cybern) 37(5):1167–1175. https://doi.org/10.1109/TSMCB.2007.903540

    Article  Google Scholar 

  4. Dong C, Loy C C, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38 (2):295–307. https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  5. Dong C, Loy C C, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Leibe B, Matas J, Sebe N, Welling M (eds) Proceedings of European Conference on Computer Vision (ECCV). Springer International Publishing, Cham, pp 391–407

  6. Freund Y, Schapire R E (1995) A desicion-theoretic generalization of on-line learning and an application to boosting. In: Vitányi P (ed) Computational Learning Theory. Springer, Berlin, pp 23–37

  7. Gafni O, Wolf L, Taigman Y (2019) Live face de-identification in video, pp 9377–9386

  8. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Computer Vision and Pattern Recognition

  9. Google Google vision. https://cloud.google.com/vision/. Accessed 16 Dec 2020

  10. Gross R, Airoldi E, Malin B, Sweeney L (2006) Integrating utility into face de-identification. In: Danezis G, Martin D (eds) Privacy Enhancing Technologies. Springer, Berlin, pp 227–242

  11. Gross R, Sweeney L, Cohn J, De la Torre F, Baker S (2009) Face de-identification, pp 129–146

  12. Grotta S W, Grotta D (2012) Not all pixels are created equal [tools toys]. IEEE Spectr 49(5):22–24. https://doi.org/10.1109/MSPEC.2012.6189568

    Article  Google Scholar 

  13. He K, Gkioxari G, Dollar P, Girshick R (2017) Mask r-cnn. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp 2980–2988

  14. Jenkins R, Kerr C (2013) Identifiable images of bystanders extracted from corneal reflections. PLOS One 8(12):1–5. https://doi.org/10.1371/journal.pone.0083325

    Article  Google Scholar 

  15. Kaufman P L, Alm A Adler’s physiology of the eye: clinical application. mosby, tenth edition. J Neuroophthalmol 24:4 348

  16. Kelby S (2010) The digital photography book, vol 3. Peachpit Press

  17. Kerr D A (2007) Apex-the additive system of photographic exposure. http://dougkerr.net/Pumpkin/articles/APEX.pdf. Accessed 16 Dec 2020

  18. Kim J, Lee J K, Lee K M (2016) Deeply-recursive convolutional network for image super-resolution. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1637–1645

  19. Kim J, Lee J K, Lee K M (2016) Accurate image super-resolution using very deep convolutional networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR Oral)

  20. Koh J, Govindaraju V, Chaudhary V (2010) A robust iris localization method using an active contour model and hough transform. In: 2010 20th International Conference on Pattern Recognition, pp 2852–2856

  21. Kumar V, Asati A, Gupta A (2015) Iris localization based on integro-differential operator for unconstrained infrared iris images. In: 2015 International Conference on Signal Processing, Computing and Control (ISPCC), pp 277–281

  22. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 105–114

  23. Lienhart R, Maydt J (2002) An extended set of haar-like features for rapid object detection. In: Proceedings. international conference on image processing, vol 1

  24. Lim B, Son S, Kim H, Nah S, Lee K M (2017) Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1132–1140

  25. Lin J, Li Y, Yang G (2021) Fpgan: Face de-identification method with generative adversarial networks for social robots. Neural Netw 133:132–147. https://doi.org/10.1016/j.neunet.2020.09.001

    Article  Google Scholar 

  26. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg A C (2016) Ssd: Single shot multibox detector. Lect Notes Comput Sci 9905:21–37. https://doi.org/10.1007/978-3-319-46448-0_2

    Article  Google Scholar 

  27. Meng L, Sun Z, Collado O T (2017) An efficient approach to de-identifying faces in videos. IET Signal Process 11(9):1039–1045. https://doi.org/10.1049/iet-spr.2016.0761

    Article  Google Scholar 

  28. Meng L, Sun Z (2014) Face de-identification with perfect privacy protection. In: 2014 37th International convention on information and communication technology, electronics and microelectronics (MIPRO), pp 1234–1239

  29. Menon S, Damian A, Hu S, Ravi N, Rudin C (2020) Pulse: Self-supervised photo upsampling via latent space exploration of generative models.. In: CVPR. IEEE, pp 2434–2442

  30. Mobile phone reviews, news, specifications and more. https://www.gsmarena.com/. Accessed 16 Dec 2020

  31. Mundhenk T N, Rivett M J, Liao X, Hall E L (2000) Techniques for fisheye lens calibration using a minimal number of measurements. In: Casasent D P (ed) Intelligent Robots and Computer Vision XIX: Algorithms, Techniques, and Active Vision, vol 4197. SPIE, pp 181–190, DOI https://doi.org/10.1117/12.403762

  32. Pech-Pacheco J L, Cristobal G, Chamorro-Martinez J, Fernandez-Valdivia J (2000) Diatom autofocusing in brightfield microscopy: a comparative study. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol 3, pp 314–317 vol.3

  33. Proenca H, Alexandre L A (2005) Ubiris: A noisy iris image database. In: Proceed. of ICIAP 2005 - Intern. confer. on image analysis and processing, vol 1, pp 970–977

  34. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 779–788

  35. Redmon J (2020) darknet. https://github.com/pjreddie/darknet. Accessed 16 Dec 2020

  36. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. In: Cortes C, Lawrence N D, Lee D D, Sugiyama M, Garnett R (eds) Advances in Neural Information Processing Systems 28. http://papers.nips.cc/paper/5638-faster-r-cnn-towards-real-time-object-detection-with-region-proposal-networks.pdf. Curran Associates, Inc., pp 91–99

  37. Ribaric S, Pavesic N (2015) An overview of face de-identification in still images and videos. In: 2015 11th IEEE International conference and workshops on automatic face and gesture recognition (FG), vol 04, pp 1–6

  38. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol 1, pp I–I

  39. Wildes R P (1997) Iris recognition: an emerging biometric technology. Proc IEEE 85(9):1348–1363. https://doi.org/10.1109/5.628669

    Article  Google Scholar 

  40. Zhang K, Zuo W, Zhang L (2018) Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Trans Image Process 27(9):4608–4622. https://doi.org/10.1109/TIP.2018.2839891

    Article  MathSciNet  Google Scholar 

  41. Zhang X-D, Dong X-P, Wang Q, Sun Y-R (2011) Location algorithm of rgb iris image based on integro-differential operators. Dongbei Daxue Xuebao/J Northeastern Univ 32:1550–1553

    Google Scholar 

  42. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Proceedings of European Conference on Computer Vision (ECCV). Springer International Publishing, Cham, pp 294–310

  43. Zhang Z, Sun R, Zhao C, Wang J, Chang C K, Gupta B B (2017) Cyvod: a novel trinity multimedia social network scheme. Multimed Tools Appl 76:18513–18529

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Management, National Taiwan University of Science and Technology, No. 43, Sec. 4, Keelung Road Taipei, 106, Taiwan, ROC

    Chia-Lin Wu, Chuan-Kai Yang & Yi-Ling Lin

Authors
  1. Chia-Lin Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Chuan-Kai Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yi-Ling Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Chuan-Kai Yang.

Ethics declarations

This work was supported in part by the Ministry of Science and Technology of Taiwan under the grants MOST 109-2221-E-011-133 and MOST 109-2228-E-011-007.

Conflict of Interests

Both authors have received the aforementioned funding support and both authors have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, CL., Yang, CK. & Lin, YL. Privacy protection and beautification of cornea images. Multimed Tools Appl 81, 32421–32448 (2022). https://doi.org/10.1007/s11042-022-12530-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12530-x

Keywords