Abstract
The Active-Matrix Organic Light-Emitting Diodes (AMOLED) technology has become the mainstream of displays in recent years. However, it will generate a lot of power consumption on AMOLED displays when displaying high-brightness content. To address this problem, an exposure correction mechanism is needed to remove high-brightness ambient light in the image. This work proposes a Power-Constrained Exposure Correction (PCEC) network based on adversarial learning. The PCEC network utilizes a U-Net-based generator with self-regularized high-exposure attention and adopts the global-local discriminator architecture for adversarial learning. To reduce the intensity of the high-exposed regions while constraining the AMOLED display’s power consumption, we include a power-constraint algorithm in the generator. The experimental results on three different datasets show that the proposed method can effectively correct high-exposed regions and achieve an average of 22.69% power-saving rate in the high-quality mode and 68.71% in the high-efficiency mode. The proposed method achieves 82.4 milliseconds of average inference time when run on a mobile device. Furthermore, the proposed method can enhance the saturation and contrast of the image and provide better visual quality than the existing power-constrained over-exposure correction methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282
Ambalathankandy P, Ikebe M, Yoshida T, Shimada T, Takamaeda S, Motomura M, Asai T (2019) An adaptive global and local tone mapping algorithm implemented on FPGA. IEEE Trans Circuits Syst Video Technol 30(9):3015–3028
Arbeláez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916
Bhandari AK (2020) A logarithmic law based histogram modification scheme for naturalness image contrast enhancement. J Ambient Intell Humaniz Comput 11(4):1605–1627
Carroll A, Heiser G (2010) An analysis of power consumption in a smartphone. In: USENIX Annual Technical Conference
Chang T, Xu SS, Su S (2016) Ssim-based quality-on-demand energy-saving schemes for oled displays. IEEE Trans Systems Man Cybern 46(5):623–635
Chen X, Chen Y, Chun Jason Xue (2015) Datum: Dynamic tone mapping technique for oled display power saving based on video classification. In: 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC), pp 1–6
Chen C, Chen Q, Xu J, Koltun V (2018a) Learning to see in the dark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3291–3300
Chen Y, Wang Y, Kao M, Chuang Y (2018b) Deep photo enhancer: unpaired learning for image enhancement from photographs with gans. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6306–6314
Chondro P, Ruan SJ (2016) Perceptually hue-oriented power-saving scheme with overexposure corrector for amoled displays. J Disp Technol 12(8):791–800
Chondro P, Chang C, Ruan S, Shen C (2018a) Advanced multimedia power-saving method using a dynamic pixel dimmer on amoled displays. IEEE Trans Circuits Syst Video Technol 28(9):2200–2209
Chondro P, Yao ZR, Ruan SJ (2018b) Depth-based dynamic lightness adjustment power-saving algorithm for amoled in head-mounted display. Opt Express 26(25):33158–33165
Dong M, Zhong L (2011) Chameleon: A color-adaptive web browser for mobile oled displays. In: Proceedings of the 9th International Conference on Mobile Systems, Applications, and Services, pp 85–98
Everingham M, Gool LV, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88:303–38
Gu K, Tao D, Qiao J, Lin W (2018) Learning a no-reference quality assessment model of enhanced images with big data. IEEE Trans Neural Netw Learn Syst 29(4):1301–1313
Guo C, Li C, Guo J, Loy CC, Hou J, Kwong S, Cong R (2020) Zero-reference deep curve estimation for low-light image enhancement. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1780–1789
Hu Y, He H, Xu C, Wang B, Lin S (2018) Exposure: a white-box photo post-processing framework. ACM Trans Graph (TOG) 37(2):26
Ignatov A, Kobyshev N, Timofte R, Vanhoey K, Van Gool L (2017) Dslr-quality photos on mobile devices with deep convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp 3277–3285
Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1125–1134
Jiang Y, Gong X, Liu D, Cheng Y, Fang C, Shen X, Yang J, Zhou P, Wang Z (2021) Enlightengan: deep light enhancement without paired supervision. IEEE Trans Image Process 30:2340–2349
Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. Eur Conf Comput Vis 9906:694–711. https://doi.org/10.1007/978-3-319-46475-6_43
Kang S (2016) Perceptual quality-aware power reduction technique for organic light emitting diodes. J Displ Technol 12(6):519–525
Kang C, Lin C, Hsiu P (2015) A win-win camera: Quality-enhanced power-saving images on mobile oled displays. In: 2015 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), pp 267–272
Kim S, Kwon HJ, Lee S, Shim H, Chun Y, Choi W, Kwack J, Han D, Song M, Kim S et al (2011) Low-power flexible organic light-emitting diode display device. Adv Mater 23(31):3511–3516
Kim S, Hyun S, Heo T, Im D, Huh J (2016) Blind: Power saving color transform method for oled displays. In: 2016 IEEE International Conference on Consumer Electronics (ICCE), pp 500–501
Kosugi S, Yamasaki T (2020) Unpaired image enhancement featuring reinforcement-learning-controlled image editing software. Proc AAAI Conf Artif Intell 34:11296–11303
Li H, Gao H, Kirca G, Lei Z (2020) Energy-saving display by color pixel re-representation. IEEE Trans Circuits Syst Video Technol 30(12):4728–4738
Mandal G, Bhattacharya D, De P (2021) Real-time fast low-light vision enhancement for driver during driving at night. J Ambient Intell Humaniz Comput 13:789–798
Mao X, Li Q, Xie H, Lau RY, Wang Z, Paul Smolley S (2017) Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2794–2802
Ponomarenko N, Lukin V, Zelensky A, Egiazarian K, Carli M, Battisti F (2009) TID2008: a database for evaluation of full-reference visual quality assessment metrics. Adv Mod Radioelectron 10:30–45
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Int Conf Med Image Comput Comput-Assist Interv 9351:234–241
Sakuishi T, Hatsumi R, Chida A, Yamashita A, Nakano M, Ishitani T, Sasaki T, Kusunoki K, Kawashima S, Seo S, Hirakata Y, Bergquist J, Yamazaki S, Nakada M, Idojiri S, Adachi H (2016) Transmissive oled and reflective LC hybrid (TR-hybrid) display with high visibility and low power consumption. SID Symp Dig Tech Pap 47(1):735–738. https://doi.org/10.1002/sdtp.10739
Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451
Shin Y, Park S, Yeo Y, Yoo M, Ko S (2020) Unsupervised deep contrast enhancement with power constraint for oled displays. IEEE Trans Image Process 29:2834–2844
Vinker Y, Huberman-Spiegelglas I, Fattal R (2021) Unpaired learning for high dynamic range image tone mapping. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp 14657–14666
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Yang X, Xu K, Song Y, Zhang Q, Wei X, Lau RW (2018) Image correction via deep reciprocating hdr transformation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1798–1807
Yeh CH, Lo KSH, Lin W (2018) Visual-quality guided global backlight dimming for video display on mobile devices. IEEE Trans Circuits Syst Video Technol 29(11):3393–3403
Funding
This work was supported by the Ministry of Science and Technology, Taiwan. [grant number: MOST 109-2221-E-011 -122 -MY3].
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chou, YY., Haq, M.A., Ruan, SJ. et al. Power constrained exposure correction network for mobile devices. J Ambient Intell Human Comput 14, 9021–9033 (2023). https://doi.org/10.1007/s12652-022-04405-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-022-04405-8