Yu Huang
ABSTRACT
The baby monitoring system is a kind of smart IoT system helpful for busy working parents. Besides providing basic safety care, it actually also delivers the moments of baby growing and development by all time recording. Hence, in baby monitoring systems nowadays, capturing and recommending the precious moments of infant growth to their parents has become a real-world demand. How to capture the interesting, worthwhile and exquisite moments with commemorative and catering to the parents' preferences is a challenging problem. In this paper, we propose a novel image rating framework, namely Infant growth Precious Moment capturing (IPreMom), which is based on the concept of continual learning for comprehensively addressing the problem of automatically capturing the precious moments during infant growth. Through a series of experiments, it was shown that our proposed framework delivers excellent performance compared to the baselines, with up to 40% improvement in terms of MAE. Moreover, the proposed framework has been implanted into a commercial baby monitor on the market. To the best of our knowledge, this is the first work that solves the problem of automatically capturing the infant precious moments using continual learning and image rating techniques, which is important and has not been well studied in the academic and industrial communities.
- Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. 2017. Expert gate: Lifelong learning with a network of experts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3366--3375.Google Scholar
Cross Ref
- Arslan Chaudhry, Marc'Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. 2018. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00420 (2018).Google Scholar
- Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. 2018. Superpoint: Self-supervised interest point detection and description. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 224--236.Google ScholarCross Ref
- Mihai Dusmanu, Ignacio Rocco, Tomas Pajdla, Marc Pollefeys, Josef Sivic, Akihiko Torii, and Torsten Sattler. 2019. D2-net: A trainable cnn for joint description and detection of local features. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition. 8092--8101.Google ScholarCross Ref
- Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web. 507--517.Google ScholarDigital Library
- Ruining He and Julian McAuley. 2016. VBPR: visual bayesian personalized ranking from implicit feedback. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.Google ScholarCross Ref
- Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web. 173--182.Google ScholarDigital Library
- Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).Google Scholar
- Waheb A. Jabbar, Hiew Kuet Shang, Saidatul N. I. S. Hamid, Akram A. Almohammedi, Roshahliza M. Ramli, and Mohammed A. H. Ali. 2019. IoT-BBMS: Internet of Things-Based Baby Monitoring System for Smart Cradle. IEEE Access 7 (2019), 93791--93805. Google ScholarCross Ref
- Madhuri P. Joshi and Deepak C. Mehetre. 2017. IoT Based Smart Cradle System with an Android App for Baby Monitoring. In 2017 International Conference on Computing, Communication, Control and Automation (ICCUBEA). 1--4. Google ScholarCross Ref
- Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 197--206.Google ScholarCross Ref
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114, 13 (2017), 3521--3526.Google ScholarCross Ref
- Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8 (2009), 30--37.Google ScholarDigital Library
- Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40, 12 (2017), 2935--2947.Google Scholar
- Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing 7, 1 (2003), 76--80.Google Scholar
- Qiang Liu, Shu Wu, and Liang Wang. 2017. DeepStyle: Learning user preferences for visual recommendation. In Proceedings of the 40th international acm sigir conference on research and development in information retrieval. 841--844.Google ScholarDigital Library
- Yinyang Liu, Xiaobin Xu, and Feixiang Li. 2018. Image feature matching based on deep learning. In 2018 IEEE 4th International Conference on Computer and Communications (ICCC). IEEE, 1752--1756.Google ScholarCross Ref
- Davide Maltoni and Vincenzo Lomonaco. 2019. Continuous learning in single-incremental-task scenarios. Neural Networks 116 (2019), 56--73.Google ScholarDigital Library
- Fei Mi, Xiaoyu Lin, and Boi Faltings. 2020. Ader: Adaptively distilled exemplar replay towards continual learning for session-based recommendation. In Fourteenth ACM Conference on Recommender Systems. 408--413.Google ScholarDigital Library
- Aditya Parameswaran, Petros Venetis, and Hector Garcia-Molina. 2011. Recommendation systems with complex constraints: A course recommendation perspective. ACM Transactions on Information Systems (TOIS) 29, 4 (2011), 1--33.Google ScholarDigital Library
- Shinjee Pyo, Eunhui Kim, et al. 2014. LDA-based unified topic modeling for similar TV user grouping and TV program recommendation. IEEE transactions on cybernetics 45, 8 (2014), 1476--1490.Google Scholar
- Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2001--2010.Google ScholarCross Ref
- Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International conference on data mining. IEEE, 995--1000.Google ScholarDigital Library
- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012. BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618 (2012).Google Scholar
- Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671 (2016).Google Scholar
- Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2000. Application of dimensionality reduction in recommender system-a case study. Technical Report. Minnesota Univ Minneapolis Dept of Computer Science.Google Scholar
- Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. arXiv preprint arXiv:1705.08690 (2017).Google ScholarDigital Library
- Soham Talukdar and Shinjini Saha. 2021. Intelligent Baby Monitoring System Using Blynk. In Advances in Electronics, Communication and Computing, Pradeep Kumar Mallick, Akash Kumar Bhoi, Gyoo-Soo Chae, and Kanak Kalita (Eds.). Springer Singapore, Singapore, 51--64.Google Scholar
- Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning. PMLR, 6105--6114.Google Scholar
- Jiaxi Tang and Ke Wang. 2018. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. 565--573.Google ScholarDigital Library
- Fajie Yuan, Guoxiao Zhang, Alexandros Karatzoglou, Joemon Jose, Beibei Kong, and Yudong Li. 2021. One person, one model, one world: Learning continual user representation without forgetting. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 696--705.Google ScholarDigital Library
- Liang Zhao, Yang Wang, Daxiang Dong, and Hao Tian. 2019. Learning to recommend via meta parameter partition. arXiv preprint arXiv:1912.04108 (2019).Google Scholar
Index Terms
- IPreMom: a continual learning-based image rating framework for infant precious moments capturing
Recommendations
Personalized Recommendation of Socially Relevant Images
WIMS '18: Proceedings of the 8th International Conference on Web Intelligence, Mining and SemanticsWe present a social image recommender system that offers a hybrid filtering approach, combining content- and knowledge-based filtering with a novel social-based filtering, that selects images of social interest to the user, by e.g. being posted by close ...
Multimodal collaborative graph for image recommendation
AbstractRecent works for personalized recommendation typically emphasize their efforts on learning users’ interests from interactions. However, users make decisions depending on multiple factors, especially various attributes of items like appearance, ...
QRGQR: Query Relevance Graph for Query Recommendation
TENSYMP '15: Proceedings of the 2015 IEEE Region 10 SymposiumQuery recommendation is an important feature of the search engine with the explosive and diverse growth of web contents. Different kind of recommendation like query, image, movies, music and book etc. Are used every day. Various types of data sources ...
Comments