Skip to main content
Log in

Mass Prediction and Analysis of an Object’s Mass from Its Image Using Deep Learning

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

The world is full of objects with different properties, such as volume, density, mass, material, etc. These properties are crucial for meaningful interaction with these objects. However, none of these properties is explicitly known. As humans rely on perception to interact with the world, robots need to understand the properties of objects to function effectively. This includes inferring properties that may not be explicitly stated, such as an object’s mass. A single robot might not be sufficient to execute complex tasks in dynamic environments. In such cases, an agent can assemble a team of robots at runtime to complete the task. The agent must estimate the object’s mass to make this decision effectively. For example, if a team of robots needs to move a heavy object, accurately estimating its mass is essential. This information allows the agent to determine the appropriate team size, ensuring enough collective strength to complete the task successfully. In this work, we describe techniques that enable robots to learn to fill these knowledge gaps (missing properties) before attempting to interact with these objects. The salient features of our work lie in our observation that the properties of an object are not entirely independent of each other, and learning to predict/infer one can give clues regarding predicting/inferring the other. Our evaluation indicates that the proposed solutions outperform the existing state-of-the-art method. We also simulated estimating the object’s properties in the ROS environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The dataset used for this reserach is available at the following link. https://github.com/tstandley/image2mass.

Notes

  1. https://github.com/penguinnnnn/Caffe2Pytorch.

References

  1. Agmon N, Stone P. Leading ad hoc agents in joint action settings with multiple teammates. In: AAMAS; 2012. pp. 341–348.

  2. Aujeszky T, Korres G, Eid M, Khorrami F. Estimating weight of unknown objects using active thermography. Robotics. 2019;8(4):1–13.

    Article  Google Scholar 

  3. Balaban MO, Ünal Şengör GF, Soriano MG, Ruiz EG. Using image analysis to predict the weight of Alaskan salmon of different species. J Food Sci. 2010;75(3):E157–62.

    Article  Google Scholar 

  4. Bell S, Upchurch P, Snavely N, Bala K. Opensurfaces: a richly annotated catalog of surface appearance. ACM Trans Gr. 2013;32(4):1–17.

    Article  Google Scholar 

  5. Bell S, Upchurch P, Snavely N, Bala K. Material recognition in the wild with the materials in context database. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015. pp. 3479–3487.

  6. Bozkurt Y, Aktan S, Ozkaya S. Body weight prediction using digital image analysis for slaughtered beef cattle. J Appl Anim Res. 2007;32(2):195–8.

    Article  Google Scholar 

  7. Cieslak MC, Castelfranco AM, Roncalli V, Lenz PH, Hartline DK. t-distributed stochastic neighbor embedding (t-sne): a tool for eco-physiological transcriptomic analysis. Mar Genom. 2020;51: 100723.

    Article  Google Scholar 

  8. Dohmen R, Catal C, Liu Q. Image-based body mass prediction of heifers using deep neural networks. Biosys Eng. 2021;204:283–93.

    Article  Google Scholar 

  9. Flanagan JR, Vetter P, Johansson RS, Wolpert DM. Prediction precedes control in motor learning. Curr Biol. 2003;13(2):146–50.

    Article  Google Scholar 

  10. Hamdan M, Rover D, Darr M, Just J. Mass estimation from images using deep neural network and sparse ground truth. In: 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA). IEEE; 2019. pp. 1987–1992.

  11. Jiang Y, Lim M, Zheng C, Saxena A. Learning to place new objects in a scene. Int J Robot Res. 2012;31(9):1021–43.

    Article  Google Scholar 

  12. Kingma DP, Ba J. Adam: a method for stochastic optimization. 2014. arXiv preprint arXiv:1412.6980

  13. Konovalov DA, Saleh A, Efremova DB, Domingos JA, Jerry DR (2019) Automatic weight estimation of harvested fish from images. In: 2019 digital image computing: techniques and applications (DICTA). IEEE. p. 1–7.

  14. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012;25:1097–105.

    Google Scholar 

  15. Krotkov E. Robotic perception of material. In: IJCAI. 1995. pp. 88–95.

  16. Le S, Lee M, Fang A. Non-linear image-based regression of body segment parameters. In: 13th International conference on biomedical engineering. Springer; 2009. pp. 2038–2042.

  17. Mavrakis N, Stolkin R. Estimation and exploitation of objects’ inertial parameters in robotic grasping and manipulation: A survey. Robot Auton Syst. 2020;124: 103374.

    Article  Google Scholar 

  18. Nath A, Arun AR, Niyogi R. A distributed approach for road clearance with multi-robot in urban search and rescue environment. Int J Intell Robot Appl. 2019;3(4):392–406.

    Article  Google Scholar 

  19. Nath A, Arun AR, Niyogi R. A distributed approach for autonomous cooperative transportation in a dynamic multi-robot environment. In: Proceedings of the 35th annual ACM symposium on applied Computing; 2020. pp. 792–799.

  20. Nof SY. Handbook of industrial robotics. Oxford: Wiley; 1999.

    Book  Google Scholar 

  21. Patel D, Nath A, Niyogi R. Adding material embedding to the image2mass problem. In: Computational science and its applications—ICCSA 2022 workshops: Malaga, Spain, July 4–7, 2022, Proceedings, Part I. Springer; 2022. pp. 77–90.

  22. Quigley M, Conley K, Gerkey B, Faust J, Foote T, Leibs J, Wheeler R, Ng AY, et al. Ros: an open-source robot operating system. In: ICRA workshop on open source software, Vol. 3. Kobe, Japan; 2009. p. 5.

  23. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014. arXiv preprint arXiv:1409.1556.

  24. Singh SK, Vidyarthi SK, Tiwari R. Machine learnt image processing to predict weight and size of rice kernels. J Food Eng. 2020;274: 109828.

    Article  Google Scholar 

  25. Standley T, Sener O, Chen D, Savarese S. image2mass: estimating the mass of an object from its image. In: Conference on Robot Learning; 2017. pp. 324–333.

  26. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2015. pp. 1–9.

  27. Tadokoro S, Kitano H, Takahashi T, Noda I, Matsubara H, Shinjoh A, Koto T, Takeuchi I, Takahashi H, Matsuno F, et al. The robocup-rescue project: a robotic approach to the disaster mitigation problem. In: Proceedings ICRA, Vol. 4. IEEE; 2000. pp. 4089–4094.

  28. Vidyarthi SK, Tiwari R, Singh SK, Xiao HW. Prediction of size and mass of pistachio kernels using random forest machine learning. J Food Process Eng. 2020;43(9): e13473.

    Article  Google Scholar 

Download references

Funding

The research of the Amar Nath was partially funded by a grant from the Department of Science and Technology: Science and Engineering Research Board (DST-SERB), grant number EEQ/2023/000792. The third author was in part supported by a research grant from Google.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Amar Nath.

Ethics declarations

Conflict of interest

The authors declare no Conflict of interest.

Ethical approval

No human or animal subjects were involved in this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nath, A., Patel, D. & Niyogi, R. Mass Prediction and Analysis of an Object’s Mass from Its Image Using Deep Learning. SN COMPUT. SCI. 5, 711 (2024). https://doi.org/10.1007/s42979-024-03050-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-03050-6

Keywords

Navigation