Exploring and Exploiting the Hierarchical Structure of a Scene for Scene Graph Generation | IEEE Conference Publication | IEEE Xplore

Exploring and Exploiting the Hierarchical Structure of a Scene for Scene Graph Generation

Publisher: IEEE

Abstract:

The scene graph of an image is an explicit, concise representation of the image; hence, it can be used in various applications such as visual question answering or robot ...View more

Abstract:

The scene graph of an image is an explicit, concise representation of the image; hence, it can be used in various applications such as visual question answering or robot vision. We propose a novel neural network model for generating scene graphs that maintain global consistency, which prevents the generation of unrealistic scene graphs; the performance in the scene graph generation task is expected to improve. Our proposed model is used to construct a hierarchical structure whose leaf nodes correspond to objects depicted in the image, and a message is passed along the estimated structure on the fly. To this end, we aggregate features of all objects into the root node of the hierarchical structure, and the global context is back-propagated to the root node to maintain all the object nodes. The experimental results on the Visual Genome dataset indicate that the proposed model outperformed the existing models in scene graph generation tasks. We further qualitatively confirmed that the hierarchical structures captured by the proposed model seemed to be valid.
Date of Conference: 10-15 January 2021
Date Added to IEEE Xplore: 05 May 2021
ISBN Information:
Print on Demand(PoD) ISSN: 1051-4651
Publisher: IEEE
Conference Location: Milan, Italy

References

References is not available for this document.