Loading [a11y]/accessibility-menu.js
Prototypical Context-Aware Dynamics for Generalization in Visual Control With Model-Based Reinforcement Learning | IEEE Journals & Magazine | IEEE Xplore

Prototypical Context-Aware Dynamics for Generalization in Visual Control With Model-Based Reinforcement Learning


Abstract:

The latent world model, which efficiently represents high-dimensional observations within a latent space, has shown promise in reinforcement learning-based policies for v...Show More

Abstract:

The latent world model, which efficiently represents high-dimensional observations within a latent space, has shown promise in reinforcement learning-based policies for visual control tasks. Due to a lack of clear environmental context comprehension, its applicability in a variety of contexts with unknown dynamics is constrained. We propose a prototypical context- aware dynamics (ProtoCAD) model to address this issue. This model captures local dynamics using temporally consistent latent contexts and aids generalization in visual control tasks. By grouping prototypes over historical experiences, ProtoCAD collects useful contextual information that improves model-based reinforcement learning dynamics generalization in two ways. First, to guarantee the consistency of prototype assignments for various temporal segments of the same latent trajectory, a temporally consistent prototypes regularizer is used. Then, a context representation is devised to combine the aggregated prototype with the projection embedding of latent states. According to extensive trials, ProtoCAD outperforms competing approaches in terms of dynamics generalization for visual robotic control and autonomous driving applications.
Published in: IEEE Transactions on Industrial Informatics ( Volume: 20, Issue: 9, September 2024)
Page(s): 10717 - 10727
Date of Publication: 15 May 2024

ISSN Information:

Funding Agency:


Contact IEEE to Subscribe

References

References is not available for this document.