Internet of Things (IoT)- based services connect the user with sensing devices through intermediary devices like a gateway. The authentication with secure key exchange assures security trio of confidentiality, integrity, and availability for the complete IoT-based system. It also ensures trusted privacy of communicated information. Recently, numerous authors proposed a Remote User Authentication scheme (RUA) on User-Gateway based secure key exchange for the tiny and lightweight sensor based paradigms such as an IoT and a wireless sensor network (WSN) using lightweight mathematical approaches. In this article, we propose a reliable and efficient lightweight key exchange scheme with the secure RUA for the user-gateway model. For designing a lightweight RUA scheme, we use elliptic curve cryptography (ECC). We use BAN-Logic for the validation of mutual authentication. We use widely accepted random oracle-based ROR model for formal security analysis and Dolev-Yao channel for informal security analysis. For the implementation and result collection of the proposed scheme, we used publisher-subscriber-based lightweight Message Queuing Telemetry Transport (MQTT) protocol. Overall, in this article, we propose a highly secure and efficient RUA scheme for the sensor-based environment with the formal security analysis and real-time implementation.