The awareness of physical network topology in a large-scale Internet of Things (IoT) system is critical to enable location-based service provisioning and performance optimization. However, due to the dynamics and complexity of IoT networks, it is usually very difficult to discover and update the physical topology of the large-scale IoT systems in real-time. Considering the stringent latency requirements in IoT systems, while the initial processing time for topology discovery can be tolerated, latency due to real-time topology update constitutes an even higher level of challenge. In this paper, a novel fast hierarchical topology update scheme is proposed for the large-scale IoT systems enabled by using the edge-cloud collaborative architecture. Specifically, an event-driven neighbor update algorithm, termed as TriggerOn, is firstly developed to update the local neighbor table of the end devices when device association or disassociation occurs. Based on the updated neighbor tables, the physical topology update of the subnet is conducted at the coordinated edge device, where a hybrid multidimensional scaling (MDS) based 3D localization algorithm is developed to locate the newly associated devices. Simulation results have indicated that as compared to the benchmark methods, the neighbor discovery latency has been reduced dramatically, and the 3D localization accuracy has been improved. Furthermore, the overall latency incurred by the proposed hierarchical physical topology update scheme is significantly lower than the distributed consensus-based update scheme, especially for the large-scale IoT subnets.