With a growing number of connected devices in the Internet-of-Things (IoT), multi-user detection (MUD) becomes a critical issue in the IoT gateway at the edge. Thanks to the feature of activity sparsity in the IoT devices, compressive sensing (CS) is a promising solution for MUD to handle massive devices under limited resources. For practical IoT applications, the CS-based IoT gateway needs to support an unknown number of active devices up to 10% of total devices and should complete a time-slot detection within 0.5 ms. However, most existing CS detection engines cannot fulfill above requirements simultaneously. In this paper, we proposed a modified sparsity adaptive matching pursuit (MSAMP) detection engine for the IoT gateway at the edge. With decomposition-free architecture, our proposed engine can efficiently operate multiple and arbitrary indices updating that can achieve adaptive sparsity realization and index set backtracking optimization with no need of prior information. Moreover, coarse-searching and turbo step-size techniques are designed to enhance 2.86× of throughput rate gain. The efficiently implemented MSAMP detection engine in TSMC 90-nm technology can complete a time-slot detection in 0.192 ms (<;0.5ms). Thus, the proposed detection engine can provide a constructive real-time design for practical IoT applications.