Neuromorphic vision sensors (NVSs) mimic the function of the human visual system, with significant energy-saving potential in IoT-based object detection systems. Unlike conventional sensors, NVSs only generate asynchronous spiking events in response to changes in light intensity. However, the inherent noise generated by NVSs causes a degradation of detection performance. Moreover, an interested object usually occupies only a portion of the entire image frame. Therefore, a real-time, accurate event-based object detection system is needed to identify the region of interest (Rol) and leverage this spatial redundancy to reduce computational load in subsequent recognition modules. In this article, we present an energy-efficient real-time object detection system featuring a hybrid event-based frame generation pipeline and a background-removal region proposal algorithm. The event-based frame is generated by aggregating active events within a programmable time interval, generating an event-based binary image (EBBI). This approach enables the utilization of low-complexity algorithms for denoising and object detection. The background-removal region proposal algorithm reduces memory requirements and removes dynamic backgrounds, leading to better detection performance. The proposed system is demonstrated on Zynq-7000 FPGA device with a DAVIS346 sensor. Experimental results show that the proposed system achieves comparable accuracy while requiring significantly less computation than existing event-based trackers.