Pedestrian and Vehicle Detection (PVD) is important to auto-driving. However, existing deep-network-based PVD methods suffer from a low inference speed due to large redundancy computations. In order to ensure detection accuracy while increasing the computing efficiency of PVD, we propose an orthogonal re-parameterized method. First, we design a Ghost Structural Re-parameterized (GS-ReP) module, which is an efficient multibranch structure that combines the low computational ghost method to generate rich features and the structural re-parameterization to simplify the multi-branch structure into a single-branch structure. Second, we propose a Feature Orthogonal Loss (FOL) function to further reduce redundancies between features from different branches of GS-ReP, producing low redundancy features useful for detection accuracy. Third, we integrate our GS-ReP to the small version of the fifth You Only Look Once (YOLOv5-s) model to raise Orthogonal Re-parameterized Networks (ORNet). Extensive experiments show that our ORNet method obtains state-of-the-art results. On the public KITTI dataset, ORNet's mean average precision reaches 91.01%, and only costs 5.5 M parameters. On the NVIDIA Jetson Xavier NX embedded device, for 640 × 640 sized images, the Frames Per Second (FPS) of ORNet varies in [25.9, 31.7] depending on the batch size setting.