2025 Volume E108.A Issue 3 Pages 482-490
To overcome the excessive memory capacity of non-volatile CiM (NV-CiM) for multi-modal AI, this paper proposes Mapping-oriented enhanced-FPN (Feature Pyramid Network) fusion (More-FPN) as an RGB-event fusion object detection model. More-FPN includes three proposals. First proposal, Mapping-aware Group Convolution (MAGC), reduces the required NV-CiM capacity by suppressing the number of subarrays in NV-CiM at a fixed subarray size. In MAGC, the number of groups is optimized with no inference accuracy degradation. By adopting MAGC to FPN fusion of an RGB-event fusion object detection model, 54.7% subarrays are reduced. The second proposal, Separable Bridge (SepBridge), further reduces the number of subarrays by 26.1% from MAGC-adopted FPN fusion. Third proposal, Top-down path trainable BiFPN (TDT-BiFPN), achieves accuracy improvement with a slight subarray increase by adding bottom-up path and making top-down path trainable. By combining three proposals, More-FPN achieves both the reduction in subarrays by 61% and the accuracy improvement by 4.6%, compared with conventional FPN fusion CiM.