Autonomous driving has immense potential to improve the safety of vehicles and pedestrians. However, safety assurances for Autonomous Vehicles (AVs) are lacking under adverse weather conditions and in unseen road environments. Unfortunately, real-world testing of AVs in such situations can be unsafe and even infeasible. Reproducing any failure cases with identical external operating conditions is practically impossible. Specifically, situations like heavy rain, low-lighting conditions, and work zones require considerable time and effort. To address this problem, we propose CoSim, a co-simulation architecture for testing of AV perception in adverse operating conditions. We specifically use the general framework of CoSim to interface CMU's autonomous software stack called CADRE with an open-source simulator named CARLA. CoSim is designed such that the AV software stack interacts with realistic simulated driving scenarios as in the real world. Using CoSim, we mimic the use of cameras and LIDARs for the real-time detection of road objects and work zones under adverse operating conditions. CoSim supports time virtualization on both CADRE and CARLA to accommodate heavy processing demands. Using CoSim, we evaluate the performance of our AV in adverse operating conditions, dramatically reducing the need for expensive real-world testing.