In this paper we present a video surveillance platform for the automatic analysis of complex environments, namely urban outdoor scenarios and crowded areas, including airports and train or metro stations. Considering the difficulty in performing continuous tracking and activity monitoring for every single subject in the scene, and due to the multiple occlusions and the complexity of the visual scene, the set of tools implemented in the system are designed to assess the situation in the monitored area according to a scalable architecture. The modules include: abandoned object detection, sterile zone and door surveillance, crowd anomaly detection, violent interactions detection, and reidentification. The modules have been developed and tested on several benchmark datasets before the deployment, to verify the compliance with the application requirements.