In a world filled with heightened vandalism and terrorist activities, video surveillance forms an integral part of any incident investigation. While aiming to provide safety and security for the citizens, CCTV cameras are exponentially deployed around metropolitan cities. However, the information thus collected and further processed should comply with the legal and ethical rules defined by the law. The primary ethical issue invoked by surveillance activities in general is that of privacy. In order to ensure that a surveillance system framework complies with the legal, ethical and privacy requirements of the law, in this paper we present a Surveillance Ontology extending the SKOS foundational ontology. The objective of this ontology is to translate the high-level linguistic rules into the information that can be processed and used to assess the compliance of the video analysis module with the rules defined.