One central task to the idea of Semantic Web is reasoning over semantic descriptions of web pages and information items available on the Web. A flagship project that is advancing the state of the art in reasoning with Web-scale data is the Large Knowledge Collider (LarKC). Having a pluggable architecture, LarKC enables the interested users to test their reasoning approaches with very little overhead. In this context, instrumenting and monitoring of the large scale reasoning systems and their components becomes essential for verifying and assuring high performance, adaptability and well-functioning. These aspects are in the end vital for any reasoning experiment. We introduce SIM, Semantic Instrumentation and Monitoring, a semantic-based instrumentation and monitoring solution. SIM enables the instrumentation and monitoring of LarKC applications in particular and any large scale reasoning system in general. It offers the means for developers to specify the metrics of interest, to instrument the code, to collect and observe how well the system and its components are performing. We identify a large set of relevant metrics for monitoring and provide ontological models for them. Finally we discuss the architecture and the role of each component and tool which is part of SIM.