A simple and effective recognition system for indoor scenes is presented. The proposed system has two phases, namely, creation of mandatory and desirable objects and an indoor scene recognition system (ISRS). In the first phase, a list of probable objects and their classification, such as mandatory and desirable objects, for any generic scene is created based on real-time indoor environment clubbed with human knowledge on standard datasets. In the second phase, the proposed system contains four stages. In the first stage, the proposed ISRS identifies and recognizes the objects of the given key frame based on a simplified version of CNN architecture of YOLO v3. In the second stage, the identified objects are divided into two sets of mandatory and desirable objects with a simple dictionary look-up. In the third stage, the objects are identified to belong to a probable scene, and this technique is called scene-object identification. In the final stage, a binary scene representation (BSR) is proposed for each probable scene, and a final scene recognition is obtained with a new scene-score, obtained after converting the binary BSR into a decimal number. The effect of the proposed ISRS has been experimented with standard datasets and measured in terms of standard metrics, besides comparison with existing schemes. The results are encouraging.