On the aged society coming soon, many studies have explored homecare technologies. In this work, the activities at home are captured by a panoramic camera located at the center of a living room, and then analyzed and classified into standing, walking, sitting, falling, and watching television. First, the background subtraction scheme accompanied with shadow removal, and morphological operators of erosion and dilation is used to find out moving subjects, and TV on/off switching. Second, the parameters associated with the subject height, the distance between the camera and subject, and the distance map between subject contour points and centroid are computed to derive features for recognizing activities of standing, walking, sitting, and falling where the Support Vector Machine (SVM) classifier is employed. Additionally, the moving object similar to a parallelogram shape is identified as a TV set. Third, the majority voting in a picture segment of 10 pictures is adopted to determine the final recognition result. When subject sitting and TV on occur simultaneously, this situation is treated as an activity of watching TV. The experimental results reveal that our activity recognition system can achieve the accuracy rate up to 91.1%. The error rates associated with subject height and distance estimations are around 5.5% and 8.2%, respectively. The accuracies of TV on and off detection are close to 95.9% and 100.0%, respectively. Therefore, the proposed activity recognition system exhibits a superior performance for homecare applications.