LwTool: A data processing toolkit for building a real-time pressure mapping smart textile software system

Abstract

Pressure mapping smart textile is a new type of sensing modality that transforms the pressure distribution over surfaces into digital ”image” and ”video”, that has rich application scenarios in Human Activity Recognition (HAR), because all human activities are linked with force change over certain surfaces. To speed up its application exploration, we propose a toolkit named LwTool for the data processing, including: (a) a feature library, including 1830 ready-to-use temporal and spatial features, (b) a hierarchical feature selection framework that automatically picks out the best features for a new application from the feature library. As real-time processing capability is important for instant user feedback, we emphasize not only on good recognition result but also on reducing time cost when selecting features. Our library and algorithms are validated on Smart-Toy and Smart-Bedsheet applications, an 89.7% accuracy for Smart-Toy and an 83.8% accuracy for Smart-Bedsheet can be achieved (10-fold cross-validation) using our feature library. Adopting the feature selection algorithm, the processing speed is increased by more than 3 times while maintaining high accuracy for both two applications. We believe our method could be a general and powerful toolkit in building real-time recognition software systems for pressure mapping smart textile.

Introduction

Human Activity Recognition (HAR) is an important sub-domain in the research field of ubiquitous computing. Pressure mapping smart textile [1] is a new type of sensor that measures pressure distribution over surfaces, both in the environment and on-body. Soft, thin, and flexible, the textile sensors can cover a large area, providing data of a large scale and high resolution, which are then mined for high-level information. Researchers have used it in dozens of applications (viz. Fig. 1), such as smart home [2], [3], [4], sports [5], [6], [7], healthcare [8], [9], [10]. Even Google started the Jacquard Project [11], releasing interactive digital textile for Human Computer Interaction.

As there are still many new applications to be explored for this emerging sensing modality, there exists a need for speeding up the application exploration. From a rough idea, first hardware needs to be constructed, then a small scale experiment to obtain the labeled data, then data mining algorithms are developed and applied. While hardware may differ from scenario to scenario, the data mining algorithms share the same pipeline: signal pre-processing, feature extraction, and classification. Our work focuses on creating a common toolkit for building the real-time data-mining pipeline for smart textile applications.

The application scenarios of smart textile are rich, and the characteristics of the collected data are the same, similar to ”video”, so it is thus feasible to establish a general feature library suitable for most applications. Researchers have proposed many feature sets for various applications, and these features can be categorized into three domains: the time domain features, the frequency domain features, and spatial features.

Real-time processing is always the ideal goal for data processing, as 1) for real-time response and instant user feedback, and 2) for offline processing, the un-processed data must be stored somewhere and with a large amount of data generated every day, the storage will become a problem sooner or later. For real-time processing, the calculation time is a critical measurement. We thus add time cost as a judging criterion when selecting feature subset from the feature library.

The contributions of this work are as follows:

• Based on the work of [15], we add new features and expand the 743 features contained in the initial version of the TPM feature set to 1830 features, including temporal and spatial features, as demonstrated in Fig. 2.

• To speed up feature calculation and enable real-time processing, we propose a hierarchical feature selection framework, that emphasizes not only on classification accuracy but also on time consumption.

• To estimate the time consumption of any given feature subset, we 1) propose the calculation dependency graph, and 2) develop the consumption estimation algorithm. Although in this paper the method is applied only to the feature library for smart textile, it is general enough to be applied to other feature sets (e.g. for feature set created from video or IMU data), too.