1 Introduction

The Government of India in December 2016 took a decision of de-monetization of the Indian currency. With the withdrawal of highly used currency denominations for every day financial transactions, this was a highly disruptive intervention with far reaching implications. The government introduced a new financial service platform – BHIM to be used on a mobile phone. This was to be used by the public at large to shift financial exchange from a cash rich to a less-cash economic mode of transaction. This dramatic and unexpected disruption brought to surface the immediacy and challenge of socio-economic and cultural factors that need to be urgently addressed if such an alternative solution is to find acceptance as an alternate service.

With the rapid growth and penetration of mobile telephony, companies in Information, Communication and Entertainment (ICE) sector, are faced with the design and development of interactive products and new services that concurrently address the challenges, aspirations and expectations of cultural diversity for their acceptance and success. With the increasing shift towards gesture based modes of product–user interaction, there is an urgent need for design research in the study of socio-cultural considerations that can affect gesture based interaction in such mass based product mediated services.

2 Related Work on Gesture Based Studies

The diversity of cultural variations in non-verbal modes of communication through body language is presented in two excellent anthropological publication titled ‘Man watching’ [1]) and ‘Body watching’ [2]. Since the early 2000, there is a distinct move towards developing gesture based interaction in the design and development of ICT based information, communication and entertainment (ICE) based products. Early attempts in the development of gesture based interaction in products were algorithm based [3]. More recent studies have explored the use of gestures for the control of mobile phones [4]. Their study was essentially non-technical in nature and focused on the study of natural human gestures that the subjects may use in devices such as a mobile phone. Some studies have explored the use of gestures in the navigation of the different functional tasks to be performed on a smart phone [5]. Their paper outlines a gesture taxonomy involving 19 different tasks to be performed on the device. Study in the use of gestures on iPad was undertaken to help identify gestures that can be used to transfer data from an iPad to a PC [6].

In India, a recent study amongst semi-literate rural users in the state of Assam, examined the potential of its use for the domain of neonatal healthcare for a television based product user interaction for the Indian context [7]. The study resulted in the development of gesture based interaction using a Kinect based platform that was developed for maternal healthcare [8] for semi-literate rural women users. This doctoral research followed a user-centered approach and identified gestures that were commonly understood amongst low literacy and low-income rural communities in northeast India. Unfortunately such gesture based user-product interactions studies are in their infancy in the Indian context.

These research studies however, show that communication through gestures is definitely not universal. They vary from one community to another, one region to another and also can be different for different religious groups - signifying different meanings in their use. The study of gestures and their communicative meanings in multi-cultural, multi-linguistic and multi-religious dimensions is an interesting subject for design research. Its implications in the designs of man/machine interaction system will necessary have to factor this cultural dimension.

This paper examines cultural variations in communication through body gestures amongst Indian user groups following the taxonomy outlined by McNeill [9]. The findings help to reflect upon their implication in design of gesture based man-machine interaction. The experiments are planned to examine the following specific questions: Are there cross-cultural variations in the use of hand gestures among a pan Indian user groups? Are there gestures that are common to the different groups? Do similar gestures mean differently to the different cultural groups? Can individuals articulate and explain the gestures common in their culture?

3 Methodology

Researchers have developed different methods for development of gesture based interactive systems [10]. Gestures have been derived from subjects identified by semantic representation of their associated function [11]. Participatory design method has been used for generation of gestures for surface computing [12]. Studies have been undertaken to examine what gestures signify to people [13].

In most of these methods user participation in the process of generation and reading of gestures is followed. Participatory approach that involved participants from the different regions of India was adopted for this study.

In the experiment planned, attempt is made to study hand gestures involved in communication by participants coming from different states in India and the findings are summarized to answer the above queries for their implication in design of gesture based man-machine interaction.

4 Planning the Experiment and Methodology Used for the Cross-Cultural Study of Gestures Amongst Indian Users

4.1 Selection of Content for the Study

The objectives of the experiment essentially focused on identifying cultural variations in the involuntary use of non-verbal communication using hand gestures. It was planned that hand gestures involved during a conversation be captured in a natural and unobtrusive manner without making the subject conscious. Engaging the participating group to discuss a cinema clip screening offered suitable scope to capture the different hand gestures involved during conversation. An Iranian film titled ‘Two and Two’ directed by Babak Anvari was selected for the screening. Duration of screening was for 7 min. Although in Persian language, the clip being rich in human emotions participants were encouraged to discuss aspects of the visual language after the screening. The Persian language used in the film not being familiar, enabled the participants to focus on the visual content in their discussion. This helped to overcome any individual bias of the region that the participant belonged to or that of his regional language and cultural background.

4.2 Selection of Subjects

Drawing strength of a multi-cultural and multi regional mix of students selected for admission to its programs, the participants selected for this study were 26 senior students of Design who were coming from seven different states in India viz. Kerala (kl), Tamil Nadu (t) and Andhra Pradesh (ap) the southern states (10 participants); Delhi (d) and Kashmir (ks) the northern states (5 participants); Maharashtra (m) from the western state (4 participants) and Jharkhand (j) and Assam (a) from the eastern states (7 participants) - a pan India cross section of 26 participants in all, covering a mix of multi-cultural diversity, religious background and regional languages (Fig. 1). They were formed into small groups comprising of two to three member belonging to each region. The film clip was screened separately for these different groups and the members of each group were asked to discuss the film after the screening.

Fig. 1.
figure 1

States in India to which participants belong.

Full size image

4.3 Selection of the Medium for Documentation

Video recording being a powerful enabler was the chosen medium for documenting the conversation of the participants. This was subsequently reviewed for the study of hand gestures they involuntarily used during their discussions. The video helped to record movement, capture gestural speed in real time, faithfully show how many repetitions were used, and even present the hand gestures along with other involuntary body nonverbal behavior (facial grimaces, postural changes, etc.) as they occur. Video could even record for the viewer the participant’s (or “encoder’s”) own contextual account about how the gesture is used, circumstances that illustrate a use of this gesture, the probable consequences of using the gesture etc.

4.4 Design and Studio Setup

The video documentation was done using Canon 650D DSLR cameras. A schematic position of the subjects, the camera positions and ceiling light settings is as shown (Fig. 2). It may be noted that people generally are found to use gestures effectively while standing. The participants from each state were allowed to discuss the cinema clipping while standing. They were positioned to stand in manner so that they come face to face as in a casual talk. Three cameras have been used to capture the discussion so as to get the front view of each person clearly.

Fig. 2.
figure 2

Studio and camera setup for 4-participant discussion.

Full size image

4.5 Identification of Gestures from Secondary Literature

Based on the taxonomy proposed by McNeill in a study of gestures in HCI [9] the video recording was reviewed to identify, categorize and analyze the following hand gesture categories (Fig. 3).

Fig. 3.
figure 3

Category of gestures

Full size image

4.6 Method for Analysis of Video Recordings

Video recording were transferred to a Macbook pro computer and reviewed on a video editing Final cut Pro application software. The video editing software helped to run over individual visual frames across a time line. The video recordings of each participating group were viewed on an average for 30 times. Around 149 video clips were sorted from the video footage, to collect and identify more than 250 non-verbal hand gestures and categorize using the following legend:

‘XYZ’: X = Taxonomy of the of gesture category, y = State the subject belongs, z = Serial number.

Non-verbal hand gestures involuntarily engaged during the conversations by each participant were analyzed for each group from the video recordings. Both duration of the gestures and the visual image frames were noted for each subject and the gesture classified under the seven categories. A sample of data logging is shown in Table 1.

Table 1. Sample of data logging
Full size table

4.7 Data on Total Gesture Types Taken for Observation and Study

The Table 2 below summarizes the list of gestures shortlisted for the study:

Table 2. State-wise distribution of gesture type
Full size table

5 Results and Observations

5.1 Deictic Gestures (D)

Deictic gestures are used to indicate objects and directions.

The study did not show significant differences in the use of this hand gesture among the 26 participants across the states. Most subjects while they were talking used the pointing gestures in similar ways with variations of use of single hand or both hands (Fig. 4).

Fig. 4.
figure 4

Deictic gestures made by participants grouped state-wise

Full size image

5.2 Iconic Gestures

Metaphoric Gesture (M).

Metaphoric gestures are also representational, but the concept they represent has no physical form; instead the form of the gesture comes from a common metaphor.

E.g. “the meeting went on and on” accompanied by a hand indicating rolling motion.

The uses of metaphoric gestures by participants across the different regions were high but also significantly varying in their meaning. The most diverse range of gestures came in this category. The participants were found to use the same kind of gesture to convey different meaning. It was also found that participants used different gestures to convey same meaning. For e.g. The metaphoric gesture (M1) used by participants - waving the hand forward keeping the palm downward (Fig. 5) - signified different meaning.

Fig. 5.
figure 5

Metaphoric gesture (M1)

Full size image

  • Delhi (Md3) - Suppressed

  • Kerala(Mkl1,2) - Forcing

  • Maharashtra(Mm2) - This has to be done

  • Tamilnadu (Mt4,8) - Forcing

Other Metaphoric gestures (Fig. 6) used often by the participants are given below.

Fig. 6.
figure 6

Metaphoric gestures used by the participants

Full size image

These five gestures (M1 to M5) were used by the participants in distinct context. Table 3 shows this.

Table 3. Similar metaphoric gesture with different meaning used by the participants
Full size table

Another important thing that can be observed from the table is that some participants used distinct gestures for the same context.

This diversity was there among the people from the same state itself. For e.g. from the observation of participants from Kerala, almost similar gesture is used to represent “focused” (Mkl1), “Forced” (Mkl2) and “forward” (Mkl3) by waving hand forward (Fig. 7).

Fig. 7.
figure 7

Metaphoric gestures made by participants grouped state-wise.

Full size image

5.3 Semaphoric Gestures (S)

Semaphoric gestures are hand postures and movements, which are used to convey specific meanings. Mostly gesture and meaning are completely unrelated and strictly learned. Therefore, semaphoric gestures are most dependent on the participant’s background and experience.

Semaphoric gestures in this experiment show complete variations in their use among the participants. A wide variety of gestures are observed to be used. Few gestures that are used frequently are shown (Fig. 8).

Fig. 8.
figure 8

Semaphoric gestures used by the participants

Full size image

All these gestures were used in entirely different context by the participants.

Here ‘S1’ is a gesture performed by showing hand palm upward like a flower. This has been used in entirely three distinct context to mean “Anything”, “but”, “I didn’t understand” etc. (Sd1, Sd2, St1) (Fig. 9).

Fig. 9.
figure 9

Semaphoric gestures made by participants grouped state-wise.

Full size image

Other three semaphoric gestures and the contexts in which they were used is shown in the Table 4:

Table 4. Semaphoric gesture with different meaning used in different states
Full size table

5.4 Pantomimic Gestures

As pantomimic gestures are used to show the movement or action performed by a tool or object. These are the gestures often used in showing the use of movement of some invisible tool or object in the speaker’s hand. The important feature of mimic gestures is that they strive for reality.

The gestures performed by the participants to indicate the object, in this instance a gun, were similar (Fig. 10). Variations if any were seen in the gesture performed using a single hand or two hands.

Fig. 10.
figure 10

Pantomimic gestures made by participants grouped state-wise.

Full size image

6 Conclusion

Insights drawn from this study have implications in the design and development of gesture based interface for product interactions. Gestures to be developed for man/machine interaction system do not follow a universal language. There are cultural differences in the meaning and use of gestures used by people belonging to different regions across India. The gestures analyzed from different part of India were categorized into Deictic, Iconic, Metaphoric, Semaphoric and Pantomimic gestures. From the study it is seen that there are only minor difference in the gestures that belong to Deictic, Iconic, Pantomimic categories. Identification of a family of Deictic, Iconic, Pantomimic gestures may prove to be useful in the design of man-machine interactive systems. Iconic gestures that specify a manner in which action is carried out; Deictic gestures that help pointing or directing the users attention to specific events in the machine environment and Pantomimic gestures showing the use of movement of some invisible tool or object in the speaker’s hand can be identified in the design of gesture based interfaces for man –machine systems.

In comparison, there are noticeable differences in Metaphoric and Semaphoric gestures used by the participants across different regions in India. Since metaphoric gestures are also representational but the concepts they represent show variations from one region to anther. It may be suggested that an inventory of region specific gestures be identified and be offered as optional choice for users to personalize them as per their own preferences. Similarly Semaphoric gestures offer a similar challenge as there are variations in the hand postures and movements used to convey specific meanings and are specific to each region and culture of use. These again need to be offered as a basket of choice to accommodate individual users preference and choice.

7 Implication for Interaction Design

The above factors have implications for the interaction designers. A detailed study with a larger sample size needs to be undertaken by the design research team to identify these commonalities and differences of use of gestures, their signification and meaning across by undertaking a pan Indian survey. It will go a long way towards understanding cultural factors that will impact the acceptance in intelligent and interactive products for their usability and market success.

As a growing economy, for a multicultural country like India, the market potential for usable and intelligent products is very large. The modes of interaction vary. The importance of making gesture based interfaces for region specific diversity will be imperative.