A man-machine interface plays an important role to convey an intention from a user to machine. Nowadays, vision-based solution has been attracting a lot of attention since it does not require any attachment sensors on the body. One of the most famous applications is to control TV operations by hand gestures. Instead of bothersome operations using a specific controller, a user can send a command to a system by intuitive hand gestures. Most of previous studies have been focused on a strategy to detect/recognize hand motions, and the design concept to realize high usability of the application got less attention. This paper discusses how to design a user-friendly and also machine-friendly hand gesture application. Our concept gives a user an opportunity to customize a TV control interface by selecting hand shapes and hand motions through interaction with a hand gesture selection system.