Expressive interaction with wearable head-mounted displays for virtual (VR) and augmented reality (AR) systems is essential for practical adoption. These systems pose new challenges and have higher performance standards compared to other computing paradigms. In this position paper, I argue that interactive devices for VR and AR systems can leverage high-precision tracking and haptics to achieve a robust set of interaction techniques and a rich sense of presence. I describe my past and proposed future research in designing interactive devices that innovate in the domains of eye tracking, wearable finger input, and handheld controllers.