With advances in wearable computing, wearable virtual reality and wearable robotics have become popular areas of research. In the present study, we consider the case in which an expert in first aid treatment at a remote critical care center receives an image from an assistant/cooperator who is with a patient, i.e., the sharing of visual information between an expert and an assistant. A head-mounted camera worn by the assistant is used to capture visual information related to the patient and his/her surroundings. The expert directs the assistant to provide the desired visual information. Information of the status of the patient, the surrounding environment, and the situation can be obtained from the shared visual information. The assistant should make observations according to the directions of the expert. However, these persons are not required to be tightly linked/synchronized at all times, but only when necessary. There are primarily two behaviors in visual information sharing, namely, searching and staring behaviors. The proposed asynchronous visual information sharing system with image stabilization can separate these two behaviors and can reduce the stress of both the expert and the assistant in visual information sharing. This system consists of three components: image stabilization, virtual viewing direction operation, and relative orientation display. We experimentally evaluate the performance of virtual viewing direction operation, image stabilization, and asynchronous visual information sharing of the proposed system.