Taking notes is frequently required in daily life. The rapid development of consumer devices provides new ways to achieve this goal such as taking notes by digital cameras, camcorders, camera phones, or voice recorders. In this paper, a novel camera-based note taking system is presented, which enables efficient and nature notes taking, management, searching and exporting. Firstly, the system automatically classifies photos and video clips taken by a variety of capturing devices into two classes, notes and non-notes, and then further classifies the class of notes into more fractional classes such as document, map, slides, bulletin board, and so on. Next the visual quality of the note photos and video clips are enhanced and adjusted by a set of image and video processing algorithms, as well as textual, color and texture information are extracted from them. And last, based on these analyses, a management system enabling efficient note importing, searching, browsing and exporting is built.