3D (or stereo) video has been a visually appealing and costly affordable technology. More sophisticated multi-view videos have also been demonstrated. Yet their remarkably increased data volume poses greater challenges to the conventional client/server streaming systems, which has already suffered from supporting 2D videos. The stringent multi-stream synchronization further complicate the system design. In this paper, we present an initial attempt toward efficient streaming of stereo/multi-view videos over a peer-to-peer network. We show that the inherent multi-stream nature of stereo video makes segment scheduling more difficult, which is particularly acute with the existence of multiple senders in a peer-to-peer overlay. We formulate the stereo segment scheduling problem as a Binary Quadratic Programming problem and optimally solve it using an MIQP solver. However, given the high peer dynamics and the stringent playback deadline in real-time streaming, the optimal solution is too costly to be obtained. Thus, we develop two efficient algorithms to allow peers frequently compute the scheduling. We show that one of the proposed algorithms can achieve an analytical guarantee in the worst case performance, in particular, the approximation factor is at most 3 comparing with the optimal solution. We implement the proposed algorithms and the optimal in a peer-to-peer simulating system, and show that the proposed algorithms can achieve near-optimal performance efficiently. We further implement two other scheduling algorithms that are used in popular peer-to-peer streaming systems for comparison, and extend our design to support multi-view video with view diversity and dynamics. Under different end-system and network configurations with both stereo and multi-view streaming, the simulation results demonstrate that our algorithms outperform others in terms of streaming quality, stream synchronization/smoothness and scalability.