The incorrect speech recognition results usually have a negative impact on the speech summarization task, especially on the meeting domain where the word error rate is often higher than other speech genres. In this paper we investigate using rich speech recognition results to improve meeting summarization performance. Two kinds of structures are considered, n-best hypotheses and confusion networks. We develop methods to utilize multiple word and sentence candidates and their recognition confidence for summarization under an unsupervised framework. Our experimental results on the ICSI meeting corpus show that our proposed method can significantly improve summarization performance over using 1-best recognition output, evaluated by both ROUGE-1 and ROUGE-2 scores. We also find that if the task is to generate speech summaries or identify salient segments, using rich speech recognition output is just as effective as using human transcripts. In addition, we discuss the difference between n-best lists and confusion networks, and analyze the word error rate in the exacted summary sentences.