The mapping and labeling of the major intra-hepatic blood vessels may facilitate planning liver interventions and surgery. However, the automatic labeling of liver veins is challenging due to imperfect segmentations caused by partial volume effects and image resolution that result in undesirable false connections between hepatic and portal veins. In this paper, we propose a novel method to model the continuity of consecutive venous branches in a probabilistic manner. Then the model is automatically labeled via inference. The method incorporates low-level metrics for neighboring nodes and mid-level metrics for neighboring branches. Making use of these metrics, the automatic labeling becomes a probabilistic tracing procedure starting from each end nodes of the vessel skeleton. The method has only one free parameter whose value is not critical to labeling results. Experiments using data from healthy and pathological patients were performed and the results illustrate an accuracy of 0.97±0.08.