Accurate segmentation of brain tumors is a challenging task and also a crucial step in diagnosis and treatment planning for cancer patients. Magnetic resonance imaging (MRI) is the standard imaging modality for detection, characterization, treatment planning and outcome evaluation of brain tumors. MRI scans are usually acquired at multiple sessions before and after the treatment. An automatic segmentation framework is highly desirable to segment brain tumors in MR images as it streamlines the image-guided radiation therapy workflow considerably. Automatic segmentation of brain tumors also facilitates an incremental development of data-driven systems for therapy outcome prediction based on radiomics analysis. In this study, an outlier-detection-based segmentation framework is proposed to delineate brain tumors in magnetic resonance (MR) images automatically. The proposed method considers the tumor and edema pixels in an MR image as outliers compared to the pixels associated with the healthy tissue. The framework generates two outlier masks using independent one-class support vector machines that operate on post-contrast T1-weighted (T1w) and T2-weighted-fluid-attenuation-inversion-recovery (T2-FLAIR) images. The outlier masks are subsequently refined and fused using a number of morphological and logical operators to estimate a tumor mask for each image slice. The framework was constructed and evaluated using the MRI data acquired from 35 and 5 patients with brain metastasis, respectively. The obtained results demonstrated an average Dice similarity coefficient and Hausdorff distance of 0.84 ± 0.06 and 1.85 ± 0.48 mm, respectively, between the manual (ground truth) and automatic tumor contours, on the independent test set.