Plug-in electric vehicles (PEVs) have the highest promise for dramatically reducing transportation emissions. No other option has comparable emission reduction potential or as a promising pathway. Still, PEVs can offer more than green transportation. In particular, their onboard storage can further serve the society by providing an energy buffer to increase the reliability, affordability, and sustainability of electric services. These benefits are only achievable by fully exploiting the multifaceted flexibility provided by PEVs' mobility, charging adaptability, and bidirectional flow of power, as well as adopting effective decision-making and control algorithms, while minding the likely unfavorable side effects, such as shortened battery life span. This work takes a closer look at different elements of this puzzle. The main subject of this survey is behind the meter energy management with vehicle to building (V2B). We focus on different V2B application ideas and review energy management methods in smart buildings with V2B integration. Recent findings on battery capacity fade resulting from the bidirectional flow of power and extra discharging cycles with V2B are reviewed, and the methods for integrating the battery degradation in energy management formulation are discussed. Finally, the main findings of this review and research gaps are summarized and clarified.