The licensed spectrum of cellular networks has become increasingly crowded, leading to the standardization of LTE licensed assisted access (LTE-LAA) and 5G NR-U for deployment in unlicensed bands such as 5 GHz. To coexist harmoniously with other unlicensed wireless technologies like WiFi, LAA and 5G NR-U enforce listen-before-talk (LBT) protocol. This paper proposes methods to enhance the overall spectrum efficiency and fairness of each coexisting heterogeneous link. To improve the overall spectrum efficiency, we propose enabling concurrent transmissions of multiple links. Motivated by the need for fair coexistence of heterogeneous networks with concurrent transmissions, we formulate a variant of the multi-armed bandit (MAB) problem that finds a probabilistic transmission strategy to maximize the minimum link throughput. We propose the Fair Probabilistic Explore-Then-Commit (FP-ETC) algorithm, which achieves the expected regret of $O\left(T^{\frac{2}{3}}(K \log T)^{\frac{1}{3}}\right)$. We compare FP-ETC with existing MAB algorithms via extensive simulations, and the results show that FP-ETC significantly outperforms the baseline algorithms.