Although edge intelligence (EI) propels the development of Internet of Things (IoT) applications to a new stage, the distributed nature of the end users in EI networks greatly hinders its practical deployment. First, the resources of distributed end devices are limited, including computing and transmission resources, while the intelligent model typically necessitates intensive computation and substantial data from the network end. Second, the resources of end devices also exhibit heterogeneity, further complicating the learning in EI. Specifically, each device varies in computational capabilities, making it challenging to synchronize updates in collaborative learning approaches. Additionally, owing to the dispersed locations, each device encounters diverse wireless conditions, impeding effective communication with the edge server. Therefore, addressing the communication and computation constraints is necessary to foster practical EI applications. While novel distributed learning (DL) algorithms and machine learning (ML)-related techniques exhibit great potential, related review work lacks. Motivated by the literature gap, we provide a comprehensive review of the latest research endeavors on facilitating efficient EI deployment via examining novel DL algorithms and ML-related techniques. We also demonstrate the interplay of computation and communication efficiency in the resource-constrained EI landscape.