Carrier frequency offset (CFO) and phase noise (PN) are major oscillator impairments in direct-conversion transceivers, and single-carrier frequency-division multiple access (SC-FDMA) is the uplink transmission scheme in the Long-Term Evolution (LTE) standard. We derive a new analytical expression for the normalized mean square error (NMSE) in asynchronous SC-FDMA systems under CFO and joint transmit-receive PN. The derived NMSE expression reveals an interesting cross-layer relationship between the subcarrier mapping scheme at the medium-access-control layer and the immunity to CFO and PN at the physical layer. Furthermore, we propose an iterative reduced-complexity joint decoding and PN compensation scheme that does not require any pilots in PN tracking and exploits the low-pass nature of the PN process without assuming a specific PN model. Simulation results show the effectiveness of our proposed digital baseband compensation scheme in PN mitigation.