The scalable video coding (SVC) extension of H.264/AVC was recently standardized. The encoder control that is described in the joint scalable video model for SVC specifies a bottom-up process in which first the base layer and then the enhancement layer is encoded. The base layer is encoded conforming to H.264/AVC without considering its impact on enhancement layers, which limits the achievable enhancement layer coding efficiency. The losses relative to single-layer H.264/AVC coding are unevenly distributed between base and enhancement layers. In this paper, we present a multi-layer encoder control for SVC, by which we can trade off coding efficiency for base and enhancement layers and generally improve the efficiency of spatial and fidelity scalable coding.