This paper studies the controllability degree of complex networks as a function of the network diameter and weights. We quantify the controllability degree of a network with the worst-case control energy to drive the network to an arbitrary state. We show that certain networks, including acyclic networks, are difficult to control whenever their diameter is a sublinear function of the network size, as the control energy grows exponentially with the network cardinality when the number of control nodes remains constant. Conversely, we show that certain anisotropic networks where the diameter depends linearly on the network cardinality are easy to control, as the control energy is bounded independently of the network cardinality and number of control nodes. We conjecture that the network diameter is a key topological property determining the controllability degree of a network.