The concept of Network Functions Virtualisation provides the rapid and flexible deployment of new services for telco operators. Horizontal scaling is applied when one or more instances of a virtual network function component or a virtual network function are added or removed in response to the traffic from the users. In this paper, we present how Reinforcement Learning can help scaling the virtual network function components. Our results show that Reinforcement Learning can be used to control the scaling decisions in the Management and Orchestration of Network Function Virtualisation.