Overlay networking makes it easy for users add new network functionalities while keeping existing Internet connectivity intact. This paper introduces SCONE (Service-COmposable InterNEt) as a networking service to facilitate the management of service overlay networking. By looking into the structure of programmable overlay nodes, SCONE provides programmable IP service gateways (PSGs) that ensure high-speed per-flow packet processing for overlay networking. In order to meet the data-rate requirements of various host applications, each PSG is accelerated by hardware packet processing for its data plane. It also leverages the space-efficient pattern matching of entity cloning and provides localized (i.e., de-centralized) services to assist the scalable support for software-defined networking (SDN). An experiment result shows that the proposed PSGs can support high-fidelity overlay networking from both performance and scalability perspectives.