We address the problem of establishing an end-to-end network slice across multiple domains and propose a Reinforcement Learning-based framework that enables multiple domains to collaborate on end-to-end network slicing admission and allocation. The objective is to maximize the long-term revenue of the network operator. We employ a Graph Neural Network (GNN) to capture the topology features as the encoder. The simulation results show that our framework improves the profit of the network operator by up to 15% compared to a greedy algorithm.