Current state-of-the-art methods for salient object detection in optical remote sensing images (RSI-SOD) primarily relies on individual image context to detect salient objects. However, the potential of cross-image contexts remains largely unexplored in existing works, which can provide valuable auxiliary and complementary information for discriminating object representations in RSIs. In this article, we investigate the utilization of cross-image contextual information for RSI-SOD. We propose a novel memory-based context propagation network (MCP-Net) to harness dataset-level contextual information. MCP-Net incorporates a cross-image dual memory (CDM) module to store dataset-level information and utilize it to generate contextual information for the current image. CDM effectively captures intra-scene variations by leveraging both foreground and background memory banks, resulting in improved object representations. Additionally, we enhance the representations by leveraging scale-aware context information within individual images. To preserve RSI details before memory modules, we introduce a shared attention-guided fusion (SAF) module to align the adjacent network-level features. Extensive evaluation results demonstrate the superior performance of our proposed method compared to state-of-the-art methods on three public benchmarks. These results affirm that the inclusion of cross-image contexts can significantly benefit salient object detection (SOD) in remote sensing images (RSIs).