A common belief in image steganalysis is that the image orientation does not matter because the statistics are similar when images are scanned horizontally or vertically. However, feature-based and deep learning-based steganalysis is sensitive to image rotations, demonstrating that images do contain directional statistics. In this paper, we systematically study how JPEG steganography and steganalysis methods interact with two relevant causes of directionality: scene content and asymmetries in the JPEG quantization table (QT). We find that (1) steganalysis detectors often achieve higher accuracy in images with directional scene content, (2) asymmetries in the QT bias the detector’s generalization towards one direction, (3) a steganalyst can benefit from training a detector tailored to the directionality, (4) rotation augmentation improves orientation robustness, but reduces detection performance on the original orientation.