Figure 1:All images rendered with our speculative ray tracing technique. First three columns: a massive channel-flow turbulence DNS dataset. Last two columns: an RM fluid instability dataset and an Enzo Astrophysics AMR dataset (left and right). The number of triangles from left to right: DNS2 (1.8 billion), DNS1-side and DNS1-back (0.9 billion), RM (108 million), and Enzo (8 million). Five images on the bottom row show ambient occlusion shading, and the rest show three-bounce path tracing. For all datasets, 32 samples per pixel were used to render images at 1024×1024 resolution, and one diffuse ray and 16 shadow rays were generated at every hit point. Using Stampede2 Skylake at the Texas Advanced Computing Center, each node with 192 GB memory, at least eight nodes are required to render the DNS dataset with speculation enabled.With modern supercomputers offering petascale compute capability, scientific simulations are now producing terascale data. For comprehensive understanding of such large data, ray tracing is becoming increasingly important for 3D-rendering in visualization due to its inherent ability to convey physically realistic visual information to the user. Implementing efficient parallel ray tracing systems on supercomputers while maximizing locality and parallelism is challenging because of the overhead incurred by ray communication across the cluster of compute nodes and data loading from storage. To address the problem, reordering rendering computations by means of ray batching and scheduling has been proposed to temporarily avoid inherent dependencies in the rendering computations and amortize the cost of expensive data moving operations over ray batches. In this paper, we introduce a novel speculative ray scheduling method that builds upon this insight but radically changes the approach to resolving dependencies by allowing redundant computations to a certain extent. To evaluate the method, we measure the performance of different implementations for...