Building a 100 Mpixel graphics device for the OptIPuter
Introduction
Sending a stream of images over a computer network is a common and well understood task. Traditionally, the focus has been on lowering the load on the network by exploiting the temporal and spatial redundancy in the stream with compression technology. For our purposes, the focus is on creating the best possible experience for the researcher doing a visualization task on the novel infrastructure. We are concerned with responsiveness and accuracy of the images generated on the display device. In accordance with the OptIPuter [1] philosophy, we assume ample bandwidth, causing less dependence on compression technology [2].
Similar work has been done before, on the iGrid2002 venue, using Griz [3]. The Griz demonstration showed a 1280×960 visualization rendered in Chicago, which was transmitted to Amsterdam, scaled up to 2560×2048 and displayed there. To achieve good frame rates, Griz batched together multiple frames, and combined UDP [4] and TCP [5] protocols to achieve a fast yet reliable connection. The successor of Griz, called Scalable Adaptive Graphics Environment, or SAGE [6], was demonstrated [7] at iGrid2005. SAGE differs from our approach in that it allows multiple windows on the tiled panel. Additionally, SAGE has decoupled the image resolutions that are used render-side (at the graphics device) and display-side. SAGE relies on display-side graphics hardware to map the resolutions.
Section snippets
OptIPuter
The OptIPuter concept is based on the realization that optical networks are faster than the computers attached to them. The bandwidth of an Ethernet card is comparable to that of the internal bus of the computer. Therefore, one could opt for connecting the components of a computer not with a PCI bus, but with optical networks. The distributed computer could have its disk in Tokyo, processor in Chicago and display in Amsterdam. Each component of this distributed computer is typically an
iGrid2005 demonstrations
The capabilities of the graphics device built for iGrid2005 were demonstrated by running two applications. The first application is a 2D image viewer that allows the users to view high resolution images. The resolution of these images is much larger than that of the 100 Mpixel tiled display at the venue (typically giga-pixel images). Functionality available to the user is panning and zooming on the image. The second application is a 3D model viewer that allows complex models, typically
Software implementation
To make effective use of the high-latency, high-bandwidth networks, one has to be careful with the use of reliable communication using TCP. Martin [8] measured the effect that a single packet loss can have on a 10 Gbits TCP stream with a 200 ms round trip time. After a single packet loss, the bandwidth is halved to 5 Gbps, and it takes 5 h to regain the 10 Gbps performance. This means that for these high latency networks, a better approach is required. The Reliable Blast UDP, or RBUDP [9]
iGrid2005 infrastructure
The graphics device created for our iGrid2005 demonstration consisted of a 29 node render cluster at SARA in Amsterdam, connected to the iGrid2005 venue with a 20 Gbps connection. Each node of the render cluster had two Xeon CPUs, an Nvidia graphics card and a 1 Gbps network connection. The nodes were connected to a high-bandwidth switch that had two 10 Gbps uplinks to NetherLight, The Netherlands optical exchange located at SARA. From there two separate lambdas were made available for the
Results
Qualitative results were positive. The applications allowed responsive control by the user. Delay between user input and visualization output was short, comparable to the network round-trip latency (hundreds of milliseconds). Both the 3D and 2D datasets could be examined at the 100 Mpixel resolution of the tiled display panel, in a fluid manner. The high throughput, low latency transmission of the images more than makes up for the artifacts induced by packet loss.
IGrid2005 is a unique event
Conclusion and recommendation
By our transatlantic demonstration at iGrid2005, we showed that using an unreliable network protocol to stream extremely high resolution video is feasible. Our application benefits from UDP’s good performance, even on high latency networks. UDP’s unreliability does not affect the perceived quality of the experience. Large MTU size (jumbo frames) is required to effectively use the available bandwidth and to prevent display nodes from CPU overload.
One part of the OptIPuter approach, separation of
Bram Stolk is a Virtual Reality specialist with SARA. He received his MSc in Computer Science from the University of Amsterdam. He is the principal author of the visual datamining tool for genomics, called SARAgene. Bram currently researches high resolution visualization technology over high speed networks. His interests include Self Organizing Feature Maps, Computer Graphics and Virtual Reality.
References (10)
The OptIPuter
Communications of the ACM
(2003)MPEG: a video compression standard for multimedia applications
Communications of the ACM
(1991)Griz: experience with remote visualization over an optical grid
Future Generation Computer Systems
(2003)
Cited by (0)
Bram Stolk is a Virtual Reality specialist with SARA. He received his MSc in Computer Science from the University of Amsterdam. He is the principal author of the visual datamining tool for genomics, called SARAgene. Bram currently researches high resolution visualization technology over high speed networks. His interests include Self Organizing Feature Maps, Computer Graphics and Virtual Reality.
Paul Wielinga is manager of the High Performance Networking business unit at SARA Computing and Networking Services, the e-Science Support Center of the Netherlands.
He has a Masters Degree in Physics from the University of Amsterdam and has over 30 years of IT experience, in the fields of Operating Systems, High Performance Networking and advanced Scientific Visualization. Since 1996 he has been involved in the developments in the field of Visualization and Virtual Reality for scientific and commercial applications. He was manager of the Visualization group at SARA from 1992 till the beginning of 2005. He is the main contact person at SARA for the international OptIPuter collaboration.
Wielinga is also coordinator for Research and Development at SARA.
- 1
Virtual Reality Specialist at SARA Computing and Networking Services.
- 2
Business Unit Manager High Performance Networking at SARA Computing and Networking Services.