Abstract
Visual computing emhpasizes visibility of information of interest and direct manipulation of visible information. It has become a dominant style of computing in recent years. In this paper, we review three major technologies that provide a platform for visual computing. These technologies reflect the needs of various people who use visual computers: programmers, end users, and scientists. We begin with a discussion of important developments in visual programming and follow with discussions of visual interfaces and visualization. We conclude with a summary of visual computing's current status and identify critical areas of research that should be emphasized in future work.
Similar content being viewed by others
References
Adams E, Muchnick SS (1986) Dbxtool: a window-based symbolic debugger. Software — practice and experience 16:653–669
Airey JM, Rohlf JH, Brooks Jr FP (1990) Towards image realism with interactive update rates in complex virtual building environments. Comput Graph (Proc Symp Interactive 3D Graphics) 24(2):41–50
Albizuri-Romero MB (1984) GRASE: a graphical syntax-direct editor for structured programming. SIGPLAN Notices 19(2):28–37
Argiro V (1990) Seeing in volume. Pixel 1(3):35–39
Azam M, Sobek S, Browne JC (1989) A unified approach to parallel programming. IEEE Software 6(4), 10–18
Beacker R (1975) Two systems which produce animated representations of the execution of computer programs. SIGCSE Bull 7(1):158–167
Baecker R (1981) Sorting out sorting. Dynamics Graphics Project. Computer Systems Research Institute, University of Toronto, Toronto, Canada, 16 mm film, 25 min
Baecker R and Marcus A (1986) Design principles for the enhanced presentation of computer program source text. Proc ACM CHI 86 Conf. on Human Factors in Computing Systems, ACM, New York, pp 51–58
Begoray JA (1990) An introduction to hypermedia issues, systems, and application areas. Int J Man-Machine Studies 33(2):121–147
Biermann AW, Krishnaswamy R (1976) Constructing programs from example computations. IEEE Trans Software Engineering SE-2(3):141–153
Bishop G, Monger M, Ramsey P (1990) A visualization programming environment for multicomputers. IEEE Comput Graph Appl 10(4):50–58
Blaschek G, Sametinger J (1989) User-adaptable pretty-printing. Software Practice and Experience 19(7):687–702
Borning A (1986) Graphically defining new building blocks in Thing Lab. Human-Computer Interaction 2(4):269–295
Bortman H (1992) Is there a pen in your future? MacUser 8(2):144–148
Brennan P (1991) Should we or shouldn't we use spoken commands in voice interfaces. Proc ACM CHI Conf on Human Factors in Computing Systems pp 369–372
Bricken M (1991) Virtual reality learning environments: potentials and challenges. Comput Graph 25(3):178–191
Brooks Jr FP (1986) Walkthrough — a dynamic graphics system for simulating virtual buildings. Proc Workshop on Interactive 3D Graphics, pp 9–21
Brown MH (1988) Perspectives on algorithm animation. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 33–38
Brown MH, Carling RT, Herot CF, Kramlich DA, Souza P (1985) Program visualization: graphical support for software development. IEEE Comput 18(8):27–35
Brown MH, Sedgewick R (1984) A system for algorithm animation. Comput Graph 18(3):177–186
Buell B, Brandt R (1990) The pen: computing's next big lap. Business Week, May 14, 1990
Butterworth J, Davidson A, Hench S, Olano MT (1991) 3dm: A 3-dimensional modeling system. University of North Carolina, Chapel Hill
Buxton W (ed) (1989) Special issue on nonspeech audio. Human-Computer Interaction 4(1)
Buxton W, Gaver W, Bly S (1992) The use of non-speech audio at the interface. Cambridge University Press (in press)
Cardelli L (1988) Building user interfaces by direct manipulation. ACM SIGGRAPH Symp on User Interface Software pp 152–166
Chan PP (1984) Learning considerations in user interface design: the room model. Tech Rep CS-84-16, Dept of Computer Science, University of Waterloo, Ontario, Canada
Chang SK (1987) Visual languages: a tutorial and survey. IEEE Software 29(1):29–39
Chignell MH, Nordhausen B, Valdez JF, Waterworth JA (1992) The HEETI model of text to hypertext conversion. Hypermedia 3(3):187–205
Clark BEJ, Robinson SK (1983) A graphically interacting program mouitor. The Computer Journal 26(3):235–238
Computer (1989) Special issue on visualization in computing. IEEE Comput 22(10)
Cook LT, Dwyer SJ, Batnitzky S, Lee KR (1983), A three dimensional display systems for diagnostic imaging applications. IEEE Comput Graph Appl 3(5):13–19
CORE (1979) Status report of the graphics standards committee. Comput Graph 13(3)
Cunniff N, Taylor RP, Black JB (1986) Does programming language affect the type of conceptual bugs in beginners' programs? A comparison of FPL and PASCAL. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 175–182
Davis AL, Keller RM (1982) Data flow program graphs. IEEE Comput 15(2):26–41
DeFanti TA, Brown MD, McCormick BH (1989) Visualization — expanding scientific and engineering research opportunities. IEEE Comput 22(8):12–25
DeFanti TA, Sandin DJ, Hart JC, Lescinsky G (1990) One picture's worth a thousand gigaflops. Pixel 1(3):40–41
Diaz-Herrera JL, Flude RC (1980) Pascal/HSD: a graphical programmung system. Proc IEEE Compsac, pp 723–728
DiSessa A (1985) A principled design for an intergrated computational environment. Human-Computer Interaction 1(1):1–47
Dongarra J, Brewer O, Kohl JA, Fineberg S (1990) A tool to aid in the design, implementation, and understanding of matrix algorithms for parallel processors. Journal of Parallel and Distributed Computing 9(2):185–202
Drebin RA, Carpenter L, Hanrahan (1988) Volume rendering. Comput Graph 22(4):65–74
Duisberg RA (1987) Visual programming of program visualizations. Proc Workshop on Visual Languages, pp. 55–66
Dyer DS (1990) A data flow toolkit for visualization. IEEE Comput Graph Appl 10(4):60–69
Ellis TO, Heafner JF, Sibley WL (1969) The Grail project: an experiment in man-machine communication. RAND Report RM-5999-Arpa
Fisher SS, MacGreevy M, Humphries J, Robinett W (1986) Virtual environment display system. Proc ACM Workshop on Interactive 3D Graphics, pp 77–87
Foley JD, Kim WC, Kovacevic S, Murray K (1989) Designing interfaces at a high level of abstractions. IEEE Software, Jan, 25–32
Fox EA (ed) (1991) Special issue on standards and the emergence of digital multimedia systems. CACM 34(4)
Frei HP, Weller DL, Williams R (1978) A graphics-based programming support system. Proc ACM SIGGRAPH 78:43–49
Frieder G, Gordon D, Reynolds RA (1985) Back-to-front display of voxel-based objects. IEEE Comput Graph Appl 5:52–60
GKS (1984) Graphical kernel system — ANSI X3H3/83-25r3. Comput Graph, Feb 1984
Glinert EP (1989) Towards software matrices for visual programming. Int J Man-Machine Studies 30(4):425–445
Glinert EP, Tanimoto SL (1986) Pict: an interactive graphical programming environment. IEEE Comput 17(11):7–25
Glinert EP, Kopache ME, McIntyre DW (1990) Exploring the general-purpose visual alternative. J Visual Lang and Computing 1(1):3–39
Glushko RJ (1989) Transforming text into hypertext for a compact disc encyclopedia. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 293–298
Gore A (1992) Pen, voice will shape interface. Mac WEEK 6(5):1
Green M (1990) Virtual reality interface: tools and techniques. Proc Computer Graphics International, pp 51–68
Green M, Jacob RJK (eds) (1991) SIGGRAPH workshop report: software architectures and metaphors for non-WIMP user interfaces. Comput Graph 25 (3):229–235
Green M, Shaw C (1990) The data paper: living in the virtual world. Proc Graphics Interface, pp 123–130
Grimes J, Potel M (eds) (1991) Special issue on multimedia. IEEE Comput Graph Appl 11(4)
Haeberli PE (1988) ConMan: a visual programming language for interactive graphics. Comput Graph 22(4):103–111
Haibit LM (1959) A program to draw multi-level flow charts. Proc Western Joint Computer Conference, pp 131–137
Halbert D (1981) An example of programming by example. Thesis, University of California, Berkley
Henderson DA, Card SK (1986) Rooms: the use of multiple virtual workspaces to reduce space contention in a windowbased graphical user-interface. ACM Trans Graph 5(3):211–243
Herman GT, Udupa JK (1983) Display of 3-D information in 3-D digital images: computational foundations and medical applications. IEEE Comput Graph Appl 3(5):39–46
Herot CF (1980) Spatial management of data. ACM Trans Database Syst 5(4):493–514
Hix D, Schulman RS (1991) Human computer interface development tools: a methodology for their evaluation. CACM 34(3):74–87
Hutchins EL, Hollan JD, Norman DA (1986) Direct manipulation interfaces. In Norman DA, Draper SW (eds) User centered system design. Lawrence Erlbaum, Hillsdale, pp. 87–142
Jacob RJK (1985) A state transitions diagram language for visual programming. IEEE Comput 8(8):51–59
Kamel R (ed) (1990) Special issue on voice in computing. IEEE Comput 23 (8)
Kimura TD, Choi JW, Mack JM (1990) Show and tell: a visual programming language. In: Glinert EP (ed) Visual programming environments: paradigms and systems IEEE Computer Society Press, Los Alamitos, pp 397–404
Kreuger MW (1983) Artificial reality. Addison, Reading
Krishnamoorthy MS, Swaminathan R (1989) Program tools for algorithm animation. Software Practice and Experience 19(6):505–513
Lakin F (1986) Spatial parsing for visual languages. In: Chang SK, Ichikawa T, Ligomenides PA (eds) Visual Languages, Plenum, New York
Lanier J (1989) Invited talk. ACM SIGGRAPH Symp on User Interface Software and Technology, Williamsburgh, Virginia, Nov 13–15
LeBlane TJ, Mellor-Crummey JM, Fowler RJ (1990) Analyzing parallel program executions using multiple views. Journal of Parallel and Distributed Computing 9(2):203–217
Lehr T, Segall Z, Vrsalovic D, Caplan E, Chung AL, Fineman CE (1989) Visualizing performance debugging. IEEE Comput 22(10):38–51
Leiberman H (1992) Mondrian: a teachable graphical editor. MIT Memo, Visible Language Workshop, Media Laboratory, MIT, Boston
Levien R (1986) Visual programming. BYTE, Feb 1986, 135–144
Levoy M (1988) Display of surfaces from volume data. IEEE Comput Graph Appl 8(3):51–58
Levoy M (1990) A hybrid ray tracer for rendering polygon and volume data. IEEE Comput Graph Appl 10(2):33–40
Levoy M, Fuchs H, Pizer SM, Rosenman J, Chaney EL, Sherouse GW, Interrante V, Kiet J (1990) Volume rendering in radiation treatment planning. Proc First Conf on Visualization in Biomedical Computing, pp 4–10
Lewis JB, Koved L, Ling DT (1991) Dialogue structures for virtaal worlds. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 131–136
Lewis TG, Handloser III F, Bose S, Yang S (1989) Prototypes from standard user interface management systems. IEEE Comput, pp 51–60
London RL, Duisberg RA (1985) Animating programs using smalltalk. IEEE Comput 18(8):61–71
Mackinlay J (1986) Automating the design of graphical presentations of relational information. ACM Trans Graph 5(2):110–141
Marcus A, Van Dam A (1991) User-interface developments for the nineties. IEEE Comput 24(9):49–57
Martin GL, Pittmann JA, Wittenburg K, Cohen R, Parish T (1990) Sign here, please. BYTE, July 1990, pp 243–251
Maulsby DL, Witten IH, Kittlitz KA (1989) Metamouse: specifying graphical procedures by example. Comput Graph 23(3):127–136
McCormick BH, DeFanti TA, Brown MA (eds) (1987) Visualization in scientific computing. Comput Graph 21(6)
Murakami K, Taguchi H (1991) Gesture recognition using recurrent neural networks. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 237–242
Myers B (1980) Displaying data structures for interactive debugging. Technical Report CSL-80-7, Xerox, Palo Alto Research Center, Palo Alto
Myers B (1983) INCENSE: a system for displaying data structures. Comput Graph 17(3):115–125
Myers BA (1988) A taxonomy of window manager user interfaces. IEEE Comput Graph Appl, pp 65–84
Myers B (1990a) Taxonomies of visual programming and program visualization. Journal of Visual Programming and Languages 1(1):97–123
Myers B (1990b) Invisible programming. Workshop on Visual Languages, pp 203–208
Myers B (1990c) Creating user interfaces using programming by example, visual programming, and constraints. ACM Trans on Programming Languages and Systems 12(2):143–177
Myers B, Giuse D, Dannenberg R, Vander Zanden B, Kosbic D, Pervin E, Mickish A, Marchal P (1990) Comprehensive support for graphical, highly-interactive user interfaces. IEEE Comput 23(11):71–85
Narasimhalu AD, Christodulakis S (eds) (1991) Special issue on multimedia information systems. IEEE Comput 24(10)
Nassi I and Shneiderman B (1973) Flowchart techniques for structured programming. SIGPLAN Notices 8(8):12–26
Newman WM (1968) A system for interactive graphical programming. Proc of the Spring Joint Computer Conf, pp 47–54
Ni LM, Tai KC (eds) (1990) Special issue on software tools for parallel programming and visualization. Journal of Parallel and Distributed Programming 9(2)
Nielsen J (ed) (1989) Coordinating user interfaces for consistency. Academic Press, San Diego
Nielsen J (1990) Hypertext and hypermedia. Academic Press, New York
Nielson GM (1991) Visualization in scientific and engineering computation. IEEE Comput 24(9):58–66
Olsen DR, Halversen BW (1988) Interface usage measurements in a user interface management system. Proc ACM SIGGRAPH Symp on User Interface Software, pp 102–108
Parasaye K, Chignell MH, Khoshafian S, Wong HKT (1989) Intelligent databases: object-oriented, deductive hypermedia technologies. Wiley, New York
Pausch R (1991) Virtual reality on five dollars a day. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 265–270
Pausch R, Williams RD (1990) Tailor: creating custom user interface based on gesture. Proc ACM SIGGRAPH Symp on User Interface Software and Technology, pp 123–134
PHIGS+ (1988) PHIGS+ functional description revision 3.0. Comput Graph 22(3):125–218
Pietrzykowski T, Matwin S, Muldner T (1983) The programming languge PROGRAPH: yet another application of graphics. Proc Graphics Interface Conf, pp 143–145
Pittman JA (1991) Recognizing handwritten text. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 271–275
Pong MC, Ng N (1983) PIGS — a system for programming with interactive graphical support. Software Practice and Experience 13:847–855
Press L (1990) Compuvision or teleputer. Commun ACM 33(9):29–36
RB2 (1991). VPI Redwood City
Reiss SP (1985) PECAN: program development environment that supports multiple views. IEEE Trans on Software Engineering SE-11(3):276–285
Reiss SP, Meyers S, Duby C (1989) Using GELO to visualize software systems. Proc ACM SIGGRAPH Symp on User Interface Software and Technology, pp 149–158
Rhyne J (1987) Dialogue management for gestural interfaces. Comput Graph 21(2):137–141
Robertson GG, Card SK, Mackinlay JD (1989) The cognitive co-processor architecture for interactive user interfaces. Proc ACM SIGGRAPH Symp on User Interface Software and Technology, pp 10–18
Robins G (1987) The ISI grapher: a portable tool for displaying graphs pictorially. Proc Symbolikka. Aug 1987
Rovner PD, Henderson Jr DA (1969) On the implementation of AMBIT/G: a graphical programming language. Proc IJCAI-69, pp 9–20
Rowe LA, Davis M, Messinger E, Meyer C, Spirakis C, Tuan A (1987) A browser for directed graphs. Software — Practice and Experience 17(1):61–76
Sabella P (1988) A rendering algorithm for visualizing 3D scalar fields. Comput Graph 22(4):51–58
Shaw C, Liang J, Green M, Sun Yungi (1992) The decoupled simulation model for virtual reality systems. Proc ACM CHI Conf on Human Factors in Computing Systems, (in press)
Shen VY, Richter C, Graf ML, Brumfield JA (1990) VERDI: a visual environment for designing distributed systems. Journal of Parallel and Distributed Computing 9(2):128–137
Shimomura T, Isoda S (1991) Linked-list visualization for debugging. IEEE Software, May 1991, pp 44–51
Shneiderman B (1983) Direct manipulation: a step beyond programming languages. IEEE Comput 16(8):57–69
Shu NC (1985a) FORMAL: forms-oriented visual directed application development system. IEEE Comput 18(8):38–49
Shu NC (1985b) Visual programming languages: a dimensional analysis. Proc IEEE International Symp on New Directions in Computing, pp 326–334
Shu NC (1989) Visual programming: prespectives and approaches. IBM Systems Journal 28(4):525–547
Singh G (1990) Vu: visual user-interface design. The Visual Computer 6(4):230–241
Singh G (1991) User interfaces of the future. Proc IT Works Coaf, pp 213–222
Singh G (1992a) Requirements for user interface programming languages. In: Myers BA (ed) Languages for developing user interface. Jotes and Bartlet Boston, pp 215–223
Singh G (1992b) A unified view of object-based virtual environments ISS Memo, Institute of Systems Science, National University of Singapore, Singapore
Singh G, Kok CH, Ngan TY (1990a) Druid: a system for demonstrational rapid user interface development. Proc ACM SIGGRAPH Symp on User Interface Software and Technology, pp 167–177
Singh G, Kok CH, Ngan TY (1990b) Creating graphical interfaces through graphical interaction. Proc IEEE Workshop on Visual Languages, pp 187–193
Singh G, Green M (1991) Automating the lexical and syntactic design of graphical user interfaces: the UofA*UIMS.ACM Trans Graph 10(3):213–254
Singh G, Green M (eds) (1992) Special issue on visual user interface design tools. The Visual Computer 8(2)
Singh G, Serra L (1990) UIMS requirements for multimedia interaction. ACM SIGGRAPH Workshop on Software Architectures and Metaphors for Non-WIMP User Interfaces, Dallas, TX, Aug 5–6
Smith DC (1977) Pygmalion: a computer program to model and stimulate creative thought, Birkhauser, Basel Stuttgart
Smith DC, Irby C, Kimball R, Verplank B, Harslem E (1982) Designing the Star user interface. BYTE 7(4):242–282
Stasko JT (1990) IANGO: a framework and sytem for algorithm animation. IEEE Comput 23(9):27–38
Stasko JT (1991) Using direct manipulation to build algorithm animations by demonstration. Proc ACM CHI Conf on Human Factors in Computing Systems, pp 307–314
Stasko JT, Appelbe WF, Kraemer E (1991) Applying program visualization techniques to and parallel and distributed program development. Technical Report GIT-GVU-91-08, Graphics, Visualization and Usability Center, College of Computing, Georgia Institute of Technology, Atlanta
Stasko JT, Patterson C (1991) Understanding and characterizing program visualization systems. Technical Report GITGVU-91-17, Graphics, Visualization and Usability Center, College of Computing, Georgia Institute of Technology, Atlanta
Suhler PA, Biswas J, Korner KM, Browne JC (1990) TDFL: a task-level dataflow language. Journal of Parallel and Distributed Computing 9(2):103–115
Teitelman W, Masinter L (1981) The Interlisp programming environment. IEEE Comput 14(4):25–34
Tiede U, Hoehne KH, Bomans M, Pommert A, Reimer M, Wiebecke G (1990) Investigation of medical 3D-rendering algorithms. IEEE Comput Graph Appl 10(2):41–53
Upson C, Keeler M (1988) VBUFFER: visible volume rendering. Comput Graph 22(4):59–64
Upson C, Faulhaber Jr, T, Kamins D, Laidlaw D, Schlegel D, Vroom J, Gurwitz R, van Dam A (1989) The application visualization system: a computational environment for scientific visualization. IEEE Comput Graph Appl (9)4:30–42
Upson C (1991) Volumetric visualization techniques. In: Rogers DF, Earnshaw RA (eds) State of the art in computer graphics: visualization and modeling. Springer-Verlag, New York, pp 313–350
VandeWettering M (1990) The application visualization system-AVS 2.0. Pixel 1(3):30–33
Wang C, Koved L, Dukach S (1990) Design for interactive performance in a virtual laboratory. Proc Symp on Interactive 3D Graphics, Comput Graph 24(2):39–40
Wasserman AI (1985) Extending state transition diagrams for the specification of human-computer interaction. IEEE Transactions on Software Engineering SE-11(8):699–713
Waterworth JAW, Chignell MH (1989) A manifesto for hypermedia usability research. Hypermedia 1(3):205–234
Wilhelms J, Gebler AV (1991) A coherent approach for direct volume rendering. Comput Graph 25(4):275–284
William G (1983) The Lisa computer, BYTE 8(2):33–50
WorldToolkit (1991). Sense8. Sausalito
Yao SB, Hevner AR, Shi Z, Luo D (1984) FORMANAGER: a office forms management system. ACM Trans on Office Information Systems 2(3):235–262
Zachary PG (1990) Computer firms see the writting on the screen. WallStreet Journal, April 30
Zimmerman TG, Lanier J (1987) A hand gesture interface device. Proc ACM CHI+GI, pp 189–192
Zloof MM (1977) Query-by-example: a database language. IBM Systems Journal 16(4):324–344
Zloof MM (1981) QBE/OBE: a language for office and business automation, IEEE Comput 14(5):13–22
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Singh, G., Chignell, M.H. Components of the visual computer: A review of relevant technologies. The Visual Computer 9, 115–142 (1992). https://doi.org/10.1007/BF01902552
Issue Date:
DOI: https://doi.org/10.1007/BF01902552