Abstract
We report progress toward a mathematical theory of line-drawing interpretation. A working framework is developed, and a variety of tools and techniques are demonstrated within it. We begin with a concise review of related work. Then, we detail our assumptions—the imaging geometry is well approximated by orthographic or perspective projection, the viewpoint is general, the world is composed of piecewise C3 surfaces, and continuous-surface-normal depth discontinuities are the only viewpoint-dependent edges. This is followed by the presentation of a projective mapping which we expect will find considerable use in the extension of results derived for orthographic projection to perspective projection. Next, Whitney's definition of an excellent mapping is reviewed and it is shown that both orthographic and perspective projections of piecewise C3 surfaces are excellent under general viewpoint. It is thus demonstrated that there are basically only two different types of local singularities—“folds” and “cusps”—associated with the projection of a C3 surface. The loci of the singular values are C3 curves in the line drawing, except for cusps at their nonjunction terminations. All previous work based on Whitney's results on excellent mappings has needed to assume stability under perturbation of the viewed surface. This assumption, which is unverifiable by a remote observer, is rendered unnecessary by our results. After characterizing viewpoint-dependent edges as folds and cusps, we turn our attention to the modes of interaction between viewpoint-independent and viewpoint-dependent edges. It is shown that only isolated points on viewpoint-dependent edges may lie on surface-patch boundaries, and that viewpoint-independent and viewpoint-dependent edges are always cotangent at their nonocclusion junctions in line drawings, but have different curvatures. Finally, we investigate continuity constraints imposed on the scene by its line drawing. We show that a viewpoint-independent edge in a line drawing is C3 if and only if the edge is C3 in space, and that a viewpoint-dependent edge in a line drawing is C3 if and only if the surface on which it lies in space is C3 along the edge.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
V.I., Arnol'd, “Singularities of systems of rays”, Russian Mathematical Surveys 38(2), pp. 87–176, 1983.
F., Attneave, “Some informational aspects of visual perception”, Psychological Review 61(3), pp. 183–193, 1954.
H.G., Barrow and J.M., Tenenbaum, “Interpreting line drawings as three-dimensional surfaces”, Artificial Intelligence 17, pp. 75–116, August 1981.
T.O., Binford, “Inferring surfaces from images”, Artificial Intelligence 17, pp. 205–244, August 1981.
M., Brady, “Computational approaches to image understanding”, Computing Surveys 14(1), pp. 3–71, 1982.
M. Brady and A. Yuille, “An extremum principle for shape from contour”, in Proc. 8th Int. Joint Conf. Artif. Intell., Karlsruhe, pp. 969–972, August 1983.
I. Chakravarty, “A generalized line and junction labeling scheme with applications to scene analysis”, IEEE Trans. PAMI-I, no. 2, pp. 202–205, 1979.
M.B., Clowes, “On seeing things”, Artificial Intelligence 2, pp. 79–116, Spring 1971.
S.W., Draper, “The use of gradient and dual space in linedrawing interpretation”, Artificial Intelligence 17, pp. 461–508, August 1981.
D., Gans, Transformations and Geometries. Appleton-Century-Crofts: New York, 1969.
D., Gans, An Introduction to Non-Euclidean Geometry. Academic Press: New York, 1973.
V., Guillemin and A., Pollack, Differential Topology. Prentice-Hall: Englewood Cliffs, NJ, 1974.
D. Hilbert and S. Cohn-Vossen, Geometry and the Imagination. Chelsea: New York, 1952.
M.W., Hirsch, Differential Topology. Springer-Verlag: New York, 1976.
B.K.P., Horn, Robot Vision. MIT Press: Cambridge, 1986.
D.A., Huffman, “Impossible objects as nonsense sentences”. In Machine Intelligence 6, B., Meltzer and D., Michie, eds., Edinburgh Univ. Press: Edinburgh, pp. 295–323, 1971.
D.A., Huffman, “Curvature and creases: A primer on paper”, IEEE Trans. Computers C- 25(10), pp. 1010–1019, 1976.
D.A., Huffman, “A duality concept for the analysis of polyhedral scenes”, and “Realizable configurations of lines in pictures of polyhedra”. In Machine Intelligence 8, E.W., Elcock and D., Michie, eds., Ellis Horwood: Chichester. pp. 475–509, 1977.
T., Kanade, “A theory of origami world”, Artificial Intelligence 13, pp. 279–311, May 1980.
T., Kanade, “Recovery of the three-dimensional shape of an object from a single view”, Anificial Intelligence 17, pp. 409–460, August 1981.
J.M., Kennedy, “Icons and information”. In Media and Symbols: The Forms of Expression, Communication, and Education, D.R., Olson, ed., The National Society for the Study of Education: Chicago, pp. 211–240, 1974.
J.J., Koenderink and A.J.van, Doorn, “The singularities of the visual mapping”, Biological Cybernetics 24, pp. 51–59, 1976.
J.J., Koenderink and A.J.van, Doorn. “The shape of smooth objects and the way contours end”, Perception 11, pp. 129–137, 1982.
J.J., Koenderink, “What does the occluding contour tell us about solid shape?”, Perception 13, pp. 321–330, 1984.
S.J., Lee, R.M., Haralick, and M.C., Zhang, “Understanding objects with curved surfaces from a single perspective view of boundaries”, Artificial Intelligence 26, pp. 145–169, May 1985.
M.M., Lipschutz, Differential Geometry. McGraw-Hill: New York, 1969.
D.G., Lowe and T.O., Binford, “The recovery of three-dimensional structure from image curves”, IEEE Trans. PAMI-7, no. 3, pp. 320–326, 1985.
A.K., Mackworth, “Interpreting pictures of polyhedral scenes”, Artificial Intelligence 4, pp. 121–137, Summer 1973.
A.K., Mackworth, “How to see a simple world: An exegesis of some computer programs for scene analysis”. In Machine Intelligence 8, E.W., Elcock and D., Michie, eds., Ellis Horwood: Chichester, pp. 510–537, 1977.
J. Malik, “Interpreting line drawings of curved objects”. Ph.D. Dissertation, Computer Science Dept., Stanford Univ., December 1985.
D., Marr, Vision. Freeman: San Francisco, 1982.
J.N., Mather, “Generic Projections”, Ann. Math. 98, pp. 226–245, 1973.
V.S. Nalwa, “Line-drawing interpretation: Straight lines and conic sections”, IEEE Trans. PAMI-10, no. 4, 1988.
M.H., Pirenne, Optics Painting and Photography. Cambridge Univ. Press: Cambridge, 1970.
R., Shapira and H., Freeman, “Computer description of bodies bounded by quadric surfaces from a set of imperfect projections”, IEEE Trans. Computers C- 27(9), pp. 841–854, 1978.
M., Spivak, Calculus on Manifolds. Benjamin/Cummings: New York, 1965.
K., Sugihara, “A necessary and sufficient condition for a picture to represent a polyhedral scene”, IEEE Trans. PAMI-6, no. 5, pp. 578–586, 1984.
K. Turner, “Computer perception of curved objects using a television camera”. Ph.D. Dissertation. School of Artificial Intelligence, Univ. Edinburg, 1974.
D., Waltz, “Understanding line drawings of scenes with shadows”. In The Psychology of Computer Vision, P.H., Winston, ed., McGraw-Hill: New York, pp. 19–91, 1975.
H., Whitney, “On singularities of mappings of Euclidean spaces. I. Mappings of the plane into the plane”, Ann. Math. 62(3), pp. 374–410, 1955.
Author information
Authors and Affiliations
Additional information
This work was supported in part by the Defense Advanced Research Projects Agency under Contract N00039-84-C-0211.
Rights and permissions
About this article
Cite this article
Nalwa, V.S. Line-drawing interpretation: A mathematical framework. Int J Comput Vision 2, 103–124 (1988). https://doi.org/10.1007/BF00133696
Issue Date:
DOI: https://doi.org/10.1007/BF00133696