Skip to main content
Springer Nature Link
Log in

Neural Space-Filling Curves

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13667))

Included in the following conference series:

  • 2981 Accesses

Abstract

We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan. Instead, our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. The resulting Neural SFC is optimized for an objective suitable for the downstream task when the image is traversed along with the scan line order. We show the advantage of using Neural SFCs in downstream applications such as image compression. Project page: https://hywang66.github.io/publication/neuralsfc.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Alexandrov, V., Alexeev, A., Gorsky, N.: A recursive algorithm for pattern recognition. In: Proceedings of IEEE International Conference Pattern Recognition, pp. 431–433 (1982)

    Google Scholar 

  2. Ansari, A., Fineberg, A.: Image data compression and ordering using Peano scan and lot. IEEE Trans. Consumer Electron. 38(3), 436–445 (1992)

    Article  Google Scholar 

  3. Bader, M.: Space-filling curves: an introduction with applications in scientific computing, vol. 9. Springer Science & Business Media (2012). https://doi.org/10.1007/978-3-642-31046-1

  4. Chen, H., He, B., Wang, H., Ren, Y., Lim, S.N., Shrivastava, A.: NeRV: neural representations for videos. Adv. Neural Inf. Process. Syst. 34, 21557–21568 (2021)

    Google Scholar 

  5. Dafner, R., Cohen-Or, D., Matias, Y.: Context-based space filling curves. In: Computer Graphics Forum, vol. 19, pp. 209–218. Wiley Online Library (2000)

    Google Scholar 

  6. Dai, H., Khalil, E.B., Zhang, Y., Dilkina, B., Song, L.: Learning combinatorial optimization algorithms over graphs. NIPS (2017)

    Google Scholar 

  7. Deudon, M., Cournut, P., Lacoste, A., Adulyasak, Y., Rousseau, L.-M.: Learning heuristics for the TSP by policy gradient. In: van Hoeve, W.-J. (ed.) CPAIOR 2018. LNCS, vol. 10848, pp. 170–181. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93031-2_12

    Chapter  Google Scholar 

  8. Drozdowski, M.: Scheduling for parallel processing, vol. 18. Springer (2009). https://doi.org/10.1007/978-1-84882-310-5

  9. Ehrlich, M., et al.: Leveraging bitstream metadata for fast and accurate video compression correction. arXiv preprint arXiv:2202.00011 (2022)

  10. Harary, F., Norman, R.Z.: Some properties of line digraphs. Rendiconti del Circolo Matematico di Palermo 9(2), 161–168 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hilbert, D.: Über die stetige abbildung einer linie auf ein flächenstück. In: Dritter Band: Analysis\(\cdot \) Grundlagen der Mathematik\(\cdot \) Physik Verschiedenes, pp. 1–2. Springer (1935). https://doi.org/10.1007/978-3-662-38452-7_1

  12. Hopfield, J.J., Tank, D.W.: “neural” computation of decisions in optimization problems. Biol. Cybern. 52(3), 141–152 (1985). https://doi.org/10.1007/BF00339943

  13. Huffman, D.A.: A method for the construction of minimum-redundancy codes. Proc. IRE 40(9), 1098–1101 (1952)

    Article  MATH  Google Scholar 

  14. Kamata, S., Eason, R.O., Kawaguchi, E.: An implementation of the Hilbert scanning algorithm and its application to data compression. IEICE Trans. Inf. Syst. 76(4), 420–428 (1993)

    Google Scholar 

  15. Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401–4410 (2019)

    Google Scholar 

  16. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. ICLR (2016)

    Google Scholar 

  17. Kool, W., Van Hoof, H., Welling, M.: Attention, learn to solve routing problems! ICLR (2019)

    Google Scholar 

  18. Lawder, J.K.: Calculation of mappings between one and n-dimensional values using the Hilbert space-filling curve. School of Computer Science and Information Systems, Birkbeck College, University of London, London Research Report BBKCS-00-01 August (2000)

    Google Scholar 

  19. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  20. Lee, J.H., Hsueh, Y.C.: Texture classification method using multiple space filling curves. Patt. Recogn. Lett. 15(12), 1241–1244 (1994)

    Article  Google Scholar 

  21. Lempel, A., Ziv, J.: Compression of two-dimensional data. IEEE Trans. Inf. Theory 32(1), 2–8 (1986)

    Article  Google Scholar 

  22. Li, Y., et al.: TGIF: a new dataset and benchmark on animated gif description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4641–4650 (2016)

    Google Scholar 

  23. Lieberman-Aiden, E., et al.: Comprehensive mapping of long-range interactions reveals folding principles of the human genome. Science 326(5950), 289–293 (2009)

    Google Scholar 

  24. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (December 2015)

    Google Scholar 

  25. Matias, Y., Shamir, A.: A video scrambling technique based on space filling curves. In: Conference on the Theory and Application of Cryptographic Techniques, pp. 398–417. Springer (1987). https://doi.org/10.1007/3-540-48184-2_35

  26. Moon, B., Jagadish, H.V., Faloutsos, C., Saltz, J.H.: Analysis of the clustering properties of the Hilbert space-filling curve. IEEE Trans. knowl. Data Eng. 13(1), 124–141 (2001)

    Article  Google Scholar 

  27. Moore, E.H.: On certain crinkly curves. Trans. Am. Math. Soc. 1(1), 72–90 (1900)

    Article  MathSciNet  MATH  Google Scholar 

  28. Munroe, R.: xkcd: Map of the internet. https://xkcd.com/195 (2006-12-11). Accessed 16 Nov 2021

  29. Oord, A.V.D., Vinyals, O., Kavukcuoglu, K.: Neural discrete representation learning. NIPS (2017)

    Google Scholar 

  30. Ouni, T., Lassoued, A., Abid, M.: Gradient-based space filling curves: application to lossless image compression. In: 2011 IEEE International Conference on Computer Applications and Industrial Electronics (ICCAIE), pp. 437–442. IEEE (2011)

    Google Scholar 

  31. Peano, G.: Sur une courbe, qui remplit toute une aire plane. Mathematische Annalen 36(1), 157–160 (1890)

    Article  MathSciNet  MATH  Google Scholar 

  32. Prim, R.C.: Shortest connection networks and some generalizations. Bell Syst. Tech. J. 36(6), 1389–1401 (1957)

    Article  Google Scholar 

  33. Ramesh, A., et al.: Zero-shot text-to-image generation. ICML (2021)

    Google Scholar 

  34. Razavi, A., Oord, A.V.D., Vinyals, O.: Generating diverse high-fidelity images with VQ-VAE-2. NeurIPS (2019)

    Google Scholar 

  35. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27(3), 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  36. Sierpínski, W.: Sur une nouvelle courbe continue qui remplit toute une aire plane. Bull. Acad. Sci. Cracovie (Sci. math. et nat. Serie A), pp. 462–478 (1912)

    Google Scholar 

  37. Thyagarajan, K., Chatterjee, S.: Fractal scanning for image compression. In: Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers, pp. 467–468. IEEE Computer Society (1991)

    Google Scholar 

  38. Veličkovič, P., Cucurull, G., Casanova, A., Romero, A., Lió, P., Bengio, Y.: Graph attention networks. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=rJXMpikCZ

  39. Vinyals, O., Bengio, S., Kudlur, M.: Order matters: sequence to sequence for sets. ICLR (2015)

    Google Scholar 

  40. Vinyals, O., Fortunato, M., Jaitly, N.: Pointer networks. NIPS (2015)

    Google Scholar 

  41. Welch, T.A.: Technique for high-performance data compression. Computer (1984)

    Google Scholar 

  42. Witten, I.H., Neal, R.M., Cleary, J.G.: Arithmetic coding for data compression. Commun. ACM 30(6), 520–540 (1987)

    Article  Google Scholar 

  43. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

  44. Yoo, A.B., Jette, M.A., Grondona, M.: SLURM: simple Linux utility for resource management. In: Feitelson, D., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 44–60. Springer, Heidelberg (2003). https://doi.org/10.1007/10968987_3

    Chapter  Google Scholar 

  45. Zhou, L., Johnson, C.R., Weiskopf, D.: Data-driven space-filling curves. IEEE Trans. Visual. Comput. Graph. 27(2), 1591–1600 (2020)

    Article  Google Scholar 

  46. Zhu, J., Hoorfar, A., Engheta, N.: Bandwidth, cross-polarization, and feed-point characteristics of matched Hilbert antennas. IEEE Antennas Wireless Propag. Lett. 2, 2–5 (2003)

    Article  Google Scholar 

  47. Ziv, J., Lempel, A.: Compression of individual sequences via variable-rate coding. IEEE Trans. Inf. Theory 24(5), 530–536 (1978)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Amazon Research Award to AS.

Author information

Authors and Affiliations

  1. University of Maryland, College Park, USA

    Hanyu Wang, Kamal Gupta, Larry Davis & Abhinav Shrivastava

Authors
  1. Hanyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamal Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Larry Davis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abhinav Shrivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanyu Wang .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 896 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, H., Gupta, K., Davis, L., Shrivastava, A. (2022). Neural Space-Filling Curves. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13667. Springer, Cham. https://doi.org/10.1007/978-3-031-20071-7_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20071-7_25

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20070-0

  • Online ISBN: 978-3-031-20071-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics