Skip to main content
Springer Nature Link
Log in

Efficient representation of higher-dimensional arrays by dimension transformations

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Array operations are important for large number of scientific and engineering applications. Two-dimensional array operations are prominent in these applications because of their simplicity and good performance. But in practical applications, the number of dimension is large and hence efficient design of multidimensional array operation is an important research issue. In this paper, we propose and evaluate a new data layout to represent a multidimensional array into a two-dimensional array, namely generalized 2-dimensional array (G2A) by dimension transformations. The G2A transforms an n-dimensional array into a two-dimensional array. Hence, it is possible to design less complicated algorithms that improve the data locality. We design efficient algorithms for matrix–matrix addition/subtraction and multiplication using G2A. Both theoretical analysis and experimental results show that the proposed scheme outperforms the traditional multidimensional array-based algorithms. This is because of the efficient index computation and improved data locality of G2A for better cache performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Sarawagi S, Stonebraker M (1994) Efficient organization of large multidimensional arrays. In: Proceedings of the 10th International Conference on Data Engineering (ICDE). Houston, Texas, pp 328–386

  2. Zhao Y, Deshpande P, Naughton JF (1997) An array-based algorithm for simultaneous multidimensional aggregates. In: Proceedings of the SIGMOD Conference, pp 159–170

  3. Lin C-Y, Liu J-S, Chung Y-C (2002) Efficient representation scheme for multidimensional array operations. IEEE Trans Comput 51(3):327–345

    Article  MathSciNet  Google Scholar 

  4. Stonebraker M, Brown P, Poliakov A, Raman S (2011) The architecture of SciDB. In: Proceedings of the 23rd International Conference on Scientific and Statistical Database Management, pp 1–16

  5. de Carvalho Junior FH, Rezende CA, de Carvalho Silva J, Mahalhães FJL, Juaçaba-Neto RC (2013) On the performance of multidimensional array representations in programming languages based on virtual execution machines. In: Proceedings of the Brazilian symposium on programming languages, pp 31–45

  6. Otoo EJ, Wang H, Nimako G (2014) New approaches to storing and manipulating multi-dimensional sparse arrays. In: Proceedings of the SSDBM’14

  7. Carr S, McKinley KS, Tseng C-W (1994) Compiler optimizations for improving data locality. In: Proceedings of the Sixth International Conference on Architectural Support for Programming Languages and Operating Systems, pp 252–262

  8. McKinley KS, Carr S, Tseng C-W (1996) Improving data locality with loop transformations. ACM Trans Program Lang Syst TOPLAS 18(4):424–453

    Article  Google Scholar 

  9. Masudul Ahsan SM, Azharul Hasan KM (2011) An implementation scheme for multidimensional extendable array operations and its evaluation. In: Proceedings of the ICIEIS, part III, CCIS 253, pp 136–150

  10. Steinbach M, Ertoz L, Kumar V (2004) The challenges of clustering high dimensional data, new directions in statistical physics. Springer, Berlin

    MATH  Google Scholar 

  11. Acar E, Yener B (2009) Unsupervised multiway data analysis: a literature survey. IEEE Trans Knowl Data Eng 21(1):6–20

    Article  Google Scholar 

  12. Bader BW, Kolda TG (2007) Efficient MATLAB computations with sparse and factored tensors. SIAM J Sci Comput 30(1):205–231

    Article  MathSciNet  MATH  Google Scholar 

  13. Kolda TG, Bader BW (2009) Tensor decompositions and applications. SIAM Rev 51(3):455–500

    Article  MathSciNet  MATH  Google Scholar 

  14. Sedaghati N, Mu T, Pouchet L-N, Parthasarathy S, Sadayappan P (2015) Automatic selection of sparse matrix representation on GPUs. In: Proceedings of the ICS’15, pp 99–108

  15. Shoshani A (1997) OLAP and statistical databases. In: Proceedings of the 16th ACM SIGACT-SIGMOD-SIGART symposium on principles of databases systems, pp 185–196

  16. Azharul Hasan KM, Shaikh MAH (2015) Representing higher dimensional arrays into a generalized two dimensional array. In: Proceedings of the 16th International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT 15). Springer, pp 39–46

  17. Bastien F, Bergeron A, Klöcknerb A, Vincent P, Bengio Y (2011) A common GPU n-dimensional array for Python and C. In: Proceedings of big learning workshop, NIPS

  18. Sung I-J, Liu GD, Hwu WMW (2012) DL: a data layout transformation system for heterogeneous computing. In: Innovative parallel computing (InPar)

  19. Koza Z, Matyka M, Szkoda S, Mirosław Ł (2014) Compressed multi-row storage format for sparse matrices on graphics processing units. SIAM J Sci Comput 36(2):219–239

    Article  MathSciNet  MATH  Google Scholar 

  20. Kennedy K, McKinley KS (2014) Optimizing for parallelism and data locality. In: Proceeding of ACM International Conference on Supercomputing 25th Anniversary Volume, pp 151–162

  21. Hall M, Chame J, Chen C, Shin J, Rudy G, Khan MM (2009) Loop transformation recipes for code generation and auto-tuning. In: 22nd international workshop, LCPC 2009. LNCS 5898, pp 50–64

  22. Cong J, Zhang P , Zou Y (2012) Optimizing memory hierarchy allocation with loop transformations for high-level synthesis. In: Proceedings of the 49th Annual Design Automation Conference, pp 1233–1238

  23. Beylkin G, Mohlenkamp MJ (2005) Algorithms for numerical analysis in high dimensions. SIAM J Sci Comput 26:2133–2159

    Article  MathSciNet  MATH  Google Scholar 

  24. Kolda TG, Bader BW, Kenny JP (2005) Higher-order web link analysis using multilinear algebra. In: Proceedings of the 5th IEEE International Conference on Data Mining, IEEE Computer Society Press, pp 242–249

  25. Azharul Hasan KM, Tsuji T, Higuchi K (2011) An efficient MOLAP basic data structure and its evaluation. In: Proceedings of the DASFAA, LNCS 4443. Springer, pp 288–299

  26. Azharul Hasan KM, Kuroda M, Azuma N, Tsuji T, Higuchi K (2005) An extendible array based implementation of relational tables for multi dimensional data bases. In: Proceedings of the 7th International Conference on Data Warehousing and Knowledge Discovery (DaWak’05), Copenhagen, Denmark, LNCS 3580. Springer, pp 233–242

  27. Zhang Y, Zhou X, Zhang Y, Zhang Y, Su M, Wang S (2016) Virtual denormalization via array index reference for main memory OLAP. IEEE Trans Knowl Data Eng 28(4):1061–1074

    Article  Google Scholar 

  28. Yan J, Liu N, Yan S, Yang Q, Fan W, Wei W, Chen Z (2011) Trace-oriented feature analysis for large-scale text data dimension reduction. IEEE Trans Knowl Data Eng 23(7):1103–1117

    Article  Google Scholar 

  29. Sun J, Tao D, Papadimitriou S, Yu PS, Faloutsos C (2008) Incremental tensor analysis: theory and applications. ACM Trans Knowl Discov Data 2(3):1–37

    Article  Google Scholar 

  30. Soroush E, Balazinska M (2011) ArrayStore: a storage manager for complex parallel array processing. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp 253–264

  31. Shaikh MAH, Azharul Hasan KM (2015) Efficient storage scheme for n-dimensional sparse array: GCRS/GCCS. In: Proceedings of the International Conference on High Performance Computing and Simulation, pp 137–142

  32. Kroonenberg PM (2008) Applied multiway data analysis. Wiley, New York

    Book  MATH  Google Scholar 

  33. Solo A (2010) Multidimensional matrix mathematics: multidimensional matrix equality, addition, subtraction, and multiplication, part 2 of 6. In: Proceedings of the world congress on engineering 2010 vol III WCE 2010, London, UK

  34. Intel\(\textregistered \) 64 and IA-32 architectures optimization reference manual. http://www.intel.com/content/dam/doc/manual/64-ia-32-architectures-optimization-manual.pdf

  35. Mano MM (2005) Digital logic and computer design. Prentice Hall, Upper Saddle River

    MATH  Google Scholar 

  36. Jang B, Mistry P, Schaa D, Dominguez R, Kaeli D (2010) Data transformations enabling loop vectorization on multithreaded data parallel architectures. In: Proceedings of the 15th ACM SIGPLAN symposium on principles and practice of parallel programming, pp 353–354

  37. Zhong Y, Dropsho SG, Shen X, Studer A, Ding C (2007) Miss rate prediction across program inputs and cache configurations. IEEE Trans Comput 56(3):328–343

    Article  MathSciNet  Google Scholar 

  38. Deshpande P, Ramasamy K, Shukla A, Naughton JF (1998) Caching multidimensional queries using chunks. In: Proceedings of the ACM SIGMOD Conference on Management of Data, pp 259–270

  39. Otoo EJ, Rotem D, Seshadri S (2007) Optimal chunking of large multidimensional arrays for data warehousing. In: Proceedings of the DOLAP, pp 25–32

  40. Joyner M, Budimlic Z, Sarkar V, Zhang R (2008) Array optimizations for parallel implementations of high productivity languages. In: Proceedings of the PPOPP, pp 1–8

  41. Shaikh MAH, Azharul Hasan KM, Nawaz Ali GGM, Chafii M, Chong PHJ (2016) Efficient matricization of n-D array with CUDA and its evaluation. In: Proceedings of the 19th IEEE International Conference on Computational Science and Engineering (CSE’2016), August 24–26, Paris, France

Download references

Author information

Authors and Affiliations

  1. Khulna University of Engineering & Technology, Khulna, Bangladesh

    K. M. Azharul Hasan & Md Abu Hanif Shaikh

Authors
  1. K. M. Azharul Hasan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Md Abu Hanif Shaikh
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to K. M. Azharul Hasan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azharul Hasan, K.M., Shaikh, M.A.H. Efficient representation of higher-dimensional arrays by dimension transformations. J Supercomput 73, 2801–2822 (2017). https://doi.org/10.1007/s11227-016-1954-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-016-1954-x

Keywords