Skip to main content
SpringerLink
Log in

Envelope-based boundary image matching for smart devices under arbitrary rotations

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Recently, a variety of smart devices have been introduced for entertainment or industrial purposes, and there have been a lot of needs for image matching applications exploiting a large number of images stored in those smart devices. Boundary image matching identifies similar boundary images using their corresponding time-series, and supporting the rotation invariance is crucial to provide more intuitive matching results not only in conventional computing devices but also in smart devices such as smartphones, smart pads, and smart cameras. Computing the rotation-invariant distance between image time-series, however, is a very time-consuming process since it requires a lot of Euclidean distance computations for all possible rotations. We here note that, for smart devices, a very efficient mechanism of computing rotation-invariant distances is required. For this purpose, in this paper we use a novel notion of envelope-based lower bound proposed by Keogh et al. (VLDB J 18:611–630, 2009) to reduce the number of distance computations dramatically. With the help of Keogh et al.’s prior work (Keogh in Proceedings of the 28th International Conference on Very Large Data Bases, 406–417, 2002; Keogh et al. in VLDB J 18:611–630, 2009), we first explain how to construct a single envelope from a query sequence and how to obtain a lower bound of the rotation-invariant distance using the envelope. We then explain that the single envelope lower bound can reduce a number of distance computations. This single envelope approach, however, may cause bad performance since it may incur a larger lower bound due to considering all possible rotated sequences in a single envelope. To solve this problem, we present a concept of rotation interval, and using the concept of multiple envelopes proposed by Keogh et al. (VLDB J 18:611–630, 2009) with these rotation intervals, we then generalize the envelope-based lower bound by exploiting multiple envelopes rather than a single envelope. We also propose equi-width and envelope-minimization divisions as the method of determining rotation intervals in the multi-envelope approach. We further present an advanced multi-step matching algorithm that progressively prunes search spaces by dividing the rotation interval in half. Experimental results show that our envelope-based solutions outperform naive solutions by one to three orders of magnitude. We believe that this performance improvement makes our algorithms very suitable for smart devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Besides the range query of Definition 2, the \(k\)-nearest neighbor (\(k\)-NN) query is also widely used. However, we can evaluate \(k\)-NN queries using range queries because we can regard the distances for current \(k\) candidates as the tolerances of range queries. Thus, in this paper we focus on the range query whose inputs are a query sequence and the tolerance.

  2. Many major concepts including (multiple) envelopes and envelope-based lower bounds come from Keogh et al.’s work [18, 19]. We thank Keogh and his colleagues for their valuable contributions. We do our best to mention their contributions throughout the paper.

  3. The concept of multiple envelopes was originally proposed by Keogh et al. [19], and in this paper we used their concept with slight modification for rotation intervals.

  4. In Sect. 7, we determine \(m\) through extensive experiments. Finding a theoretical optimal \(m\) is another challenging issue since it varies by types of boundary images and lengths of sequences, and we leave this issue as a further study.

  5. In RI-MS, we compute all possible query envelopes in advance and maintain them in main memory of smart devices. Without loss of generality, let us assume that the sequence length \(n\) is \(2^n\). Then, in RI-MS, the number of envelopes will be \((2^k - 1)\,(= 2^0 + 2^1 + \cdots + 2^{k-1})\). That is, the number of all envelopes becomes \((n-1)\), and if \(n < 2^k\), the number will be less than \((n-1)\). For each envelope, we maintain two sequences \(U\) and \(L\), each of which consists of \(n\) entries. Thus, we need to maintain total \(2n^2\) entries (=\(n\) envelopes \(\times \) \(2n\) entries) for a query sequence. For example, if the sequence length is \(360\) and each entry requires four bytes, we then require 1,036,800 bytes to store all the envelopes, which is \(<\)1 MB and is small enough to be maintained in main memory, even for smart devices with limited memory space. Therefore, we maintain all possible envelopes in main memory through the pre-computation, which is negligible in the overall matching performance since the usual environment contains a huge number of data sequences to be compared.

  6. WEB_DATA can be publicly downloaded at http://cs.kangwon.ac.kr/~ysmoon/zips/web-data.zip.

References

  1. Abbasi, S., Mokhtarian, F., Kittler, J.: Search for similar shapes in the SQUID system: shape queries using image databases. http://www.ee.surrey.ac.uk/CVSSP/demos/css/demo.html (2005)

  2. Agrawal, R., Faloutsos, C., Swami, A.: Efficient similarity search in sequence databases. In: Proceedings of the 4th International Conference on Foundations of Data Organization and Algorithms, pp. 69–84. Chicago, Oct 1993

  3. Boas, P.V.E., Kaas, R., Zulstra, E.: Design and implementation of an efficient priority queue. Theory Comput. Syst. 10(1), 99–127 (1997)

    Google Scholar 

  4. Boulos, M., Wheeler, S., Tavares, C., Jones, R.: How smartphones are changing the face of mobile and participatory healthcare: an overview with examples from eGAALYX. BioMed. Eng. Online 10(1), 24–38 (2011)

    Article  Google Scholar 

  5. Bu, Y., Chen, L., Fu, A.W.-C., Liu, D.: Efficient anomaly monitoring over moving object trajectory streams. In: Proceedings of the 15th International Conference on Knowledge Discovery and Data Mining, pp. 159–168. ACM SIGKDD, Paris, June 2009

  6. Chan, K.-P., Fu, A.W.-C., Yu, C.T.: Haar wavelets for efficient similarity search of time-series: with and without time warping. IEEE Trans. Knowl. Data Eng. 15(3), 686–705 (2003)

    Article  Google Scholar 

  7. Cho, H.: Distributed multidimensional clustering based on spatial correlation in wireless sensor networks. Int. J. Comput. Syst. Sci. Eng. 26(4), 275–283 (2011)

    Google Scholar 

  8. Choi, Y., Tak, Y., Rho, S., Hwang, E.: Skin feature extraction and processing model for statistical skin age estimation. Multimed. Tools Appl. 64(2), 227–247 (2013)

    Article  Google Scholar 

  9. Do, M.N.: Wavelet-based texture retrieval using generalized Gaussian density and Kullback–Leibler distance. IEEE Trans. Image Process. 11(2), 146–158 (2002)

    Article  MathSciNet  Google Scholar 

  10. Esling, P., Agon, C.: Time-series data mining. ACM Comput. Surv. 45(1), 1–34 (2013). (article 12)

    Article  Google Scholar 

  11. Faloutsos, C., Ranganathan, M., Manolopoulos, Y.: Fast subsequence matching in time-series databases. In: Proceedings of International Conference on Management of Data, pp. 419–429. ACM SIGMOD, Minneapolis, May 1994

  12. Gao, X., Qiu, B., Shen, J.J., Ng, T.-T., Shi, Y.Q.: A smart phone image database for single image recapture detection. In: Proceedings of the 9th Internationall Workshop on Digital Watermarking, pp. 90–104. Seoul, Oct 2011

  13. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 2nd edn. Prentice Hall, New Jersey (2002)

    Google Scholar 

  14. Han, W.-S., Lee, J., Moon, Y.-S., Jiang, H.: Ranked subsequence matching in time-series databases. In: Proceedings of the 33rd Internationall Conference on Very Large Data Bases, Vienna, pp. 423–434, Sept 2007

  15. Han, W.-S., Lee, J., Moon, Y.-S., Hwang, S.-W., Yu, H.: A new approach for processing ranked subsequence matching based on ranked union. In: Proceedings of International Conference on Management of Data. ACM SIGMOD, Athens, June 2011

  16. Kashyap, S., Lee, M.L., Hsu, W.: Similar subsequence search in time series databases. In: Proceedings of the 22nd International Conference on Database and Expert Systems Applicaitons, Toulouse, pp. 232–246, Aug 2011

  17. Kekre, H.B., Thepade, S.D., Chaturvedi, R.N.: Block based information hiding using Cosine, Hartley, Walsh and Haar wavelets. Int. J. Adv. Comput. Res. 3(1), 1–6 (2013)

    Article  Google Scholar 

  18. Keogh, E.J.: Exact indexing of dynamic time warping. In: Proceedings of the 28th International Conference on Very Large Data Bases, Hong Kong, pp. 406–417, Aug 2002

  19. Keogh, E.J., Wei, L., Xi, X., Vlachos, M., Lee, S.-H., Protopapas, P.: Supporting exact indexing of arbitrarily rotated shapes and periodic time series under Euclidean and Warping distance measures. VLDB J. 18(3), 611–630 (2009)

    Article  Google Scholar 

  20. Kim, B.-S., Moon, Y.-S., Kim, J.: Noise control boundary image matching using time-series moving average transform. In: Proceedings of the 19th International Conference on Database and Expert Systems Applications, Turin, pp. 362–375, Sept 2008

  21. Kim, S.-P., Moon, Y.-S., Hong, S.-K.: An envelope-based approach to rotation-invariant boundary image matching. In: Proceedings of the 13th International Conference on Data Warehousing and Knowledge Discovery, Toulouse, pp. 382–393, Aug 2011

  22. Kim, M., Whang, K.-Y., Moon, Y.-S.: Horizontal reduction: instance-level dimensionality reduction for similarity search in large document databases. In: Proceedings of the 28th IEEE International Conference on Data Engineering, Washington DC, pp. 1061–1072, April 2012

  23. Kong, F., Tan, J.: DietCam: automatic dietary assessment with mobile camera phones. Pervas. Mobile Comput. 8(1), 147–163 (2012)

    Article  Google Scholar 

  24. Lee, A.J.T., Lin, C.-W., Lo, W.-H., Chen, C.-C., Chen, J.-X.: A novel filtration method in biological sequence databases. Pattern Recognit. Lett. 28(4), 447–458 (2007)

    Article  Google Scholar 

  25. Lee, A.J.T., Wu, H.W., Lee, T.-Y., Liu, Y.-H., Chen, K.-T.: Mining closed patterns in multi-sequence time-series databases. Data Knowl. Eng. 68(10), 1071–1090 (2009)

    Article  Google Scholar 

  26. Lee, S., Rho, S., Park, J.: Multimedia contents adaptation by modality conversion with user preference in wireless network. J. Netw. Comput. Appl. 37(1), 1–8 (2014)

    Google Scholar 

  27. Lim, H.-S., Whang, K.-Y., Moon, Y.-S.: Similar sequence matching supporting variable-length and variable-tolerance continuous queries on time-series data stream. Inf. Sci. 178(6), 1461–1478 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  28. Lin, C.-H., Lin, W.-C.: Image retrieval system based on adaptive color histogram and texture features. Comput. J. 54(7), 1136–1147 (2011)

    Article  Google Scholar 

  29. Loh, W.-K., Park, Y.-H., Yoon, Y.-I.: Fast recognition of asian characters based on database methodologies. In: Proceedings of the 24th British National Conference on Databases, Glasgow, pp. 37–48, July 2007

  30. Loh, W.-K., Moon, Y.-S., Srivastava, J.: Distortion-free predictive streaming time-series matching. Inf. Sci. 180(8), 1458–1476 (2010)

    Article  Google Scholar 

  31. Moon, Y.-S., Whang, K.-Y., Han, W.-S.: General match: a subsequence matching method in time-series databases based on generalized windows. In: Proceedings of International Conference on Management of Data, pp. 382–393. ACM SIGMOD, Madison, June 2002

  32. Moon, Y.-S., Kim, J.: Efficient moving average transform-based subsequence matching algorithms in time-series databases. Inf. Sci. 177(23), 5415–5431 (2007)

    Article  MATH  Google Scholar 

  33. Moon, Y.-S., Kim, H.-S., Kim, S.-P., Bertino, E.: Publishing time-series data under preservation of privacy and distance orders. In: Proceedings of the 21th International Conference on Database and Expert Systems Application, Bilbao, pp. 17–31, Sept 2010

  34. Moon, Y.-S., Kim, B.-S., Kim, M.S., Whang, K.-Y.: Scaling-invariant boundary image matching using time-series matching techniques. Data Knowl. Eng. 69(10), 1022–1042 (2010)

    Article  Google Scholar 

  35. Nam, Y., Rho, S., Park, J.: Inference topology of distributed camera networks with multiple cameras. Multimed. Tools Appl. 67(1), 289–309 (2013)

    Article  Google Scholar 

  36. Peng, P., Shou, L., Chen, K., Chen, G., Wu, S.: The knowing camera, recognizing places-of-interest in smartphone photos. In: Proceedings of the 36th International Conference on Research and Development in Information Retrieval, Dublin, pp. 969–972, July 2013

  37. Pratt, W.K.: Digital Image Processing, 4th edn. Eastman Kodak Company, Rochester (2007)

    Book  MATH  Google Scholar 

  38. Rasheed, F., Al-Shalalfa, M., Alhajj, R.: Efficient periodicity mining in time series databases using suffix trees. IEEE Trans. Knowl. Data Eng. 23(1), 79–94 (2011)

    Article  Google Scholar 

  39. Scully, C.G., et al.: Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Trans. Biomed. Eng. 59(2), 303–306 (2012)

    Article  Google Scholar 

  40. Theoharatos, C.: A generic scheme for color image retrieval based on the multivariate Wald–Wolfowitz test. IEEE Trans. Knowl. Data Eng. 17(6), 808–819 (2005)

    Article  Google Scholar 

  41. Vlachos, M., Vagena, Z., Yu, P.S., Athitsos, V.: Rotation invariant indexing of shapes and line drawings. In: Proceedings of ACM Conference on Information and Knowledge Management, Bremen, pp. 131–138, Oct 2005

  42. Wang, Z., Chi, Z., Feng, D., Wang, Q.: Leaf image retrieval with shape features. In: Proceedings of the 4th International Conference on Advances in Visual Information Systems, Lyon, pp. 477–487, Nov 2000

  43. Yang, X., Bai, X., Koknar-Tezel, S., Latecki, L.J.: Densifying distance spaces for shape and image retrieval. J. Math. Imaging Vis. 46(1), 12–28 (2013)

    Article  MathSciNet  Google Scholar 

  44. You, J., Park, S., Kim, I.: An efficient frequent melody indexing method to improve the performance of query-by-humming systems. J. Inf. Sci. 34(6), 777–798 (2008)

    Article  Google Scholar 

  45. Zhang, D.Z., Lu, G.: Review of shape representation and description techniques. Pattern Recognit. 37(1), 1–19 (2003)

    Article  Google Scholar 

  46. Zhou, H., Wang, R., Wang, C.: A novel extended local-binary-pattern operator for texture analysis. Inf. Sci. 178(22), 4314–4325 (2008)

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (NRF-2011-0013235).

Author information

Authors and Affiliations

  1. Department of Software Design and Management, Gachon University, 1342 Seongnam-daero, Sujeong-gu, Seongnam-shi, Gyeonggi-do, 461-701, Republic of Korea

    Woong-Kee Loh

  2. Department of Computer Science, Kangwon National University, 192-1 Hyoja2-Dong, Chunchon, Kangwon, 200-701, Republic of Korea

    Sang-Pil Kim, Sun-Kyong Hong & Yang-Sae Moon

Authors
  1. Woong-Kee Loh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sang-Pil Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sun-Kyong Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yang-Sae Moon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang-Sae Moon.

Additional information

The preliminary version of this paper was published in Proc. of the 13th Int’l Conf. on Data Warehousing and Knowledge Discovery (DaWaK 2011), Toulouse, France, pp. 382–393, Aug 2011, and its Korean version was published in The KIPS Transactions: Part D, Vol. 18-D, No. 1, pp. 9–22, Feb 2011.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Loh, WK., Kim, SP., Hong, SK. et al. Envelope-based boundary image matching for smart devices under arbitrary rotations. Multimedia Systems 21, 29–47 (2015). https://doi.org/10.1007/s00530-014-0386-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0386-9

Keywords

Search

Navigation