skip to main content
10.1145/2616498.2616501acmotherconferencesArticle/Chapter ViewAbstractPublication PagesxsedeConference Proceedingsconference-collections
research-article

Fast, Low-Memory Algorithm for Construction of Nanosecond Level Snapshots of Financial Markets

Published: 13 July 2014 Publication History

Abstract

We present a fast, low-memory algorithm for constructing an order-by-order level snapshot of financial markets with nanosecond resolution. This new implementation is 20-30x faster than an earlier version of the code. In addition, since message data are retained only for as long as it they are needed, the memory footprint is greatly reduced. We find that even the heaviest days of trading spanning the NASDAQ, NYSE and BATS exchanges can now easily be handled using compute nodes with very modest memory (~ 4 GB). A tradeoff of this new approach is that the ability to efficiently manage large numbers of small files is more critical. We demonstrate how we can accommodate these new I/O requirements using the solid-state storage devices (SSDs) on SDSC's Gordon system.

References

[1]
Conway, B. Wall Street's Need for Trading Speed: The Nanosecond Age, 2011.
[2]
Cochrane, J. Weird stuff in high frequency markets. The Grumpy Economist, 2012.
[3]
Kirilenko, A., Kyle, A. S., Samadi, M. and Tuzun, T. The flash crash: The impact of high frequency trading on an electronic market. Manuscript, U of Maryland (2011).
[4]
Biais, B. and Woolley, P. High frequency trading. Manuscript, Toulouse University, IDEI (2011).
[5]
NANEX, LLC Nanex Flash Crash Summary Report, http://www.nanex.net/FlashCrashFinal/FlashCrashSummary.html, 2010.
[6]
Hasbrouck, J. and Saar, G. Low-latency trading. Journal of Financial Markets, 16, 4 (2013), 646--679.
[7]
Lo, A. W. and MacKinlay, A. C. Stock market prices do not follow random walks: Evidence from a simple specification test. Review of financial studies, 1, 1 (1988), 41--66.
[8]
Gai, J., Choi, D. J., O'Neal, D., Ye, M. and Sinkovits, R. S. Fast construction of nanosecond level snapshots of financial markets. ACM, 2013.
[9]
NASDAQ Reference Manual for TotalView-ITCH 2.0, 2006.
[10]
NASDAQ Reference Manual for TotalView-ITCH 3.0, 2008.
[11]
NASDAQ Reference Manual for TotalView-ITCH 4.0f, 2009.
[12]
NASDAQ Reference Manual for TotalView-ITCH 4.0, 2010.
[13]
NASDAQ Reference Manual for TotalView-ITCH 4.1, 2011.
[14]
Szabó, B. Parallel::ForkManager - A simple parallel processing fork manager.

Cited By

View all
  • (2021)When two worlds collide: Using particle physics tools to visualize the limit order bookJournal of Futures Markets10.1002/fut.2225141:11(1715-1734)Online publication date: 21-Jul-2021

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
XSEDE '14: Proceedings of the 2014 Annual Conference on Extreme Science and Engineering Discovery Environment
July 2014
445 pages
ISBN:9781450328937
DOI:10.1145/2616498
  • General Chair:
  • Scott Lathrop,
  • Program Chair:
  • Jay Alameda
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • NSF: National Science Foundation
  • Drexel University
  • Indiana University: Indiana University

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2014

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. High-frequency trading
  2. parallel computing
  3. performance tuning

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

XSEDE '14

Acceptance Rates

XSEDE '14 Paper Acceptance Rate 80 of 120 submissions, 67%;
Overall Acceptance Rate 129 of 190 submissions, 68%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 01 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2021)When two worlds collide: Using particle physics tools to visualize the limit order bookJournal of Futures Markets10.1002/fut.2225141:11(1715-1734)Online publication date: 21-Jul-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media