research-article

Challenges of using wireless network testbeds: a case study on ORBIT

Authors:

WiNTECH '11: Proceedings of the 6th ACM international workshop on Wireless network testbeds, experimental evaluation and characterization

Pages 11 - 18

https://doi.org/10.1145/2030718.2030723

Published: 19 September 2011 Publication History

Get Access

Abstract

Wireless network testbeds have emerged as a valuable alternative to network simulation, but actually using them for experiments presents a number of challenges. We use the ORBIT testbed at Rutgers University, New Jersey, USA as a case study to illustrate the most important issues in this context. We performed several types of evaluation tests on ORBIT, as well as a series of equivalent experiments with up to 50 nodes, which were done also on the QOMB wireless network emulation testbed, and by using the QualNet network simulator. We conclude that a testbed such as ORBIT provides a viable approach to network testing, although this is conditioned by an awareness of the potential pitfalls that may affect experimental results; a series of lessons we learned from our experience is included in the paper. We also show that real-world trials on ORBIT can be broadly reproduced through emulation and even through simulation.

References

[1]

R. Beuran, J. Nakata, T. Okada, L. T. Nguyen, Y. Tan, and Y. Shinoda. A multi-purpose wireless network emulator: QOMET. In Proc. of IEEE International Conference on Advanced Information Networking and Applications (AINA 2008) Workshops, FINA 2008 symposium, pages 223--228, Okinawa, Japan, March 2008.

Digital Library

Google Scholar

[2]

R. Beuran, L. T. Nguyen, T. Miyachi, J. Nakata, K. Chinen, Y. Tan, and Y. Shinoda. QOMB: A wireless network emulation testbed. In Proc. of IEEE Global Communications Conference (GLOBECOM 2009), Honolulu, Hawaii, USA, November-December 2009.

Digital Library

Google Scholar

[3]

R. Beuran, L. T. Nguyen, and Y. Shinoda. QOMB wireless network emulation testbed: Evaluation and case study. In Proc. of ACM International Workshop on Wireless Network Testbeds, Experimental evaluation and Characterization (WiNTECH 2010), pages 1--8, Chicago, Illinois, USA, September 2010.

Digital Library

Google Scholar

[4]

D. S. J. De Couto, D. Aguayo, J. Bicket, and R. Morris. A high-throughput path metric for multi-hop wireless routing. In Proc. of ACM International Conference on Mobile Computing and Networking (MobiCom 2003), San Diego, California, September 2003.

Digital Library

Google Scholar

[5]

T. Miyachi, K. Chinen, and Y. Shinoda. StarBED and SpringOS: Large-scale general purpose network testbed and supporting software. In Proc. of International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2006), Pisa, Italy, October 2006.

Digital Library

Google Scholar

[6]

NICTA. OMF website. http://mytestbed.net/.

Google Scholar

[7]

olsr.org. Olsrd: An adhoc wireless mesh routing daemon. http://www.olsr.org/.

Google Scholar

[8]

D. Raychaudhuri, I. Seskar, M. Ott, S. Ganu, K. Ramachandran, H. Kremo, R. Siracusa, H. Liu, and M. Singh. Overview of the ORBIT radio grid testbed for evaluation of next-generation wireless network protocols. In Proc. of IEEE WCNC 2005, New Orleans, Louisiana, USA, March 2005.

Crossref

Google Scholar

[9]

L. Rizzo. Dummynet: A simple approach to the evaluation of network protocols. ACM SIGCOMM Computer Communication Review, 27(1):31--41, 1997.

Digital Library

Google Scholar

[10]

Scalable Network Technologies, Inc. QualNet Developer. http://www.scalablenetworks.com/.

Google Scholar

[11]

SourceForge. Iperf homepage. http://sourceforge.net/projects/iperf/.

Google Scholar

[12]

WINLAB. ORBIT website. http://www.orbit-lab.org/.

Google Scholar

Cited By

View all

Lunar MSun JWensowitch JFay MTulay HKaranam VQiu BNadig DAttebury GYu HCamp JKoksal CPompili DRamamurthy BHashemi MEkici EVuran M(2022)OneLNKProceedings of the 15th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization10.1145/3477086.3480835(85-92)Online publication date: 31-Jan-2022
https://dl.acm.org/doi/10.1145/3477086.3480835
Mikkelsen LMadsen TSchwefel H(2019)On the Benefits and Challenges of Crowd-Sourced Network Performance Measurements for IoT ScenariosWireless Personal Communications10.1007/s11277-019-06801-4Online publication date: 3-Oct-2019
https://doi.org/10.1007/s11277-019-06801-4
Beuran RMiwa SShinoda YNavda VYoussef M(2012)Making the best of two worldsProceedings of the seventh ACM international workshop on Wireless network testbeds, experimental evaluation and characterization10.1145/2348688.2348704(75-82)Online publication date: 22-Aug-2012
https://dl.acm.org/doi/10.1145/2348688.2348704

Index Terms

Challenges of using wireless network testbeds: a case study on ORBIT
1. General and reference
  1. Cross-computing tools and techniques
    1. Measurement
    2. Metrics

Recommendations

QOMB wireless network emulation testbed: evaluation and case study
WiNTECH '10: Proceedings of the fifth ACM international workshop on Wireless network testbeds, experimental evaluation and characterization

This paper presents several evaluation tests, as well as a case study of large-scale wireless network emulation experiments performed using QOMB, the wireless network emulation testbed at the National Institute of Information and Communications ...
Enabling SDN Experimentation in Network Testbeds
SDN-NFVSec '17: Proceedings of the ACM International Workshop on Security in Software Defined Networks & Network Function Virtualization

Software-defined networking (SDN) has become a popular technology, being adopted in operational networks and being a hot research topic. Many network testbeds today are used to test new research solutions and would benefit from offering SDN ...
Effects of carrier sense modeling on wireless network simulation results
MSWiM '11: Proceedings of the 14th ACM international conference on Modeling, analysis and simulation of wireless and mobile systems

We investigate the effect of radio modeling on the simulation results of low-power wireless networks. In particular we focus on the modeling of carrier sensing and state transition delays and describe how these aspects of low-power radios are captured ...

Comments

Information & Contributors

Information

Published In

WiNTECH '11: Proceedings of the 6th ACM international workshop on Wireless network testbeds, experimental evaluation and characterization

September 2011

118 pages

ISBN:9781450308670

DOI:10.1145/2030718

Program Chairs:
Giuseppe Bianchi
University of Roma Tor Vergata, Italy
,
Joseph Camp
Southern Methodist University -- SMU, Dallas, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 September 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

Mobicom'11

Sponsor:

SIGMOBILE

Mobicom'11: The 17th Annual International Conference on Mobile Computing and Networking

September 19, 2011

Nevada, Las Vegas, USA

Acceptance Rates

Overall Acceptance Rate 63 of 100 submissions, 63%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
146
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Lunar MSun JWensowitch JFay MTulay HKaranam VQiu BNadig DAttebury GYu HCamp JKoksal CPompili DRamamurthy BHashemi MEkici EVuran M(2022)OneLNKProceedings of the 15th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization10.1145/3477086.3480835(85-92)Online publication date: 31-Jan-2022
https://dl.acm.org/doi/10.1145/3477086.3480835
Mikkelsen LMadsen TSchwefel H(2019)On the Benefits and Challenges of Crowd-Sourced Network Performance Measurements for IoT ScenariosWireless Personal Communications10.1007/s11277-019-06801-4Online publication date: 3-Oct-2019
https://doi.org/10.1007/s11277-019-06801-4
Beuran RMiwa SShinoda YNavda VYoussef M(2012)Making the best of two worldsProceedings of the seventh ACM international workshop on Wireless network testbeds, experimental evaluation and characterization10.1145/2348688.2348704(75-82)Online publication date: 22-Aug-2012
https://dl.acm.org/doi/10.1145/2348688.2348704

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

QOMB wireless network emulation testbed: evaluation and case study

Enabling SDN Experimentation in Network Testbeds

Effects of carrier sense modeling on wireless network simulation results

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations