skip to main content
10.1145/1455770.1455806acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
research-article

Privacy oracle: a system for finding application leaks with black box differential testing

Published: 27 October 2008 Publication History

Abstract

We describe the design and implementation of Privacy Oracle, a system that reports on application leaks of user information via the network traffic that they send. Privacy Oracle treats each application as a black box, without access to either its internal structure or communication protocols. This means that it can be used over a broad range of applications and information leaks (i.e., not only Web traffic or credit card numbers). To accomplish this, we develop a differential testing technique in which perturbations in the application inputs are mapped to perturbations in the application outputs to discover likely leaks; we leverage alignment algorithms from computational biology to find high quality mappings between different byte-sequences efficiently. Privacy Oracle includes this technique and a virtual machine-based testing system. To evaluate it, we tested 26 popular applications, including system and file utilities, media players, and IM clients. We found that Privacy Oracle discovered many small and previously undisclosed information leaks. In several cases, these are leaks of directly identifying information that are regularly sent in the clear (without end-to-end encryption) and which could make users vulnerable to tracking by third parties or providers.

References

[1]
http://www.autoitscript.com/autoit3/.
[2]
Marshall Beddoe. The protocol informatics project. http://www4tphi.net/~awaiters/PI/PI.html, 2004.
[3]
Gerald Combs. Wireshark. http://www.wireshark.org.
[4]
Weidong Cui, Vern Paxson, and Nicholas Weaver. Protocol-Independent Adaptive Replay of Application Dialog. In NDSS, 2006.
[5]
Robert B. Evans and Alberto Savoia. Differential testing: a new approach to change detection. In ESEC-FSE posters, 2007.
[6]
Leita Corrado gand Ken Mermoud and Marc Dacier. Scriptgen: an automated script generation tool for honeyd. In ACSAC, December 2005.
[7]
J. W. Hunt and M. D. McIlroy. An algorithm for differential file comparison, 1976.
[8]
IEInspector Software LLC. IEInspector HTTP Analyzer -- HTTP Sniffer, HTTP Monitor, HTTP Trace, HTTP Debug. http://www.ieinspector.com/httpanalyzer/, 2007.
[9]
Marc Fisher II, Sebastian Elbaum, and Gregg Rothermel. Dynamic characterization of web application interfaces. FASE 2007, LNCS, 4422:260--275, 2007.
[10]
Christian Kreibich and Jon Crowcroft. Efficient sequence alignment of network traffic. In IMC, 2006.
[11]
Last Bit Software. RegSnap. http://www.lastbit.com/regsnap/.
[12]
Stephen McCamant and Michael D. Ernst. Quantitative information flow as network flow capacity. In PLDI, 2008.
[13]
Barton P. Miller, Lars Fredriksen, and Bryan So. An empirical study of the reliability of UNIX utilities. CACM, 33(12):32--44, 1990.
[14]
Burkhard Morgenstern, Andreas Dress, and Thomas Werner. Multiple DNA and protein sequence alignment based on segment-to-segment comparison. PNAS, 93(22):12098--12103, October 1996.
[15]
Burkhard Morgenstern, Kornelie Frech, Andreas Dress, and Thomas Werner. Dialign: finding local similarities by multiple sequence alignment. Bioinformatics, 14(3):290--294, 1998.
[16]
S.B. Needleman and C.D. Wunsch. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of Molecular Biology, 1970.
[17]
NMMI. What is my machine Windows name? http://faq.nmmi.edu/fom- serve/cache/338.html, April 2005.
[18]
Objective Development. Little Snitch. http://www.obdev.at/products/littlesnitch/.
[19]
Ruoming Pang, Vinod Yegneswaran, Paul Barford, Vern Paxson, and Larry Peterson. Characteristics of internet background radiation. In IMC, 2004.
[20]
Vern Paxson. Bro: a system for detecting network intruders in real-time. Computer Networks, 31(23--24):2435--2463, 1999.
[21]
T. Scott Saponas, Jonathan Lester, Carl Hartung, Sameer Agarwal, and Tadayoshi Kohno. Devices that tell on you: Privacy trends in consumer ubiquitous computing. In 16th Usenix Security Symposium, August 2007.
[22]
http://yro.slashdot.org/yro/07/12/29/2120202.shtml.
[23]
http://yro.slashdot.org/yro/08/01/03/1630203.shtml.
[24]
Stuart Cheshire and Marc Krochmal. Multicast DNS. http://files.multicastdns.org/draft-cheshire-dnsext-multicastdns.txt, 2006.
[25]
The Canadian Internet Policy and Public Interest Clinic. Digital Rights Management and Consumer Privacy. http://www.cippic.ca, September 2007.
[26]
VIP Defense: privacy and anonymity keeping company. VIP Privacy. http://www.vipdefense.com/.
[27]
http://www.vmware.com/.
[28]
WebSense. WebSense Content Protection Suite. http://www.websense.com/, 2008.
[29]
Heng Yin, Dawn Song, Manuel Egele, Christopher Kruegel, and Engin Kirda. Panorama: capturing system-wide information flow for malware detection and analysis. In CCS, 2007.
[30]
Aydan R. Yumerefendi, Benjamin Mickle, and Landon P. Cox. Tightlip: Keeping applications from spilling the beans. In NSDI, 2007.

Cited By

View all
  • (2024)Securing Personally Identifiable Information: A Survey of SOTA Techniques, and a Way ForwardIEEE Access10.1109/ACCESS.2024.344701712(116740-116770)Online publication date: 2024
  • (2024)PrivySeC: A secure and privacy-compliant distributed framework for personal data sharing in IoT ecosystemsBlockchain: Research and Applications10.1016/j.bcra.2024.100220(100220)Online publication date: Jul-2024
  • (2023)SBDT: Search-Based Differential Testing of Certificate Parsers in SSL/TLS ImplementationsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598110(967-979)Online publication date: 12-Jul-2023
  • Show More Cited By

Index Terms

  1. Privacy oracle: a system for finding application leaks with black box differential testing

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CCS '08: Proceedings of the 15th ACM conference on Computer and communications security
    October 2008
    590 pages
    ISBN:9781595938107
    DOI:10.1145/1455770
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 October 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. black-box testing
    2. data loss prevention
    3. differential fuzz testing
    4. personal information leaks
    5. sequence alignment algorithm

    Qualifiers

    • Research-article

    Conference

    CCS08
    Sponsor:

    Acceptance Rates

    CCS '08 Paper Acceptance Rate 51 of 280 submissions, 18%;
    Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

    Upcoming Conference

    CCS '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)33
    • Downloads (Last 6 weeks)6
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Securing Personally Identifiable Information: A Survey of SOTA Techniques, and a Way ForwardIEEE Access10.1109/ACCESS.2024.344701712(116740-116770)Online publication date: 2024
    • (2024)PrivySeC: A secure and privacy-compliant distributed framework for personal data sharing in IoT ecosystemsBlockchain: Research and Applications10.1016/j.bcra.2024.100220(100220)Online publication date: Jul-2024
    • (2023)SBDT: Search-Based Differential Testing of Certificate Parsers in SSL/TLS ImplementationsProceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3597926.3598110(967-979)Online publication date: 12-Jul-2023
    • (2023)Operand-Variation-Oriented Differential Analysis for Fuzzing Binding Calls in PDF Readers2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00020(95-107)Online publication date: May-2023
    • (2023)Blind-trust: Raising awareness of the dangers of using unsecured public Wi-Fi networksComputer Communications10.1016/j.comcom.2023.07.011209(359-367)Online publication date: Sep-2023
    • (2022)PosterProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security10.1145/3548606.3564253(3383-3385)Online publication date: 7-Nov-2022
    • (2022)DisTA: Generic Dynamic Taint Tracking for Java-Based Distributed Systems2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN)10.1109/DSN53405.2022.00060(547-558)Online publication date: Jun-2022
    • (2022)An overview of security and privacy in smart cities' IoT communicationsTransactions on Emerging Telecommunications Technologies10.1002/ett.367733:3Online publication date: 21-Mar-2022
    • (2021)Systematic Mutation-Based Evaluation of the Soundness of Security-Focused Android Static Analysis TechniquesACM Transactions on Privacy and Security10.1145/343980224:3(1-37)Online publication date: 9-Feb-2021
    • (2021)The Art, Science, and Engineering of Fuzzing: A SurveyIEEE Transactions on Software Engineering10.1109/TSE.2019.294656347:11(2312-2331)Online publication date: 1-Nov-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media