A Passive External Web Surveillance Technique for Private Networks

Abstract

The variety and richness of what users browse on the Internet has made the communications of web-browsing hosts an attractive target for surveillance. We show that passive external surveillance of web-browsing hosts in private networks is possible despite the anonymizing effects of NATs and HTTP proxies at the gateway. These devices effectively anonymize the origin of communication streams, and remove many identifying features, making it difficult to group web traffic into mutually disjoint same-host single user sets called sessions. Sessions offer a complete picture of each user’s web browsing experience. Without them, passive external surveillance is of little use. This paper offers a content analysis technique called Link Chaining that aids the sessionization process by recovering large pieces of sessions called session fragments. The technique is based on the knowledge that the majority of downloaded web resources are clicked-to from other web pages. By following hyperlinks in the bodies of HTTP messages in passively collected trace data, web traffic can be be coalesced into session fragments and used by human analysts to isolate individual users’ sessions. The technique gives the human analyst a significant advantage over manual methods. The implementation presented here has been tested on accumulated local data and demonstrates the feasibility of the scheme.