A semantic-aware log generation method for network activities

Abstract

Context-aware network logging is becoming more prevalent for enterprise networks, data centers, and forensics. Monitoring agents are strategically placed to generate log files from the activity of interests from various network points. In a distributed architecture, these agents are scattered across multiple nodes, and they have limited network visibility. Consequently, the resulting logs become fragmented and less perceptible without a unified network context. Besides, aggregating useful information from a diverse management protocol with various languages, syntax styles, and notations requires complex semantic understanding to synthesize these log files. Currently, general-purpose logs like SNMP's logs only provide parametric values at connection levels but lacks incident-specific information. Meanwhile, proprietary services like AWS CloudTrail identify more contexts at the incident-level, but they only work on selected products and infrastructure. This paper proposed a platform-agnostic log decoding and generation algorithm (SAG) for network logging that is semantic aware using context aggregation. Firstly, a protocol-agnostic controller acts as a master to collect logs from agents running in routers, firewall, IDS/IPS, load balancers, managed switches, and servers. From these logs, three traffic models, namely (1) service-activity model (SaM), (2) general-activity model (GaM), and (3) device-activity model (DaM), are trained using artificial neural network (ANN). The log generator then uses the context-filling technique to resolve and construct log entries using a generic sentence template while inferring from these machine-learning models. A sentence smoothing technique is designed to restructure entities in the logs based on traffic directionality for semantic correctness. The experimental result shows that SAG's logs have 1.8 times more contexts resolved for improved log's perceptibility.