This paper is highlighting the complexity associated with sensors aggregation in an autonomous driving system regarding aspects of sensing technologies, space coverage around ego vehicle, detection issues and software architecture. Building Advanced Driving Assistance systems (ADAS) and autonomous driving functions of Level 3 ([1]) and above mandates that these sources of complexity are properly addressed by novel approaches in dealing with sensors raw data and produced features. The paper tries to depict a structured view of the sources of complexity related to the driving scene that must be apprehended, then goes through situational examples for each theme and pitches key directions for improvement.