Recent advancements in deep learning for face recognition have led to concerns regarding privacy and algorithmic bias, particularly in inferring demographic attributes from facial templates. Existing methods often struggle to balance privacy preservation with utility and operational efficiency. In this paper, we propose a filter-based privacy-enhancing method inspired by the information bottleneck concept. Our approach involves training a filter estimator that assigns scores to intermediate-layer elements based on their sensitivity to target attributes. By selectively replacing sensitive elements with noise while allowing less sensitive ones to pass through, our method aims to enhance privacy while managing the trade-off with verification performance. Most importantly, our approach allows for post-training tuning of the privacy-utility trade-off, providing flexibility for different operational requirements. Evaluation across multiple face recognition networks and datasets demonstrates that our approach can achieve substantial gains in the gender obfuscation task while maintaining adequate verification performance and computational efficiency suitable for real-time applications.