Image edge detection plays a fundamental role in image processing as well as in computer and machine vision while when applied in medical images can improve medical diagnosis. Thus, efficient hardware implementation of an edge detection system is highly appreciated. In this work we propose a novel architecture for image filtering and edge detection using the Residue Number System (RNS). A VLSI implementation of the proposed architecture dictates that the employment of RNS for the arithmetic processing leads to small hardware complexity and high operation frequency.