Pattern recognition and image analysis find application in numerous areas. They support law enforcement, transportation security and warfighting. They also have numerous commercial purposes, ranging from agriculture to manufacturing to facility security. This paper discusses the security needs of image recognition systems and the particular concerns that the algorithms and methods used for these activities pose. It also discusses how to secure these analysis systems and the future work required in these areas.