This paper presents a chaotic map based image encoding scheme that allows pixel-level operations to be applied on an image while it remains encoded. Using this image encoding scheme, sensitive images can be manipulated by untrusted public cloud servers. Before the images are uploaded onto the public cloud servers they are encoded using user supplied keys by trusted-cloud servers. We fully describe the chaotic map based image encoding algorithm and perform a detailed security analysis of it. Our security analysis demonstrates that the encoding scheme is robust against a wide variety of threats. Additionally, we implemented many pixel- level filtering operations on images over the proposed encoding and found that the overhead is less than 20%.