Pointcloud is a very precise digital format for recording objects in space. Pointclouds have received increasing attention lately, due to the higher amount of information it provides compared to images. In this paper, we propose a new deep learning architecture called R-CovNet, designed for 3D object recognition. Unlike previous architectures that usually sample or convert pointcloud into three-dimensional grids before processing, R-CovNet does not require any preprocessing. Our main goal is to provide a permutation invariant architecture specially designed for pointclouds data of any size. Experiments with well-known benchmarks show that R-CovNet can achieve an accuracy of 92.7%, thus outperforming all the volumetric methods.