Programmability requirement in reconfigurable systems necessitates the integration of soft processors in FPGAs. The extensive memory bandwidth sets a major performance bottleneck in soft processors for media applications. While the parallel memory system is a viable solution to account for a large amount of memory transactions in media processors, the memory access conflicts caused by multiple memory buses limit the overall performance. We propose and evaluate the configurable memory address shuffler to be integrated in the memory access arbiter for the parallel memory system in a soft processor. The novel address shuffling algorithm reallocates the decomposed memory sub-pages based on the access conflict graph obtained by profiling the memory access pattern of the application to produce the synthesizable code. The address shuffler efficiently translates the requested memory addresses into the shuffled addresses such that the amount of simultaneous accesses to the identical physical memory block diminishes. The reconfigurability of the address shuffler enables the adaptive address shuffling depending on the memory access pattern of an application running on the soft processor. The configurable address shuffler reduces the amount of access conflicts by 80% on average utilizing 1592 LUTs which is 14% of that of the processor.