Blockchain is a publicly distributed ledger used to record transactions in Bitcoin-like cryptocurrencies. In recent years, the successful integrations of Public-Key Cryptographic (PKC) algorithms with cryptocurrencies have driven researchers to pursue the study of PKC. However, it is challenging to technically integrate PKC algorithms with blockchain properly in that the studies of blockchain leverage to broad domains and each existing problem can lead to diverse solutions. For cryptographically-solvable problems, it is important to find a secure and practical integration of PKC algorithm with blockchain. We systematically review three major topics in cryptocurrencies, including security, privacy and scalability. We conduct a case analysis which demonstrates how to integrate PKC with blockchains. As an illustration, we propose mutable blockchain which incorporates multiple PKC schemes and show how to use it to remove double-spending transactions via redaction. We then give a concrete construction. As suggested by our performance evaluation, the adopted PKC algorithms can run scalably and efficiently and avoid bottlenecks in the system.