Abstract
Authorship verification is the task of determining whether two texts were written by the same author based on a writing style analysis. Author obfuscation is the adversarial task of preventing a successful verification by altering a text’s style so that it does not resemble that of its original author anymore. This paper introduces new algorithms for both tasks and reports on a comprehensive evaluation to ascertain the merits of the state of the art in authorship verification to withstand obfuscation.
After introducing a new generalization of the well-known unmasking algorithm for short texts, thus completing our collection of state-of-the-art algorithms for verification, we introduce an approach that (1) models writing style difference as the Jensen-Shannon distance between the character n-gram distributions of texts, and (2) manipulates an author’s writing style in a sophisticated manner using heuristic search. For obfuscation, we explore the huge space of textual variants in order to find a paraphrased version of the to-be-obfuscated text that has a sufficiently high Jensen-Shannon distance at minimal costs in terms of text quality loss. We analyze, quantify, and illustrate the rationale of this approach, define paraphrasing operators, derive text length-invariant thresholds for termination, and develop an effective obfuscation framework. Our authorship obfuscation approach defeats the presented state-of-the-art verification approaches, while keeping text changes at a minimum. As a final contribution, we discuss and experimentally evaluate a reverse obfuscation attack against our obfuscation approach as well as possible remedies.
About the authors
Janek Bevendorff graduated in Computer Science at the Bauhaus-Universität Weimar in 2018 and has since worked with the Webis group as a PhD candidate in the fields of natural language processing and big data analytics with focus on stylometry and authorship verification. In his Master’s thesis, he wrote about “Authorship Obfuscation Using Heuristic Search”, which part of the research presented in this paper is based on.
Tobias Wenzel did his Master’s in Computer Science in 2019 at Leipzig University on the topic of authorship boosting for attacking KLD-based authorship obfuscation. His work established the ground work for the reverse obfuscation attacks discussed in this paper.
Martin Potthast is head of the Text Mining and Retrieval group at Leipzig University. His research areas include information retrieval and natural language processing, as well as applied machine learning, data mining, and crowdsourcing. Focus of his research is the development of algorithms and machine learning models for information systems and computational stylometry. Martin is co-initiator of the PAN network of excellence for the digital text forensic. Martin studied computer science at Paderborn University, obtained a PhD from the Bauhaus-Universität Weimar in 2011, where he also spent his Postdoc time at the Digital Bauhaus Lab, and was appointed Juniorprofessor at Leipzig University in 2017.
Matthias Hagen is Professor for “Big Data Analytics” at the Martin-Luther-Universität Halle-Wittenberg. His current research interests include information retrieval and web search (e. g., query understanding, conversational search), natural language processing (e. g., argumentation), and data analytics + mining (e. g., simulation and sensor data). Matthias studied computer science at the Friedrich-Schiller-Universität Jena where he also obtained his PhD on algorithmic and computational complexity issues of the equivalence test of monotone Boolean formulas. Afterwards, he moved to the Bauhaus-Universität Weimar where he lead the junior research group “Intelligentes Lernen” (intelligent learning) from 2008–2013. From 2013–2018, Matthias was Juniorprofessor for “Big Data Analytics” and lead the corresponding junior research group at the Bauhaus-Universität Weimar.
Benno Stein is chair of the Web-Technology and Information Systems Group at the Bauhaus-Universität Weimar. His research focuses on modeling and solving data- and knowledge-intensive information processing tasks. Common ground of his research are the principles and methods of symbolic Artificial Intelligence. Benno has developed theories, algorithms, and tools for information retrieval, machine learning, natural language processing, knowledge processing, as well as for engineering design and simulation. He studied at Karlsruhe University (1984–1989), did his PhD (1995) and his habilitation (2002) in computer science at Paderborn University, and was appointed as a full professor for Web Technology and Information Systems at the Bauhaus-Universität Weimar (2005). He is cofounder and spokesperson of the Digital Bauhaus Lab, an interdisciplinary research center for Computer Science, Arts, and Engineering.
Literature
1. A. Abbasi and H. Chen. Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace. ACM Trans. Inf. Syst., 26 (2): 7:1–7:29, Apr. 2008.10.1145/1344411.1344413Search in Google Scholar
2. D. Bagnall. Author Identification using multi-headed Recurrent Neural Networks—Notebook for PAN at CLEF 2015. In CLEF 2015 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
3. J. Bevendorff, M. Potthast, M. Hagen, and B. Stein. Heuristic Authorship Obfuscation. In 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 1098–1108. Association for Computational Linguistics, July 2019.10.18653/v1/P19-1104Search in Google Scholar
4. J. Bevendorff, B. Stein, M. Hagen, and M. Potthast. Generalizing Unmasking for Short Texts. In 14th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2019), pages 654–659. Association for Computational Linguistics, June 2019.10.18653/v1/N19-1068Search in Google Scholar
5. J. Bevendorff, B. Stein, M. Hagen, and M. Potthast. Bias Analysis and Mitigation in the Evaluation of Authorship Verification. In 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 6301–6306. Association for Computational Linguistics, July 2019.10.18653/v1/P19-1634Search in Google Scholar
6. H. Bo, S. H. H. Ding, B. C. M. Fung, and F. Iqbal. ER-AE: differentially-private text generation for authorship anonymization. CoRR, abs/1907.08736, 2019.Search in Google Scholar
7. B. T. Boenninghoff, S. Hessler, D. Kolossa, and R. M. Nickel. Explainable authorship verification in social media via attention-based similarity learning. CoRR, abs/1910.08144, 2019.10.1109/BigData47090.2019.9005650Search in Google Scholar
8. D. Boumber, Y. Zhang, M. Hosseinia, and A. Mukherjee. Robust Authorship Verification with Transfer Learning, 2019.10.29007/9nf3Search in Google Scholar
9. E. G. Bourne. The authorship of the federalist. The American Historical Review, 2 (3): 443–460, 1897.10.2307/1833399Search in Google Scholar
10. M. Brennan, S. Afroz, and R. Greenstadt. Adversarial Stylometry: Circumventing Authorship Recognition to Preserve Privacy and Anonymity. ACM Trans. Inf. Syst. Secur., 15 (3): 12, 2012.10.1145/2382448.2382450Search in Google Scholar
11. P. F. Brown, S. D. Pietra, V. J. D. Pietra, J. C. Lai, and R. L. Mercer. An estimate of an upper bound for the entropy of English. Computational Linguistics, 18 (1): 31–40, 1992.Search in Google Scholar
12. D. Castro, Y. Adame, M. Pelaez, and R. Muñoz. Authorship Verification, Combining Linguistic Features and Different Similarity Functions—Notebook for PAN at CLEF 2015. In CLEF 2015 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
13. C. Emmery, E. M. Arévalo, and G. Chrupala. Style obfuscation by invariance. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20–26, 2018, pages 984–996, 2018.Search in Google Scholar
14. D. M. Endres and J. E. Schindelin. A new metric for probability distributions. IEEE Trans. Information Theory, 49 (7): 1858–1860, 2003.10.1109/TIT.2003.813506Search in Google Scholar
15. J. Fréry, C. Largeron, and M. Juganaru-Mathieu. UJM at CLEF in Author Identification—Notebook for PAN at CLEF 2014. In CLEF 2014 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
16. D. Grangier and M. Auli. Quickedit: Editing text & translations via simple delete actions. CoRR, abs/1711.04805, 2017.Search in Google Scholar
17. K. Guu, T. B. Hashimoto, Y. Oren, and P. Liang. Generating sentences by editing prototypes. CoRR, abs/1709.08878, 2017.Search in Google Scholar
18. M. Hagen, M. Potthast, and B. Stein. Overview of the Author Obfuscation Task at PAN 2017: Safety Evaluation Revisited. In Working Notes Papers of the CLEF 2017 Evaluation Labs, volume 1866 of CEUR Workshop Proceedings.Search in Google Scholar
19. O. Halvani, C. Winter, and L. Graner. Authorship verification based on compression-models. CoRR, abs/1706.00516, 2017.Search in Google Scholar
20. O. Halvani, C. Winter, and L. Graner. Assessing the applicability of authorship verification methods. In Proceedings of the 14th International Conference on Availability, Reliability and Security, ARES 2019, Canterbury, UK, August 26–29, 2019, pages 38:1–38:10, 2019.10.1145/3339252.3340508Search in Google Scholar
21. P. G. Howard. The design and analysis of efficient lossless data compression systems. Brown University, 1993.Search in Google Scholar
22. F. Iqbal, R. Hadjidj, B. C. Fung, and M. Debbabi. A novel approach of mining write-prints for authorship attribution in e-mail forensics. Digital Investigation, 5: S42–S51, 2008.10.1016/j.diin.2008.05.001Search in Google Scholar
23. P. Juola. Authorship Attribution. Foundations and Trends Information Retrieval, 1 (3): 233–334, Dec. 2006.10.1561/9781601981196Search in Google Scholar
24. P. Juola and E. Stamatatos. Overview of the Author Identification Task at PAN 2013. In P. Forner, R. Navigli, and D. Tufis, editors, CLEF 2013 Evaluation Labs and Workshop—Working Notes Papers, 23–26 September, Valencia, Spain. CEUR-WS.org, Sept. 2013.Search in Google Scholar
25. P. Juola and D. Vescovi. Analyzing Stylometric Approaches to Author Obfuscation. In Advances in Digital Forensics VII—7th IFIP WG 11.9 International Conference on Digital Forensics, Revised Selected Papers, Orlando, FL, USA, January 31–February 2, 2011, volume 361 of IFIP Advances in Information and Communication Technology, pages 115–125.10.1007/978-3-642-24212-0_9Search in Google Scholar
26. G. Kacmarcik and M. Gamon. Obfuscating Document Stylometry to Preserve Author Anonymity. In ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17–21 July 2006.10.3115/1273073.1273131Search in Google Scholar
27. M. Kestemont, K. Luyckx, W. Daelemans, and T. Crombez. Cross-genre authorship verification using unmasking. English Studies, 93 (3): 340–356, 2012.10.1080/0013838X.2012.668793Search in Google Scholar
28. D. V. Khmelev and W. J. Teahan. A repetition based measure for verification of text collections and for text categorization. In SIGIR 2003: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, July 28–August 1, 2003, Toronto, Canada, pages 104–110.10.1145/860435.860456Search in Google Scholar
29. M. Khonji and Y. Iraqi. A Slightly-modified GI-based Author-verifier with Lots of Features (ASGALF)—Notebook for PAN at CLEF 2014. In CLEF 2014 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
30. M. Kocher and J. Savoy. UniNE at CLEF 2015: Author Identification—Notebook for PAN at CLEF 2015. In CLEF 2015 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
31. M. Kocher and J. Savoy. A simple and efficient algorithm for authorship verification. JASIST, 68 (1): 259–269, 2017.10.1002/asi.23648Search in Google Scholar
32. M. Kocher and J. Savoy. Distance measures in author profiling. Inf. Process. Manage., 53 (5): 1103–1119, 2017.10.1016/j.ipm.2017.04.004Search in Google Scholar
33. M. Koppel and J. Schler. Authorship Verification as a One-Class Classification Problem. In C. Brodley, editor, Proceedings of the Twenty-First International Conference on Machine Learning, pages 1–7.Search in Google Scholar
34. A. Mahmood, F. Ahmad, Z. Shafiq, P. Srinivasan, and F. Zaffar. A girl has no name: Automated authorship obfuscation using mutant-x. PoPETs, 2019 (4): 54–71, 2019.10.2478/popets-2019-0058Search in Google Scholar
35. A. McDonald, S. Afroz, A. Caliskan, A. Stolerman, and R. Greenstadt. Use Fewer Instances of the Letter “i”: Toward Writing Style Anonymization. In S. Fischer-Hübner and M. Wright, editors, Privacy Enhancing Technologies—12th International Symposium, PETS 2012, Vigo, Spain, July 11–13, 2012, volume 7384 of Lecture Notes in Computer Science, pages 299–318.10.1007/978-3-642-31680-7_16Search in Google Scholar
36. G. A. Miller. Wordnet: A lexical database for english. Commun. ACM, 38 (11): 39–41, Nov. 1995.10.1145/219717.219748Search in Google Scholar
37. A. Narayanan, H. S. Paskov, N. Z. Gong, J. Bethencourt, E. Stefanov, E. C. R. Shin, and D. Song. On the feasibility of internet-scale author identification. In IEEE Symposium on Security and Privacy, SP 2012, 21–23 May 2012, San Francisco, California, USA, pages 300–314.10.1109/SP.2012.46Search in Google Scholar
38. J. Pearl. Heuristics—intelligent search strategies for computer problem solving. Addison-Wesley series in artificial intelligence.Search in Google Scholar
39. N. Potha and E. Stamatatos. Improved algorithms for extrinsic author verification. Knowledge and Information Systems, Oct. 2019.10.1007/s10115-019-01408-4Search in Google Scholar
40. M. Potthast, M. Hagen, and B. Stein. Author Obfuscation: Attacking the State of the Art in Authorship Verification. In Working Notes Papers of the CLEF 2016 Evaluation Labs, volume 1609 of CEUR Workshop Proceedings.Search in Google Scholar
41. M. Potthast, F. Schremmer, M. Hagen, and B. Stein. Overview of the Author Obfuscation Task at PAN 2018: A New Approach to Measuring Safety. In L. Cappellato, N. Ferro, J.-Y. Nie, and L. Soulier, editors, Working Notes Papers of the CLEF 2018 Evaluation Labs, volume 2125 of CEUR Workshop Proceedings.Search in Google Scholar
42. J. Rao and P. Rohatgi. Can Pseudonymity Really Guarantee Privacy? In S. Bellovin and G. Rose, editors, 9th USENIX Security Symposium, Denver, Colorado, USA, August 14–17, 2000. USENIX Association, 2000.Search in Google Scholar
43. P. Rosso, F. Rangel, M. Potthast, E. Stamatatos, M. Tschuggnall, and B. Stein. Overview of PAN 2016—New Challenges for Authorship Analysis: Cross-genre Profiling, Clustering, Diarization, and Obfuscation. In Experimental IR Meets Multilinguality, Multimodality, and Interaction. 7th International Conference of the CLEF Initiative (CLEF 2016), Berlin Heidelberg New York, Sept. 2016.Search in Google Scholar
44. C. Sanderson and S. Guenter. Short text authorship attribution via sequence kernels, markov chains and author unmasking: An investigation. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 482–491, July 2006.10.3115/1610075.1610142Search in Google Scholar
45. D. Sculley and C. E. Brodley. Compression and machine learning: A new perspective on feature space vectors. In 2006 Data Compression Conference (DCC 2006), 28–30 March 2006, Snowbird, UT, USA, pages 332. IEEE Computer Society, 2006.Search in Google Scholar
46. E. Stamatatos. A Survey of Modern Authorship Attribution Methods. Journal of the American Society for Information Science and Technology, 60 (3): 538–556, Mar. 2009.10.1002/asi.21001Search in Google Scholar
47. E. Stamatatos, W. Daelemans, B. Verhoeven, M. Potthast, B. Stein, P. Juola, M. Sanchez-Perez, and A. Barrón-Cedeño. Overview of the Author Identification Task at PAN 2014. In Working Notes Papers of the CLEF 2014 Evaluation Labs, CEUR Workshop Proceedings. CLEF and CEUR-WS.org, Sept. 2014.Search in Google Scholar
48. E. Stamatatos, W. Daelemans, B. Verhoeven, M. Potthast, B. Stein, P. Juola, M. Sanchez-Perez, and A. Barrón-Cedeño. Overview of the Author Identification Task at PAN 2014. In CLEF 2014 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
49. E. Stamatatos, W. D. amd Ben Verhoeven, P. Juola, A. López-López, M. Potthast, and B. Stein. Overview of the Author Identification Task at PAN 2015. In CLEF 2015 Evaluation Labs and Workshop—Working Notes Papers.Search in Google Scholar
50. E. Stamatatos, W. Daelemans, B. Verhoeven, P. Juola, A. López López, M. Potthast, and B. Stein. Overview of the Author Identification Task at PAN 2015. In Working Notes Papers of the CLEF 2015 Evaluation Labs, CEUR Workshop Proceedings.10.1007/978-3-319-24027-5_49Search in Google Scholar
51. B. Stein, N. Lipka, and S. Meyer zu Eißen. Meta Analysis within Authorship Verification. In A. Tjoa and R. Wagner, editors, 5th International Workshop on Text-Based Information Retrieval (TIR 2008) at DEXA, pages 34–39. IEEE, Sept. 2008.10.1109/DEXA.2008.20Search in Google Scholar
52. B. Stein, M. Potthast, and M. Trenkmann. Retrieving Customary Web Language to Assist Writers. In C. Gurrin, Y. He, G. Kazai, U. Kruschwitz, S. Little, T. Roelleke, S. M. Rüger, and K. van Rijsbergen, editors, Advances in Information Retrieval. 32nd European Conference on Information Retrieval (ECIR 2010), volume 5993 of Lecture Notes in Computer Science, pages 631–635, Berlin Heidelberg New York, Mar. 2010.10.1007/978-3-642-12275-0_64Search in Google Scholar
53. B. Stein, M. Hagen, and C. Bräutigam. Generating Acrostics via Paraphrasing and Heuristic Search. In J. Tsujii and J. Hajic, editors, 25th International Conference on Computational Linguistics (COLING 2014), pages 2014–2029. Association for Computational Linguistics, Aug. 2014.Search in Google Scholar
54. W. J. Teahan and D. J. Harper. Using compression-based language models for text categorization. In Language modeling for information retrieval, pages 141–165.10.1007/978-94-017-0171-6_7Search in Google Scholar
55. C. Wu, X. Ren, F. Luo, and X. Sun. A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28–August 2, 2019, volume 1 of Long Papers, pages 4873–4883, July 2019.10.18653/v1/P19-1482Search in Google Scholar
56. W. Xu, A. Ritter, B. Dolan, R. Grishman, and C. Cherry. Paraphrasing for style. In Proceedings of COLING 2012, pages 2899–2914, Mumbai, India, December 2012.Search in Google Scholar
57. Y. Zhao and J. Zobel. Searching with style: Authorship attribution in classic literature. In Computer Science 2007. Proceedings of the Thirtieth Australasian Computer Science Conference (ACSC2007), Ballarat, Victoria, Australia, January 30–February 2, 2007, pages 59–68, 2007.Search in Google Scholar
58. Y. Zhao, J. Zobel, and P. Vines. Using Relative Entropy for Authorship Attribution. In H. T. Ng, M. Leong, M. Kan, and D. Ji, editors, Information Retrieval Technology, Third Asia Information Retrieval Symposium, AIRS 2006, Singapore, October 16–18, 2006, volume 4182 of Lecture Notes in Computer Science, pages 92–105.10.1007/11880592_8Search in Google Scholar
59. R. Zheng, J. Li, H. Chen, and Z. Huang. A framework for authorship identification of online messages: Writing-style features and classification techniques. Journal of the American Society for Information Science and Technology, 57 (3): 378–393, 2006.10.1002/asi.20316Search in Google Scholar
© 2020 Walter de Gruyter GmbH, Berlin/Boston
