Lena Armstrong
Amy J. Ko
ABSTRACT
Automated hiring algorithms are increasingly used in computing job recruitment. Prior work has examined perceptions of algorithmic fairness and established bias in hiring algorithms, but there is limited work on the ability of computer science students, who are applying for their first computing job, to overcome new barriers posed by automated hiring. To investigate what challenges students face, how they work through them, and their perceptions of these systems, we conducted semi-structured interviews with post-secondary students who were first-time computing job applicants. Analyses revealed that participants had diverse knowledge of hiring algorithms; some people knew to use strategies, such as keywords in resumes, online assessment practice, and referrals to circumvent automated processes to progress to in-person interviews, but others were entirely unaware of the automation. Participants also expressed that current systems prevented them from demonstrating the full extent of their skills and attributed job offers to personal contacts within the company. While some deemed automation a "necessary evil" to combat scale, many struggled with the inequity automated hiring processes perpetuated. Understanding student experiences and perspectives with automated hiring has relevance for how current computer science curricula prepares students for the transition to computing jobs post-graduation. Our findings have implications for how to develop new practices to better support students in their transitions amid a changing hiring landscape.
- Chaza Abdul, Wenli Wang, and Yating Li. 2020. The Impact of Technology on Recruitment Process. Issues in information systems 21, 4 (2020).Google Scholar
- Ifeoma Ajunwa. 2019. An Auditing Imperative for Automated Hiring. (2019).Google Scholar
- Joseph Appianing and Richard N Van Eck. 2015. Gender differences in college students’ perceptions of technology-related jobs in computer science. International Journal of Gender, Science and Technology 7, 1 (2015), 28–56.Google Scholar
- Andrew Begel and Beth Simon. 2008. Novice software developers, all over again. In Proceedings of the fourth international workshop on computing education research. 3–14.Google ScholarDigital Library
- Mahnaz Behroozi, Chris Parnin, and Titus Barik. 2019. Hiring is broken: What do developers say about technical interviews?. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). IEEE, 1–9.Google ScholarCross Ref
- Mahnaz Behroozi, Shivani Shirolkar, Titus Barik, and Chris Parnin. 2020. Debugging hiring: What went right and what went wrong in the technical interview process. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Society. 71–80.Google Scholar
- Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. ’It’s Reducing a Human Being to a Percentage’ Perceptions of Justice in Algorithmic Decisions. In Proceedings of the 2018 Chi conference on human factors in computing systems. 1–14.Google ScholarDigital Library
- Skyler J Bock, Lindsay J Taylor, Zachary E Phillips, and Wenying Sun. 2013. Women and minorities in computer science majors: Results on barriers from interviews and a survey. Issues in Information Systems 14, 1 (2013), 143–152.Google Scholar
- Steven D Brown and Robert W Lent. 2017. Social cognitive career theory in a diverse world: Closing thoughts. Journal of Career Assessment 25, 1 (2017), 173–180.Google ScholarCross Ref
- Maarten Buyl, Christina Cociancig, Cristina Frattone, and Nele Roekens. 2022. Tackling Algorithmic Disability Discrimination in the Hiring Process: An Ethical, Legal and Technical Analysis. In 2022 ACM Conference on Fairness, Accountability, and Transparency. 1071–1082.Google Scholar
- Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science 356, 6334 (2017), 183–186.Google Scholar
- Lei Chen, Gary Feng, Chee Wee Leong, Blair Lehman, Michelle Martin-Raugh, Harrison Kell, Chong Min Lee, and Su-Youn Yoon. 2016. Automated scoring of interview videos using Doc2Vec multimodal feature extraction paradigm. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 161–168.Google ScholarDigital Library
- Lei Chen, Ru Zhao, Chee Wee Leong, Blair Lehman, Gary Feng, and Mohammed Ehsan Hoque. 2017. Automated video interview judgment on a large-sized corpus collected online. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 504–509.Google ScholarCross Ref
- Phoebe K Chua, Hillary Abraham, and Melissa Mazmanian. 2021. Playing the Hiring Game: Class-Based Emotional Experiences and Tactics in Elite Hiring.Proc. ACM Hum. Comput. Interact. 5, CSCW2 (2021), 1–27.Google Scholar
- Phoebe K Chua and Melissa Mazmanian. 2020. Are you one of us? Current hiring practices suggest the potential for class biases in large tech companies. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1–20.Google Scholar
- John W Creswell and Cheryl N Poth. 2016. Qualitative inquiry and research design: Choosing among five approaches. Sage publications.Google Scholar
- Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. In Ethics of Data and Analytics. Auerbach Publications, 296–299.Google Scholar
- Meenakshi Dhingra and Subhash C Kundu. 2021. Factors affecting placement and hiring decisions: A study of students’ perceptions. Industry and Higher Education 35, 3 (2021), 223–232.Google ScholarCross Ref
- Allan Fisher and Jane Margolis. 2003. Unlocking the clubhouse: Women in computing. In Proceedings of the 34th SIGCSE technical symposium on Computer science education. 23.Google ScholarDigital Library
- Marcelline R Fusilier and Michael A Hitt. 1983. Effects of age, race, sex, and employment experience on students’ perceptions of job applications. Perceptual and Motor Skills 57, 3_suppl (1983), 1127–1134.Google Scholar
- Michail N Giannakos, Ilias O Pappas, Letizia Jaccheri, and Demetrios G Sampson. 2017. Understanding student retention in computer science education: The role of environment, gains, barriers and usefulness. Education and Information Technologies 22, 5 (2017), 2365–2382.Google ScholarDigital Library
- Manuel F Gonzalez, Weiwei Liu, Lei Shirase, David L Tomczak, Carmen E Lobbe, Richard Justenhoven, and Nicholas R Martin. 2022. Allying with AI? Reactions toward human-based, AI/ML-based, and augmented hiring processes. Computers in Human Behavior 130 (2022), 107179.Google ScholarDigital Library
- Brian Patrick Green, Patricia Graybeal, and Roland L Madison. 2011. An exploratory study of the effect of professional internships on students’ perception of the importance of employment traits. Journal of Education for Business 86, 2 (2011), 100–110.Google ScholarCross Ref
- David Hammer and Leema K Berland. 2014. Confusing claims for data: A critique of common practices for presenting qualitative research on learning. Journal of the Learning Sciences 23, 1 (2014), 37–46.Google ScholarCross Ref
- Louis Hickman, Nigel Bosch, Vincent Ng, Rachel Saef, Louis Tay, and Sang Eun Woo. 2022. Automated video interview personality assessments: Reliability, validity, and generalizability investigations.Journal of Applied Psychology 107, 8 (2022), 1323.Google Scholar
- Kay A Hodge and Janet L Lear. 2011. Employment skills for 21st century workplace: The gap between faculty and student perceptions.Journal of Career and Technical Education 26, 2 (2011), 28–41.Google Scholar
- James Hu. 2021. Report: 98 of fortune 500 companies use ATS. https://www.jobscan.co/blog/fortune-500-use-applicant-tracking-systems/Google Scholar
- Gareth R Jones. 1986. Socialization tactics, self-efficacy, and newcomers’ adjustments to organizations. Academy of Management journal 29, 2 (1986), 262–279.Google ScholarCross Ref
- Maria Kasinidou, Styliani Kleanthous, Pınar Barlas, and Jahna Otterbacher. 2021. I agree with the decision, but they didn’t deserve this: Future Developers’ Perception of Fairness in Algorithmic Decisions. In Proceedings of the 2021 acm conference on fairness, accountability, and transparency. 690–700.Google ScholarDigital Library
- Pauline T Kim and Sharion Scott. 2018. Discrimination in online employment recruiting. Louis ULJ 63 (2018), 93.Google Scholar
- Styliani Kleanthous, Maria Kasinidou, Pınar Barlas, and Jahna Otterbacher. 2022. Perception of fairness in algorithmic decisions: future developers’ perspective. Patterns 3, 1 (2022), 100380.Google ScholarCross Ref
- Sven Laumer, Christian Maier, and Andreas Eckhardt. 2015. The impact of business process management and applicant tracking systems on recruiting process performance: an empirical study. Journal of Business Economics 85 (2015), 421–453.Google ScholarCross Ref
- Min Kyung Lee. 2018. Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data & Society 5, 1 (2018), 2053951718756684.Google ScholarCross Ref
- Robert W Lent, Steven D Brown, and Gail Hackett. 2002. Social cognitive career theory. Career choice and development 4, 1 (2002), 255–311.Google Scholar
- Lan Li, Tina Lassiter, Joohee Oh, and Min Kyung Lee. 2021. Algorithmic hiring in practice: Recruiter and HR Professional’s perspectives on AI use in hiring. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society. 166–176.Google ScholarDigital Library
- Iftekhar Naim, Md Iftekhar Tanveer, Daniel Gildea, and Mohammed Ehsan Hoque. 2016. Automated analysis and prediction of job interview performance. IEEE Transactions on Affective Computing 9, 2 (2016), 191–204.Google ScholarCross Ref
- Selin E Nugent and Susan Scott-Parker. 2022. Recruitment AI has a Disability Problem: anticipating and mitigating unfair automated hiring decisions. In Towards Trustworthy Artificial Intelligent Systems. Springer, 85–96.Google Scholar
- Joann S Olson. 2014. Opportunities, obstacles, and options: First-generation college graduates and social cognitive career theory. Journal of Career Development 41, 3 (2014), 199–217.Google ScholarCross Ref
- Cathy O’Neil. 2016. Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway books.Google Scholar
- Prasanna Parasurama and João Sedoc. 2022. Gendered information in resumes and its role in algorithmic and human hiring bias. In Academy of Management Proceedings, Vol. 2022. Academy of Management Briarcliff Manor, NY 10510, 17133.Google Scholar
- Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy. 2020. Mitigating bias in algorithmic hiring: Evaluating claims and practices. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 469–481.Google ScholarDigital Library
- Alene Rhea, Kelsey Markey, Lauren D’Arinzo, Hilke Schellmann, Mona Sloane, Paul Squires, and Julia Stoyanovich. 2022. Resume Format, LinkedIn URLs and Other Unexpected Influences on AI Personality Prediction in Hiring: Results of an Audit. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society. 572–587.Google ScholarDigital Library
- Alene K Rhea, Kelsey Markey, Lauren D’Arinzo, Hilke Schellmann, Mona Sloane, Paul Squires, Falaah Arif Khan, and Julia Stoyanovich. 2022. An external stability audit framework to test the validity of personality prediction in AI hiring. Data Mining and Knowledge Discovery 36, 6 (2022), 2153–2193.Google ScholarDigital Library
- Harriet Rodney, Katarina Valaskova, and Pavol Durana. 2019. The artificial intelligence recruitment process: How technological advancements have reshaped job application and selection practices. Psychosociological Issues in Human Resource Management 7, 1 (2019), 42–47.Google Scholar
- Javier Sánchez-Monedero, Lina Dencik, and Lilian Edwards. 2020. What does it mean to’solve’the problem of discrimination in hiring? Social, technical and legal perspectives from the UK on automated hiring systems. In Proceedings of the 2020 conference on fairness, accountability, and transparency. 458–468.Google ScholarDigital Library
- Curtis J Simon and John T Warner. 1992. Matchmaker, matchmaker: The effect of old boy networks on job match quality, earnings, and tenure. Journal of labor economics 10, 3 (1992), 306–330.Google ScholarCross Ref
- Mona Sloane, Emanuel Moss, and Rumman Chowdhury. 2022. A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability. Patterns 3, 2 (2022), 100425.Google ScholarCross Ref
- Mindi N Thompson, Jason J Dahling, Mun Yuk Chin, and Robert C Melloy. 2017. Integrating job loss, unemployment, and reemployment with social cognitive career theory. Journal of Career Assessment 25, 1 (2017), 40–57.Google ScholarCross Ref
- Ruotong Wang, F Maxwell Harper, and Haiyi Zhu. 2020. Factors influencing perceived fairness in algorithmic decision-making: Algorithm outcomes, development procedures, and individual differences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–14.Google ScholarDigital Library
- Allison Woodruff, Sarah E Fox, Steven Rousso-Schindler, and Jeffrey Warshaw. 2018. A qualitative exploration of perceptions of algorithmic fairness. In Proceedings of the 2018 chi conference on human factors in computing systems. 1–14.Google ScholarDigital Library
- Lixuan Zhang and Christopher Yencha. 2022. Examining perceptions towards hiring algorithms. Technology in Society 68 (2022), 101848.Google ScholarCross Ref
Index Terms
- Navigating a Black Box: Students’ Experiences and Perceptions of Automated Hiring
Recommendations
IT Hiring Growth Modest, But Steady
IT hiring and salaries are expected to increase in 2006, but to thrive in the IT marketplace, employees will need to demonstrate an expertise in more than just a programming language. Soft skills, such as Cobol and SAP can be key differentiators. ...
Student hiring: why not make it an event?
SIGUCCS '12: Proceedings of the 40th annual ACM SIGUCCS conference on User servicesManaging student employees is like raising a family. Like most parents, seeing their child grow and move away to school is hard. One of the more difficult things about managing students is when you watch the employee, who you've seen change from student ...
A Hiring Story: Experiences of Employers in Hiring CS Graduates in Software Startups
CSCW'22 Companion: Companion Publication of the 2022 Conference on Computer Supported Cooperative Work and Social ComputingSoftware startups develop products under extreme conditions of uncertainty. Therefore, in this study, we perform an interview study through semi-structured interviews with 11 employers of software startups in a developing country (Bangladesh). We find ...
Comments