Exploring the Role of AI in Career Access Through a Social Justice Lens

Digital career assessment systems

Digital career assessment systems are contemporary adaptations of traditional paper-and-pencil career development instruments administered on digital platforms. Such tools have their origins in the work of Frank Parsons and are underpinned by psychological theories used in career guidance such the trait-factor approach, where the aim is to help individuals learn about themselves (their traits), learn about available careers and choose from those that match their traits (Leung, 2022). Digital technologies allow for rapid scoring of assessment, a high degree of accuracy in scoring, personalized test results interpretation, and visually enhanced presentation of profiles (Leung, 2022). Digital assessments have evolved with the changing affordances of available technologies. Computer programs offering assessment of personality and/or vocational interests have been supplanted by web and mobile app-based assessments which have been adapted to offer forced-choice response formats, the provision of pictures alongside descriptions of occupations, and the use of emojis instead of rating scales, gamification, simulation (e.g., virtual reality) as assessment, and assessment by AI of individuals based on their social media posts (e.g. Nye, 2022).

Digital career information systems

The delivery of career information dates back to the beginning of industrialisation (Borbély-Pecze, 2020). Now the provision of career information is an ‘integral component’ of career education and guidance (Milosheva et al., 2021). Digital career information systems are platforms that provide occupational, educational, employment, and job-related information. With digitalisation such information can be presented in different media formats, including but not limited to written narratives, videos, and simulations, and interactive websites (e.g. Leung, 2022). The success of digital career information provision has been found to be contingent on how well these are connected to traditional career services (Vigurs et al., 2017).

Digital career guidance systems

Digital career guidance systems provide an opportunity for career exploration by connecting career assessment tools with career information (e.g. Leung, 2022; Osborn et al., 2021). Inputs and outcomes from the assessment inform and interact with the information presented to help users make informed career and educational choices (e.g. Leung, 2022). Leung (2022) argues that digital career guidance systems have the potential to engage and serve the needs of new generations of digitally connected users. While digital career guidance systems have been found to be effective in helping to solve career problems (Osborn et al., 2021), research by Kettunen and Sampson (2019) indicates a lack of readiness by some career service providers in the use of digital career guidance systems due to inadequate ICT infrastructure; a lack of up-to-date information; a lack of ICT competencies and skills; and insufficient integration between ICT and service providers. The COVID19 pandemic showed the value of digital career guidance systems as practitioners were able to provide online service delivery to users in a variety of modes – self-help, brief assisted, and individual case management (Osborn et al., 2022).

Digital career coaching and support systems

Career education and guidance has reached another stage of digitisation with the emergence of digital career coaching and support systems. With the increasing availability of user-friendly large language model (LLM) generative AI apps, individuals can readily access tools that can be used to support them (Haleem et al., 2022) in the development of their careers. These tools can overcome the previously noted limitations of highly complex non-adaptive career guidance systems that are difficult to maintain (Cedefop et al, 2021; Mehraj & Baba, 2019). According to Kelly (2023, p. na), LLM generative AI-based apps have the potential to:

• Review a jobseeker’s resume and provide general input on grammar, spelling, layout, readability, content, and keywords (when compared with a job advertisement). ChatGPT can even be asked to pick up any red flags or weak spots that a recruiter may identify.

• Help candidates identify their skills and strengths from their outlined work experience or past activities.

• Suggest potential jobs or careers suited to a specified set of skills or offer similar alternative paths to a specific job.

• Identify skills sought for a particular job title.

• Find recruiters in a particular sector or geographical area.

• Research an organisation; its employee reviews, values, competitors, or core products.

• Generate conversation starters for networking.

• Assist in creating a cover letter, e-note or thank you note.

• Assist in salary research.

• Support interview practice by prompting common questions or helping a job seeker summarise or refine their answer.

• Help a jobseeker create an elevator pitch, their LinkedIn summary, and LinkedIn posts or assist with writing other online profiles or career biographies.

Such apps are not without its limitations for job seekers (these may generate false information; could produce harmful, discriminatory, misleading, or biased content; reliance on such apps could be perceived as unethical and may harm job seekers’ credibility; these pose a privacy risk; apps may not be up to date with industry norms and emerging trends; and asking the wrong questions can render the responses unhelpful or misleading) (Haleem et al., 2022; Kelly, 2023). While noting that users need to ensure they undertake their own research, thinking, preparation and exercise due diligence, Kelly states that she’d ‘also love to see them [jobseekers] partner with a career practitioner who may be able to challenge, guide and support candidates in safe ways to use [such tools] for career applications’ (Kelly, 2023, p. na). The next step in the digitalisation of career guidance is career practitioners educating people in the use of AI tools as a support system for career access and development. Careers practitioners can make use of these technologies to co-create chatbots to work with clients (Cedefop et al, 2021) and AI can be used in career education to support students with developing skills such self-regulation and motivation, and provide personalised learning support, feedback, assessment, profiling and prediction and competence management (Westman et al., 2021). Career guidance professionals will need to understand the potential and the limitations of such tools to support people in their use (Wilson et al., 2022), and to continue in the tradition of utilising contemporary technology in their practice.

Westman et al. (2021) propose a conceptual model for understanding the ways that AI can be used to support career guidance: AI as coach, AI as collaborator; AI as assistant; and AI as tool. However, such uses of AI come with risk and the potential for harm. ‘Algorithmic decision-making might also lead to the unfair treatment of certain groups of people, implicit discrimination and perceived unfairness’ (Köchling & Wehner, 2020, p. 795) even in systems designed to provide education and guidance. The use of prediction in the determination of the appropriate information or options to give a user is predicated on data (which reflects past performance and historical occurrences) which forecloses particular possibilities and is reproductive rather than transformative. Career guidance is relational, and AI disrupts relational systems, as relations are mediated via the programming driving the algorithm. An AI tool is only as good as its programming, the assumptions underpinning the algorithm, and the data that the decision making is based upon. Given these limitations of AI systems, guidelines are needed for the development of career-focused AI coaching and support tools. Before examining what such guidelines might consist of, the following section examines the ways in which AI is being used in HRM.

The Ethical Risks of using AI in Recruitment and Selection Processes

The ethical risks of using AI in recruitment and selection occur when the process does not take into account ethical principles. The ethical principles involved in recruitment and selection processes include transparency, accountability, autonomy, fairness, non-discrimination, and privacy.

• Transparency: Candidates need to know what AI tools are being used, what data is collected about them and should be provided with an opportunity to verify such information (Budhwar et al., 2022). There should be transparency about the decision-making processes, even when such decisions are outside the control of individuals. The right to be told the process by which a decision was reached and the right to lodge a complaint when candidates feel treated unfairly make it necessary for AI recruiting to be transparent (Hunkenschroer & Kriebitz, 2023).

• Accountability: Candidates need to know how AI-based decision-making affects their outcomes (Budhwar et al., 2022). The use of AI-tools carry an epistemic risk that no one can provide a rationale for a given decision and a culpability risk that no one can be held responsible for a given decision (European Commission, 2020).

• Autonomy: AI potentially reduces human involvement, as human beings cede decision-making or analytical tasks to AI-based automated systems. This means specific applications of AI potentially conflict with the right to human self-determination if they render certain choices obsolete. Therefore, AI recruiting tools should only be used to the extent that they do not limit human autonomy, so as not to conflict with human dignity and the right to occupation (Hunkenschroer & Kriebitz, 2023).

• Fairness: Wachter et al. argue that fairness cannot be automated (2021). The susceptibility of AI tools to algorithmic bias demonstrates the difficulties of ensuring fairness. Drage and Mackereth (2022) argue that AI-powered hiring tools are inherently unfair, noting that attempts to remove race and gender from AI systems often misunderstand these, treating them as isolatable attributes rather than being enmeshed in broader systems of power. Additionally, the attempted outsourcing of “diversity work” to AI tools may (unintentionally) entrench cultures of inequality and discrimination through failure to address the systemic problems within organizations. They describe how the racialised history of character analysis and its associated processes of classification and categorization are a part of longer histories of taxonomical sorting and reflect the current demands and desires of the job market rather than representing a ‘fair’ system (Drage & Mackereth, 2022).

• Non-discrimination: Data sets are susceptible to many types of bias increasing the likelihood that AI that is reliant on historic data will fail in realizing its aims. If a decision made by AI impacts human beings, especially in the selection of job candidates, AI might lead to discrimination (Hunkenschroer & Kriebitz, 2023). If recruitment and selection processes are dependent upon analysis of candidates’ social media profiles, those candidates without public profiles may be discriminated against.

• Privacy and data privacy: AI decisions are typically based on the input of specific data. The input used by an AI solution could conflict with the right to privacy if the data was obtained without the applicant having consented to its use. This risk is magnified by AI’s ability to access applicants’ personal information using, for example, facial recognition software, or social media/digital footprint data mining (Hunkenschroer & Kriebitz, 2023).

Hunkenschroder and Kriebitz (2023) argue rather than being inevitable, the risks related to AI recruiting are a result of inflated expectations and the unreflective use of AI recruiting tools. They state that ethical Al recruitment is possible, but it presents a daunting and complex challenge for businesses.

Applying a social lens to explore the use of AI in recruitment and selection

The brief overview above of the risks posed by the use of AI-powered hiring tools suggests that dependence upon such tools represents an ethical risk through the violation of individuals’ rights (e.g. privacy, fairness, autonomy, etc.) and comes with the risk of algorithmic discrimination. Additionally, the embedding of specific AI-tools in the application processes presupposes a level of technological skill and awareness by candidates. This raises questions about whether the use of AI in recruitment and selection is socially just.

Social justice can be understood as the socially just distribution of resources in society (Bengtsson, 2022). Such resources could include access to education and services to enable individuals to live a meaningful life, for many this would involve access to employment. Bengtsson (2022) suggests that to this idea of social justice, we could add access to information; individuals’ capacity to share information and experiences is essential for being able to participate in the knowledge economy operating in our technological society. In this context socially just career guidance can intervene through the provision of information and building individuals’ capacity to share information, and access employment. However, is the provision of information enough when career access is mediated through AI-powered hiring systems that are arguably incapable of fairness (Wachter et al., 2021)? If we consider the concept of agency, broadly understood as ‘the capacity of persons, or subjects, to deliberate or reach decisions and to act upon those deliberations or decisions to effect intentional change’ (McCrory, 2022, p. 504) it seems that AI-systems can exercise agency given that they are capable of exercising judgement regarding candidates’ suitability based on their application, video interview and/or social media presence. How are candidates to negotiate AI systems that are agentic in this context? One way of determining whether such use of AI is appropriate is offered by the concept of epistemic justice.

Epistemic injustice has been defined as ‘wrong done to someone specifically in their capacity as knower’ (Fricker, 2007, p. 1). Fricker describes two forms of epistemic injustice: hermeneutical and testimonial injustice (Fricker, 2007; 2013). Hermeneutical injustice is a systemic injustice where the experiences of a particular social group are not understood due to prejudicial structural injustices. In terms of the use of AI in career access, this form of injustice was evident in the experiences of women in the Amazon AI selection process. Testimonial injustice is situationally based and occurs when a ‘knower’s’ testimony is not seen as credible due to the way the knowledge is presented (Fricker, 2007). There is potential for AI selection systems to perpetuate testimonial injustice when judging video interviews and assessing social media presence. The judgment exercised by AI systems is limited by the data such systems are trained on, homogenous data will lead to the exclusion of minority groups, including people of colour, gender non-conforming people, and neurodivergent individuals. Fricker’s concept of epistemic injustice is useful for highlighting the ways in which AI recruitment and selection can work against social justice aims. The potential for epistemic injustice to occur when using AI systems is clear, but what are the implications of this for career guidance?

Moving beyond learning how to ‘please the machine’

Based on his findings, Albert recommends that candidates ‘research how to please the machine and the interviewers. Find out what the system looks for to increase your chances of getting hired (e.g. displaying appropriate body language in video screening software)’ (2019, p. 220). This response allows businesses using such tools to abrogate responsibility for the potential harms that could be caused and puts the burden of adjusting to AI-mediated career access on to individual candidates. This response relies on candidates’ ability to access knowledge about what systems are in place and what is needed to perform well in context of such systems. This solution reinscribes existing inequities. While candidates may be able to learn how to ‘please the machine’, career practitioners and recruiters also have a responsibility to provide an educative role – teaching people about these emerging practices in recruitment and selection.

To mitigate the ethical risks, companies developing the AI systems, the businesses employing them, and regulatory, legislative, and legal authorities have a role to play in minimising the potential harm that could be caused by the use of AI in career access. It is argued that AI systems should be ethical, and the ‘Build and Design’ of such systems should have ethics as a part of design within the software code (Nasim et al., 2022). Organisations should adopt best practices such as those used for the governance of algorithms in sectors such as healthcare and pharmaceuticals (Tursunbayeva et al., 2021). Self-regulations and self-audits should be part of HRM practice, including regular analysis and monitoring of the use of their data (Sonderling et al., 2022). Key to mitigating biased algorithms are regulation, transparency and inclusive development cultures (Harvis-Nazzario, 2022).

Guidelines for the development of AI tools in Career Access and Career Guidance

Many of guidelines developed in relation to the use of AI in connected and automated vehicles (European Commission, 2020) apply to the use of AI tools which exercise judgement, both in the recruitment and selection processes and in career coaching and support systems. Methods for the explainability of AI-based forms of decision making should be developed that are user-centred. The methods and vocabulary used to explain the role and functioning of AI systems should be transparent and cognitively accessible, the purposes and capabilities of AI should be openly communicated, and the outcomes traceable.

Individuals need to be adequately informed where AI systems are in use and be able to exercise their rights, such as the right to privacy, and to actively and independently, scrutinise, question, refrain from using, or negotiate AI modes of use and terms of service. Alternative routes for application and access should be available to candidates who have do not use social media for example, or do not want their data used and collected by AI tools. The obligations of the agents involved in the use of AI need to be identified. Given the size and complexity of the network of individuals and organisations in the creation, deployment, and use, it may become unclear who is responsible for ensuring that AI and its users comply with ethical and legal standards. Every individual and organisation should know who is required to do what and how. This can be done by creating a shared understanding of different actors’ obligations toward the ethical design, deployment, and use of AI.

A new culture of responsibility should be fostered in relation to the use of AI in career access. This culture should include accountability – defined here as a form of responsibility arising from the obligation to explain something that has happened and one’s role in that happening. A fair system of accountability requires that: (a) formal and informal mechanisms of accountability are created with respect to the use of AI; (b) different actors are aware of and able to discharge their duty to justify the operation of the system; (c) and the system of which AI are a part is not too complex, opaque, or unpredictable (European Commission, 2020).