Ling-yi Tsai, an HRTech expert with decades of experience helping organizations navigate technological change, joins us today to shed light on a critical issue: the hidden risks of using artificial intelligence in hiring. As companies lean more heavily on AI to sift through candidates, especially in a slow hiring market, they may be unintentionally creating systems that are both legally perilous and unfair. We’ll explore how these automated tools can perpetuate bias, the danger of “black box” algorithms that offer no explanation for their decisions, and why human oversight is often applied far too late in the process. We will also discuss how the very design of a job role can exclude diverse talent before an algorithm even sees an application.
AI screening tools can infer demographic details from neutral data like employment gaps or years of experience, potentially filtering out qualified candidates. How can a company proactively audit its AI for these hidden biases, and what specific human oversight is needed at this early stage to ensure fairness?
This is the central challenge we’re facing. It’s a dangerous illusion to think that by simply removing fields like age or race from an application, you’ve created a neutral system. The reality is that AI models are incredibly adept at finding proxies. As legal scholars like Solon Barocas and Andrew Selbst have pointed out, things like zip codes, gaps in a resume, or even the number of years of experience can strongly correlate with protected characteristics. A proactive audit, therefore, cannot just be a surface-level check. It must involve rigorously testing the outcomes of the AI against your applicant pool’s demographics. You need a dedicated team, a mix of HR, legal, and data science, to constantly ask, “Is our system disproportionately screening out candidates from a certain age group, gender, or background?” The most critical human oversight is at the very beginning—before the tool is even fully deployed. Humans must evaluate the training data, question the historical hiring patterns it’s learning from, and set clear, non-negotiable guardrails for the system.
Many AI hiring systems lack clear, interpretable rationales for their decisions, creating significant legal and operational risk. When an algorithm quietly screens out thousands, what steps can an employer take to document and defend the fairness of its process if it can’t explain individual outcomes?
The lack of explainability is a legal minefield. When you can’t explain why a specific candidate was rejected, you’re left incredibly vulnerable. Imagine standing before a regulator or in a courtroom and saying, “The algorithm did it, but we don’t know why.” It’s an indefensible position. A flawed human might reject one person, but a flawed algorithm can silently sideline thousands, as you said. Since you can’t defend the individual outcome, you must pivot to rigorously defending the process. This means meticulous documentation is your only shield. You must document every step: the bias audits you conducted on the training data, the specific guardrails you implemented, the results of regular outcome testing, and the exact points where human judgment was required to intervene. You have to be able to demonstrate that you took every reasonable step to build a fair system, even if its inner workings are opaque. This shifts the focus from a single, unexplainable decision to a documented, good-faith effort to ensure equity at a systemic level.
Human review is often applied only to a small pool of finalists, long after AI and role design have narrowed the field. What are the biggest blind spots created by this late-stage intervention, and how can leaders restructure their hiring process to integrate meaningful human judgment much earlier?
This is a classic case of acting too late. When human review is saved for the final handful of candidates, the biggest blind spot is that you have no idea who you’ve missed. The AI and the initial job requirements have already made the most impactful cuts. By the time a hiring manager sees the “top candidates,” the pool has been sanitized of anyone who didn’t fit a very narrow, predefined mold. You’ve created an echo chamber. The most valuable, diverse, and innovative candidates may have been filtered out on day one for having a non-traditional career path or an employment gap. Restructuring the process means fundamentally changing your philosophy: AI should be a tool to support human judgment, not a gatekeeper that replaces it. Leaders need to insert human checkpoints much earlier. For example, a human should review a random sample of the AI’s rejections to see if the system is making sensible decisions. Another key step is to have a diverse panel of humans review the initial role requirements themselves, challenging assumptions before they ever get coded into an algorithm.
Beyond technology, narrowly defined jobs with rigid experience requirements often favor candidates with linear career paths. How does this practice unintentionally exclude skilled talent, and what are some practical ways to design roles that prioritize adaptability and diverse life experiences over conformity?
This is where bias begins, long before any technology is involved. We’ve become obsessed with designing jobs for a specific type of precision, demanding, for instance, “seven years of experience in X,” when what the role truly needs is a particular skill set that could have been acquired in countless ways. This inherently favors candidates with uninterrupted, traditional careers and penalizes those who took time off for caregiving, switched industries, or built their skills through a mosaic of different roles. It’s a subtle but powerful form of exclusion that filters not for capability, but for conformity to an outdated ideal. A practical first step is to shift job descriptions from being a rigid list of past experiences to a profile of necessary skills and capabilities. Instead of “must have 5 years in marketing,” try “demonstrated ability to lead successful multi-channel campaigns.” This simple change opens the door to a much wider array of talented people whose life experiences have made them adaptable, resilient, and creative problem-solvers.
During slowed hiring cycles, groups like younger workers, older workers, and women are often disproportionately affected by screening. Could you share a specific example of how a common AI filter or rigid job requirement might disadvantage one of these groups, and what is the first step to fixing it?
Certainly. A perfect example that impacts older workers is an aggressive filter based on years of experience. Let’s say a company uses an AI tool to automatically screen out anyone with more than 15 years of experience for a mid-level role, assuming they’d be overqualified or too expensive. This acts as a direct proxy for age, and as research from economists like David Neumark shows, these signals are powerful filters. This exact issue has even led to EEOC enforcement actions. The system isn’t explicitly looking for age, but the outcome is the same: a generation of highly skilled workers is shut out. The first step to fixing this is to remove these arbitrary, numerical cutoffs from your screening protocol. Instead of filtering by a number, the system should be trained to look for core competencies and recent, relevant achievements. The focus must shift from a candidate’s timeline to their actual capabilities.
What is your forecast for how hiring technology and practices will evolve by 2026 to address these challenges?
By 2026, I forecast a significant shift from a focus on automated efficiency to a demand for accountable transparency. The legal and reputational risks, highlighted by cases like Mobley v. Workday, Inc., will become too large for organizations to ignore. We’ll see a new generation of HR technology that will be forced to build explainability and auditability into its core design, moving away from the “black box” models. Companies will stop seeing AI as a replacement for human recruiters and start treating it as a sophisticated co-pilot that requires skilled human oversight. I also believe we’ll see a broader movement in job design itself, with a trend toward “skills-based” hiring finally gaining real traction. The pressure of a changing workforce and the glaring failures of current systems will compel employers to recognize that talent doesn’t follow a straight line, forcing them to build hiring processes that value capability over conformity.