Your ICIMS Assessment Integrations Might Be Serving the Same Rejection on Repeat

A system administrator’s risk guide to AI screening vendors

If you administer ICIMS or any ATS, you probably treat your assessment integrations as plumbing. A candidate applies, a score comes back, the workflow advances or it does not. A new Stanford study suggests that plumbing may be quietly handing the same candidate the same rejection at every company that uses the same vendor. For the ICIMS consultant or system admin who owns those integrations, that is not a candidate problem. It is a vendor-governance problem with your name on it.

This guide walks through what the study actually found, which assessment vendors recycle a candidate’s score across employers, how the new AI hiring laws assign the risk, and how to defend a keep-or-drop decision to your internal team. The short version: most vendors are fine on this front, two are not, and the difference is now something you can be asked to account for.

The Stanford study that should worry every hiring team

The research is titled “Algorithmic Monocultures in Hiring,” led out of Stanford with Chapman and Northeastern. It is scheduled to be presented at the ACM FAccT conference in Montreal in late June 2026, so for now it is a forthcoming paper rather than a finalized one.

Here is what the team did. They analyzed more than 4 million applications across 3 million applicants, all screened by a single vendor, Pymetrics. They examined each of the 1,746 positions separately, the way US discrimination law is actually applied, rather than pooling everyone together. They found clear racial disparities, with roughly 26 percent of applications from Black candidates and 15 percent from Asian candidates landing in positions where the tool produced outcomes that trigger federal adverse-impact scrutiny.

The mechanism is the part worth sitting with. They call it an algorithmic monoculture. When many employers lean on the same screening algorithm, the same candidate gets the same answer over and over. A rejection is not a fresh read at each company. It is the same verdict on repeat, because the same engine is doing the scoring. The study calls the resulting pattern systemic rejection.

One number you will see quoted everywhere deserves a footnote. About 90 percent of employers now run applicants through automated screening before a human ever opens the file. That figure comes from the World Economic Forum, not from the Stanford study itself, but it is the backdrop the research was built against.

Why this is an ICIMS/ATS system admin problem

Assessment vendors rarely live on their own. They live inside your stack, wired into the ATS through the implementation and configuration work that you or your ICIMS/ATS consulting partner set up. The moment a score gates a candidate inside an ICIMS/ATS workflow, the tool is influencing a hiring decision, and that puts it inside the new regulatory definitions.

The monoculture only bites in one specific condition: when a score is reused across employers. Most assessment vendors do not do this. A small number do, and those carry the structural risk. The first job, then, is knowing which bucket each of your integrated tools sits in.

The second reason this is yours: the law just changed in a way that lands on the person who owns the vendor contracts. Colorado SB 189, signed in May 2026 and effective January 1, 2027, voids any contract clause that tries to shield a party from liability for its own discriminatory automated-decision acts. In plain terms, the indemnification language in your assessment contracts needs a fresh read now.

The two vendors that recycle scores

Of the ten leading pre-screen assessment vendors, only two operate a portable score that follows a candidate from employer to employer. The other eight run a fresh evaluation for each client and return results only to the company that requested them.

The two outliers are Pymetrics, now part of Harver, and Plum. Plum states the practice in its own Terms of Service: for repeat applications, “your existing Discovery Survey results are shared with those employers,” and candidates may retake only once every 12 months. That is the same portable-profile model the Stanford study was measuring.

Pymetrics needs one careful distinction, and it protects you to make it. The widely repeated 330-day reuse figure comes from candidate-facing third-party guides, not from Harver’s own policy, which discloses no cross-employer reuse at all and references only an eight-week storage period. So the honest framing is that the reuse is candidate-reported, and that Harver does not document it. That is both accurate and far harder to attack if someone pushes back.

This is the heart of the rebuttal when an executive waves the study off as AI panic. The risk is not that one tool is biased once. It is that when one vendor screens for many employers, a biased result stops being contained to a single company and starts repeating across every employer that uses it. Switching from a reuse vendor to a per-employer vendor is a concrete, defensible risk reduction, not a lateral move.

The ten-vendor exposure matrix

All ten tools below are pre-interview gatekeepers. They sit at the front of the funnel and shape the advance-or-reject decision before a human looks at the candidate. “Reuse” means the score is shared with other employers using the same vendor, without a fresh assessment. Each pointed claim is backed by the vendor’s own public language in the receipts section further down.

A few flags worth noting beyond the reuse question. SHL keeps de-identified data for up to seven years for its own research. Mercer Mettl’s license grants it a perpetual anonymized-data right to improve its assessments. Wonderlic, Aon, and Korn Ferry publish no clear retention figure, which is itself a diligence gap. TestGorilla acts as a data controller rather than a processor, which shifts some duties onto the vendor and changes your deletion workflow.

Which new laws apply, and who they actually target

Every tool above is in scope under the major new regimes, because being a gatekeeper is the trigger. The more useful question for your team is who carries the obligation: the vendor that builds the tool, or you as the employer that uses it. The laws use different words for these roles, so a quick translation. The vendor is the “developer” in Colorado, the “provider” in the EU AI Act, and usually the “processor” in data-protection law. The employer is the “deployer” in both AI laws, and usually the “controller” in data terms.

The bottom line is the part to repeat in a meeting. Two of these laws put the obligation squarely on you and cannot be offloaded to the vendor. NYC LL144 and the Illinois Human Rights Act make the employer the responsible party, and no contract clause moves that duty. The audit, the notice, and the non-discrimination obligation are yours. This is exactly where advisory consulting earns its keep, because the gap between “the vendor handles compliance” and “the law names us” is where organizations get caught.

The other two split the duty. SB 189 allocates liability by comparative fault and voids self-indemnification for discrimination. The EU AI Act puts most of the build-side burden on the vendor, while leaving oversight and worker notification with you. The timing on the EU side moved recently and is worth tracking. The high-risk obligations that cover hiring systems were originally set to apply on August 2, 2026, but the EU’s Digital Omnibus agreement pushes that date to December 2, 2027. It cleared the European Parliament in June 2026 and is awaiting final formal adoption. The delay is not a repeal. Employment AI stays in the high-risk category, so the oversight and notice duties are still coming; you just have more runway to prepare.

How to defend a tool decision

When you choose, keep, or pull a tool, four questions make the decision defensible. Run them in order.

1. Does the tool reuse scores across employers? If yes, meaning Pymetrics or Plum, it carries the monoculture risk the per-employer vendors do not. That alone is reason to prefer an alternative or to demand contractual limits on reuse.
2. What is the retention period, and is it disclosed? A clear, short, employer-controlled window is defensible. An undisclosed or perpetual one is a gap to close before you sign or renew.
3. Is the vendor a processor or a controller? Processor status keeps the deletion decision with you. Controller-style vendors hold more independent rights over the data, which changes both your workflow and your liability story.
4. Can the vendor produce position-level adverse-impact metrics, not blended cross-client reports? That is the audit posture US law and NYC LL144 expect. A vendor that can only hand you company-wide numbers is a flag.

A decision to drop a tool is easiest to defend when it failed the first or second question and the vendor would not fix it by contract. A decision to keep one is easiest to defend when it clears all four and you have the position-level audit to prove it. This is also where a clear-eyed look at ICIMS ROI helps, because the cost of switching a tool is real, and so is the cost of carrying one you cannot defend. Building this review into your ICIMS managed services cadence turns a one-time scramble into a routine vendor check.

What the vendors actually say

Receipts, so any claim above can be backed by a vendor’s own words. Each is a short excerpt from the named public policy.

• Harver Privacy Policy: Harver calls itself a processor and says it will “store your personal data up to eight (8) weeks” after the application process ends. It states no cross-employer reuse.
• SHL Data Protection Notice: SHL keeps data “for our own purposes in a de-identified form for a maximum of seven years.”
• Criteria Privacy Policy: “We will not share the scores with any other prospective employer” without consent.
• Predictive Index Services Privacy Policy: data is “anonymized 120 days following the end of our agreement with the Customer.”
• Mercer Mettl License Agreement: grants a “perpetual, non-exclusive, royalty-free license to copy, modify and use any information” in anonymized form.
• Korn Ferry Assessment Privacy Policy: Korn Ferry “will not share your assessment results with other clients” without consent.
• TestGorilla Privacy Policy: it and customers “retain your test scores on (an) anonymized profile(s) for score benchmarking.”
• Plum Terms of Service: for repeat applications, “your existing Discovery Survey results are shared with those employers.”

The takeaway is not that assessments are evil. It is that a quiet design choice, score portability, turns a normal screening tool into a market-wide gatekeeper, and you are now the person expected to know the difference.

FAQ

Does ICIMS use AI to screen candidates?

ICIMS itself is the ATS, but most teams connect third-party assessment vendors into their ICIMS workflows, and many of those vendors score candidates with AI before a human reviews them. The screening risk usually lives in the integrated assessment tool, not in ICIMS itself, which is why your integration choices matter.

What is an algorithmic monoculture in hiring?

It is what happens when many employers rely on the same screening algorithm. A candidate who scores poorly does not get a fresh evaluation at the next company. They get the same result repeated, because the same engine is doing the scoring across employers.

Which assessment vendors reuse candidate scores across employers?

Among the ten leading pre-screen vendors, Plum states score reuse in its own Terms, and Pymetrics is reported by candidates to reuse scores, though Harver does not document this in its policy. The other eight run a fresh assessment per employer.

Who is liable under the new AI hiring laws, the vendor or the employer?

It depends on the law. NYC Local Law 144 and the Illinois Human Rights Act put the duty on the employer. Colorado SB 189 splits liability by comparative fault, and the EU AI Act puts most of the burden on the vendor as provider while leaving oversight with the employer.

How do I evaluate an assessment vendor’s data retention policy?

Ask whether the vendor reuses scores across employers, what its retention period is and whether it is disclosed, whether it acts as a processor or a controller, and whether it can produce position-level adverse-impact metrics. A vendor that is vague on any of these is a flag worth raising before you renew.