In late 2021, New York City enacted a first-in-the-nation law regulating the use of certain automated tools in employment decision-making. NYC is poised to begin enforcement, starting July 5, 2023.
Download this FYI as a printable PDF.
Background
With the enactment of Local Law 144 at the end of 2021, NYC became the first jurisdiction in the country to regulate the use of automated employment decision tools (“AEDTs”). Intended to prevent bias in employment decisions, the law makes it unlawful for an employer or an employment agency to use AEDTs to screen candidates for hire or promotion unless the tools satisfy certain criteria. NYC is poised to begin enforcement, starting July 5, 2023.
Buck comment. As emerging technologies and the use of artificial intelligence (“AI”) and automated decision tools in the workplace gain traction, employers can expect increased scrutiny — and regulation — at the federal, state, and local levels. Multiple federal agencies have already pledged to use their collective authorities to prevent unlawful discrimination resulting from the use of automated systems. Recent EEOC guidance has addressed the use of algorithmic decision-making tools in employment selection procedures, and legislation regulating AI in employment has been introduced in states such as New York, New Jersey and California.
NYC regulates use of automated employment decision tools
Local Law 144 regulates employers’ use of AEDTs to screen job applicants and evaluate current employees for promotion within the city. The new law prohibits employers and employment agencies from using an AEDT unless: (1) the tool has been subject to a bias audit within one year of its use and a summary of the audit results are publicly available; (2) required notices have been provided to job candidates and employees who reside in NYC that an AEDT will be used in connection with their assessments; and (3) candidates and employees are given instructions on how to request an alternative selection process or a reasonable accommodation under other laws. Final rules recently issued by the NYC Department of Consumer and Worker Protection implement the new law and clarify its requirements for bias audits, notices, and disclosures.
What is an AEDT?
Local Law 144 defines an AEDT as:
“any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.”
The final rules further define “machine learning, statistical modeling, data analytics, or artificial intelligence” as “a group of mathematical, computer-based techniques: (i) that generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate’s fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and (ii) for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.”
As the final rules also explain, substantially assisting or replacing discretionary decision-making means: (i) relying solely on a simplified output such as a score, classification, or ranking; (ii) using a simplified output that is weighted more than any other criterion; or (iii) using a simplified output to overrule conclusions derived from other factors including human decision-making.
Bias audits
The NYC law prohibits employers and employment agencies from using an AEDT unless:
- It has been subject to a bias audit within one year of its use; and
- A summary of the bias audit has been made publicly available on the employer’s or employment agency’s website.
The law defines the term “bias audit” as an impartial evaluation by an independent auditor. The audit includes testing the AEDT to assess its disparate impact on candidates or employees. Bias audits must calculate selection or scoring rates for each race/ethnicity and sex category that is required to be reported on the employer’s EEO-1 Component 1 report and compare them with the most selected or highest scored category to determine an impact ratio. The required impact ratio must be calculated separately to compare sex categories, race/ethnicity categories, and intersectional categories of sex, ethnicity, and race. While audit requirements may differ depending on how an AEDT operates, its continued use is prohibited if more than a year has passed since the most recent bias audit.
Publishing audit results
Before using an AEDT, employers or employment agencies must post on the employment section of their website:
- The date of the most recent bias audit of the tool;
- A summary of the audit results, including the data source for the audit, number of individuals the tool assessed that fall within an unknown category, number of applicants or candidates, selection or scoring rates, and impact ratios for all categories; and
- The distribution date of the tool (i.e., the date the employer or employment agency began using the AEDT).
The employer or employment agency must keep the summary of results and distribution date posted for at least six months after its latest use of the AEDT for an employment decision.
Required notices and disclosures
Under the new law, employers or employment agencies must notify candidates and employees who reside in the city that they will use an AEDT in assessing or evaluating them along with the job qualifications and characteristics that the tool will use. While they must also include instructions for requesting an alternative selection process or a reasonable accommodation under other laws (e.g., Americans with Disabilities Act or NYC Human Rights Law), the final rules confirm that an employer or employment agency is not obligated to provide an alternative selection process under NYC’s AEDT Law.
Notice must be provided 10 business days prior to use of the tool. It can be provided in a job posting, by U.S. mail or email. Notice for candidates may be provided on the careers or job section of the website. Notice for employees may be provided in a written policy or procedure.
Employers or employment agencies must also disclose on their website information about the type of data collected for the AEDT, the source of the data collection and their data retention policy or make such information available to a candidate or employee within 30 days of receiving a written request.
In closing
With NYC poised to begin enforcement, employers who are currently using — or plan to use — automated tools in the hiring or promotion process should review their current practices to determine whether they are covered by the new law and what, if any, steps may be necessary to ensure compliance with its bias audit, posting and notice requirements.
Volume 46 | Issue 08