How Does AI Lead to Discrimination?

Artificial intelligence (AI) has many benefits, but organizations must be mindful when using this technology. Without proper safeguards in place to prevent it, AI can lead to bias and discrimination.

What is one potential cause of bias in an AI system?

While there are many ways that bias can be introduced into AI systems, it most frequently occurs due to bias in the algorithms or data. The bias can be the result of conscious preconceived ideas or unconscious stereotypes or assumptions about a group of people, but either way, bias and discrimination can be embedded into the programming and perpetuated in the outcomes. This is why it is imperative for developing teams, as well as the organizations using the AI systems, to evaluate the algorithms for bias. One way in which algorithms are frequently used is predictive modeling, which is the process of using algorithms to make predictions based on the results of past data analysis. If bias exists within the predictive model, it can result in not only to discrimination, but could even be potentially dangerous. Many hospitals and healthcare organizations use predictive modeling to make important decisions, and bias in this environment can lead to patients not receiving the care they require.

An article in Science dissected racial bias in an algorithm used to manage the health of populations and focused on one example in healthcare when predictive modeling AI led to a disparity in inequity in healthcare. A healthcare algorithm used by many US hospitals for millions of patients assigned risk scores to predict how much additional care a patient needs. However, it was trained using data that only took into account previous healthcare costs, and because of the unequal access to care and the differences in the way that black and white people were paying for healthcare, it was miscalculating the risk scores of black patients. Sicker black patients were assigned the same level of risk score as healthier white people, which meant they didn’t qualify for extra care as often.

What is an example of AI discrimination hiring?

Healthcare isn’t the only area where key decisions are being made using AI. Several businesses now use AI technology to aid them throughout the hiring process, but using AI can lead to discrimination here as well. One situation where discrimination can occur is during AI interviews, which can use personality or behavioral assessments, AI-scored video interviews, or gamification tasks to evaluate potential candidates. These types of tools can discriminate against people with disabilities because they may fail to consider necessary accommodations.

Can artificial intelligence eliminate bias in hiring?

AI has the potential to eliminate bias in hiring, but one of the negative impacts of AI is that it also has the capacity to perpetuate it. If the algorithms are biased or incomprehensive data sets are used, AI can discriminate against certain demographics, leading to biased hiring decisions. However, there can be positive impacts of AI on hiring bias as well. It can identify language in job descriptions that could be discriminatory and focus on specific skills rather than identifiers like gender, names, or education. Companies must be vigilant to ensure that the AI technology they are using throughout the hiring process is inclusive and fair.