What Can Business Analysts Do To Remove Bias In Machine Learning?

Machine learning and other artificial intelligence capabilities are expanding to every industry. Machines are analyzing data to write news articles, predict when car parts will fail, identify customer emotions, detect credit card fraud, drive cars, diagnose cancer and identify criminal behavior.

What are the consequences of bad decision making in these areas? Biased data could prompt machines to make decisions with consequences ranging from fake news and irrelevant advertisements to false arrests and even death.

Many companies are wasting millions of dollars developing innovative products that fail to meet expectations. A few companies are getting it right and thriving. What’s the difference? Well defined outcomes and good analysis!

But it’s not traditional analysis. With machine learning, the machine learns the decision logic on its own through patterns and does not have to be explicitly programmed with a logical specification of requirements. The machine, not the BA, uses data to identify business rules and many requirements. Does this mean BAs are obsolete? Absolutely not! BA skills are even more important when we are relying on machines to make decisions. Why?

Because Machines Still Don’t Make Good Decisions Without People

People play a different role in machine learning! BAs working on machine learning projects shift their efforts from traditional requirements elicitation and analysis to have more focus on experiments and solution evaluation.

Solution evaluation is about looking at a constructed solution, machine learning algorithm perhaps, and how it is serving user and business needs. The solution doesn’t have to be fully built to be evaluated—a prototype or minimum viable product is great starting point.

Here’s what the solution evaluation cycle looks like:

Define Desired Outcomes

Many machine learning solutions fail because teams focus solely on technology and ignore the context, purpose and desired outcomes. At the beginning of any new project ,this is a key step for BAs. In the case of machine learning BAs will define (and continuously refine) desired outcomes based on understanding four factors:

• The users/humans who are the source the data the machine is using
• The empathy state of the user and their processes and workflows
• The potential biases of user group and the source data
• How to analyze the machine’s output and clean up the data to remove bias that creates practical and ethical issues

 

Understand Data

We’ve all heard of GIGO, right? Garbage in, garbage out. That’s especially true for the data used by systems that make automated decisions. If one data point is wrong, missing or misinterpreted, the organization and the user will suffer consequences.

It’s the BAs role to understand the data and continuously evaluate the data set. BAs help the team gain confidence that the machine has the right data to solve the problem and move the outcome in the right direction. BAs need to know:

• Where the data is coming from
• How the data is influencing the decisions made by the machine
• Which pieces of data are meaningful to achieve the desired outcomes

 

Let The Machine Learn

One pitfall for teams is spending most of their time and energy defining, building and analyzing this part of the process. They get so focused on the actual machine learning technology that they miss out on the importance of learning iteratively through experimentation and analysis.

This leads teams down a path of self-destruction as teams fails to make connections between the technology, the data, the customers and the desired outcomes.

Experiment And Analyze

Successful machine learning solutions require multiple cycles of experiments and analysis. This effort gives us insights into what needs to be fixed. These insights are much more meaningful than reacting to a list of user requests. BAs use these insights to change the data, refine the questions and the modify the variables to discover the impact on how the machine learns.

But BAs do not do this by themselves in their cube! Instead, it’s a collaborative effort that includes the customer or at least a customer-centered approach. To help the machine make good decisions, BAs use experimentation and analysis to:

• Observe what is and isn’t working for the users/customers
• Analyze the business and user metrics that matter to outcomes
• Analyze the user experience, customer experience, data flows and processes to determine how the overall process and solution is performing
• Analyze real-time data to see how users are interacting with the solution
• Identify any data that is introducing bias into the machine’s decisions

 

Remove Bias

After teams experiment and analyze, it’s time to refine the desired outcomes and update the data as needed to achieve the outcomes. A big part of that process calls BAs to remove biases identified in the machine learning results.

The team manages biased data in several ways. They can:

• Remove biased data
• Retrain the machine to handle bias data differently
• Adjust the input process of users to remove/address the bias data

 

Teams that skip or minimize experimentation and analysis waste millions of dollars on projects that fail. Help your machines make better decisions by advocating for the importance of customer interaction, context, data quality, outcomes and refining the data based on experiments.

Machine decisions should make sense for our customers and our organization. And that requires a human (BA) touch!