Tech & Society

A person decides, not the algorithm

A sorting algorithm does not choose what to sort by. A person chooses what to sort by. If a hiring system ranks candidates by "years of continuous experience," this gives a lower rank to anyone who took time off work, for a family member, an illness, or military service. The math in the algorithm does not have bias. The person's choice of what to measure can have bias.

Training data can contain old bias

Machine learning systems learn from historical data. If that data contains bias in who got approved for loans, who got stopped by police, or who got interviewed for a job, the system repeats that bias. The system's output can still look objective, because the system only uses numbers. Programmers need to recognize this. This is a required skill, not an optional skill.

Open source software depends on people

Volunteers maintain much of the software used around the world, including the tools used to build this site. Volunteers write this software in public, for free. This system works because people who do not know each other trust each other. It also works because of shared rules for contributing, and because people give credit to the correct person. Reading other people's code, writing clear commit messages, and explaining your reasoning are skills. These skills matter as much as writing code.

The team that builds a product changes the product

A product often works best for the people who built it. A team with no one who uses a screen reader may release a product that a blind person cannot use. This does not require anyone to intend harm. No one on the team thought about it. The people who write software affect what gets built. The people who write software affect who the product works for.