Algorithmic Bias and Fairness: Crash Course AI #18
video description
Date: 2022-04-04
Related videos
Comments and reviews: 10
Qilin
An interesting paper published in the journal of Psychological Science in 2018 looked at cross-cultural differences between international databases of achievement in STEM programs, and found that the lower Gender Gap Index of a country, the more likely it is to have -equal- rate of women to men among STEM graduates of universities. That is to say in countries with high Global Gender Gap Index like Finland, Norway, and Sweden, they tend to have significantly lower rates of women graduates of STEM programs (-20-25%, whereas in some of the countries with the lowest Global Gender Gap Index like UAE, Turkey, Algeria, they have some of the highest rates of women graduates of STEM programs (-36-41%.
The paper is titled -The Gender-Equality Paradox in Science, Technology, Engineering, and Mathematics Education- DOI: 10. 1177/0956797617741719
In the Nursing and Programmer example, you mention data reflecting hidden biases in society, and certainly there must be some hidden biases that influence this population distribution. But it would be apt to also note that bias can exist in the way the data is presented to. This bias is called -Algorithmic Fairness- and is used by google, and ties in with data manipulation mentioned in section 5, though arguably it's not -malicious-.
At its core, algorithmic fairness manipulates data to over-represent groups. There are examples of this that anybody can test out, where results on image searches produce nearly 50: 50 results between two subpopulations, despite their actual ratio in reality not being 50: 50. This isn't malicious, but it could be harmful all the same. In an ideal world where there is true equality, we can pursue whatever career we want without worrying about the statistics of who makes up what job. And in the most -equal- societies, we find that there are fewer women in STEM. While using algorithmic fairness to show even pictures of men and women might make women in STEM feel better, it might also make women who don't wish to pursue STEM feel bad for not contributing to that equality. And this may sound silly, but we see the consequences of this in the -STEAM- movement where they try to include Arts into STEM to be more inclusive of women.
Ultimately this boils down into the problem of equality of outcome, versus equality of opportunity. We know that in the most equal societies, they have close to equality of opportunity but not enough equality of outcome. And we know that in the least equal society, they have close to equality of outcome, but nowhere near enough equality of opportunity. The key question, then, is -Should algorithms reflect the actual data even if it's biased, or should algorithmic fairness be implemented to to makeup for biases hidden in society? - because there are arguments for both sides. If we truly believe that more equal societies are better, I think there's merit in accepting disproportionate gender representation.
reply
An interesting paper published in the journal of Psychological Science in 2018 looked at cross-cultural differences between international databases of achievement in STEM programs, and found that the lower Gender Gap Index of a country, the more likely it is to have -equal- rate of women to men among STEM graduates of universities. That is to say in countries with high Global Gender Gap Index like Finland, Norway, and Sweden, they tend to have significantly lower rates of women graduates of STEM programs (-20-25%, whereas in some of the countries with the lowest Global Gender Gap Index like UAE, Turkey, Algeria, they have some of the highest rates of women graduates of STEM programs (-36-41%.
The paper is titled -The Gender-Equality Paradox in Science, Technology, Engineering, and Mathematics Education- DOI: 10. 1177/0956797617741719
In the Nursing and Programmer example, you mention data reflecting hidden biases in society, and certainly there must be some hidden biases that influence this population distribution. But it would be apt to also note that bias can exist in the way the data is presented to. This bias is called -Algorithmic Fairness- and is used by google, and ties in with data manipulation mentioned in section 5, though arguably it's not -malicious-.
At its core, algorithmic fairness manipulates data to over-represent groups. There are examples of this that anybody can test out, where results on image searches produce nearly 50: 50 results between two subpopulations, despite their actual ratio in reality not being 50: 50. This isn't malicious, but it could be harmful all the same. In an ideal world where there is true equality, we can pursue whatever career we want without worrying about the statistics of who makes up what job. And in the most -equal- societies, we find that there are fewer women in STEM. While using algorithmic fairness to show even pictures of men and women might make women in STEM feel better, it might also make women who don't wish to pursue STEM feel bad for not contributing to that equality. And this may sound silly, but we see the consequences of this in the -STEAM- movement where they try to include Arts into STEM to be more inclusive of women.
Ultimately this boils down into the problem of equality of outcome, versus equality of opportunity. We know that in the most equal societies, they have close to equality of opportunity but not enough equality of outcome. And we know that in the least equal society, they have close to equality of outcome, but nowhere near enough equality of opportunity. The key question, then, is -Should algorithms reflect the actual data even if it's biased, or should algorithmic fairness be implemented to to makeup for biases hidden in society? - because there are arguments for both sides. If we truly believe that more equal societies are better, I think there's merit in accepting disproportionate gender representation.
reply
TapIntoTheEssence
During the debate that followed ProPublia's accusations of the COMPAS-algorithm being discriminatory against black people, Kleinberg, Mullainathan and Raghavan showed that there are inherent trade-offs between different notions of fairness.
In the case of COMPAS, for example, the algorithm was -well-calobrated among groups-, which means that, independent of skin colour, a group of people classified as, say, 70% to recidive, actually had 70% of people that would recidive.
However, ProPublia objected, that the algorithm produced more false positive predictions for blacks (meaning that blacks were labeled more often wrongly as high risk) and more false negative predictions for whites (meaning that whites were more often labeled wrongly as low risk.
In their paper, the authors showed that these notions of fairness, namely -well balanced among groups-, -balance for the negative class- and -balance for the positive class- are mathematically incompatible and exclude each other. One can't have the one and the other at the same time.
So yes, AI-systems will be biased, as insisted upon in the video. But it raises questions about what kind of fairness we want to be implemented and what we're willing to give up.
reply
During the debate that followed ProPublia's accusations of the COMPAS-algorithm being discriminatory against black people, Kleinberg, Mullainathan and Raghavan showed that there are inherent trade-offs between different notions of fairness.
In the case of COMPAS, for example, the algorithm was -well-calobrated among groups-, which means that, independent of skin colour, a group of people classified as, say, 70% to recidive, actually had 70% of people that would recidive.
However, ProPublia objected, that the algorithm produced more false positive predictions for blacks (meaning that blacks were labeled more often wrongly as high risk) and more false negative predictions for whites (meaning that whites were more often labeled wrongly as low risk.
In their paper, the authors showed that these notions of fairness, namely -well balanced among groups-, -balance for the negative class- and -balance for the positive class- are mathematically incompatible and exclude each other. One can't have the one and the other at the same time.
So yes, AI-systems will be biased, as insisted upon in the video. But it raises questions about what kind of fairness we want to be implemented and what we're willing to give up.
reply
Hetare
The point of the Google image search example isn't to accuse Google of some grave injustice, it's just an easy to understand example of how just because a computer is generating it doesn't mean its output isn't biased. The society it's getting its data from is biased in favour of female nurses, so it will return mostly pictures of female nurses even when the user is just looking for -nurse- without specifying gender. Once you understand that, it's easy to understand how that can become a problem when the situation is more complicated, the stakes are higher, which is the whole point of the episode.
Let's say there's 10 male nurses in the world and 90 female nurses. Out of those 100 nurses, one man and two women have committed the same misdemeanour on the job. Given that, would it be fair to make decisions on who to to employ as nurse based on the idea that 10% of men have committed this misdemeanour but only -2% of women have? An AI trained with this data might. Worse yet, you don't even know it's doing this because its decision-making process is more or less a black box.
reply
The point of the Google image search example isn't to accuse Google of some grave injustice, it's just an easy to understand example of how just because a computer is generating it doesn't mean its output isn't biased. The society it's getting its data from is biased in favour of female nurses, so it will return mostly pictures of female nurses even when the user is just looking for -nurse- without specifying gender. Once you understand that, it's easy to understand how that can become a problem when the situation is more complicated, the stakes are higher, which is the whole point of the episode.
Let's say there's 10 male nurses in the world and 90 female nurses. Out of those 100 nurses, one man and two women have committed the same misdemeanour on the job. Given that, would it be fair to make decisions on who to to employ as nurse based on the idea that 10% of men have committed this misdemeanour but only -2% of women have? An AI trained with this data might. Worse yet, you don't even know it's doing this because its decision-making process is more or less a black box.
reply
james
Yeah when you complain about the AI making things -a little more difficult- or -frustrating- then you've really got nothing to complain about. So Google image shows pictures of nurses as women and programmers as men. More women are nurses and more programmers are men. Nobody is keeping anyone from being a programmer if they're female or being a nurse if they're male. I'm sorry that's just a non-issue. We don't need to try and ensure that every single vocation has a perfect balance or race and/or gender. All we need to do is make sure that nobody is barred from any career path based only on their gender or race. Thinking like this should just be called -too many straight white men over there- because that seems to be the only group anybody is interested in making sure there aren't too many of in a given area. This just about is never applied to any other group.
reply
Yeah when you complain about the AI making things -a little more difficult- or -frustrating- then you've really got nothing to complain about. So Google image shows pictures of nurses as women and programmers as men. More women are nurses and more programmers are men. Nobody is keeping anyone from being a programmer if they're female or being a nurse if they're male. I'm sorry that's just a non-issue. We don't need to try and ensure that every single vocation has a perfect balance or race and/or gender. All we need to do is make sure that nobody is barred from any career path based only on their gender or race. Thinking like this should just be called -too many straight white men over there- because that seems to be the only group anybody is interested in making sure there aren't too many of in a given area. This just about is never applied to any other group.
reply
education
Many people are missing the point to the Google analogy. AI hiring systems will learn associated characteristics of a nurse or programmer or what have you from similar datasets. That's not so much the problem- it's what happens next. It discriminates against people who don't meet the average characteristics. The AI system may throw out a resume for a nursing position that has the words -Boy Scout troop leader- because that's not something associated with the average nurse. It may throw out qualified programmer resumes from people who attended HBCUs, because most programmers haven't. If you don't quite get this, please look up the scrapped Amazon AI hiring program. It downgraded resumes from applicants who attended women's colleges.
reply
Many people are missing the point to the Google analogy. AI hiring systems will learn associated characteristics of a nurse or programmer or what have you from similar datasets. That's not so much the problem- it's what happens next. It discriminates against people who don't meet the average characteristics. The AI system may throw out a resume for a nursing position that has the words -Boy Scout troop leader- because that's not something associated with the average nurse. It may throw out qualified programmer resumes from people who attended HBCUs, because most programmers haven't. If you don't quite get this, please look up the scrapped Amazon AI hiring program. It downgraded resumes from applicants who attended women's colleges.
reply
nantukoprime
Did a short stint working on an algorithm that looked for potential pickpockets, trained on video of actual incidents that led to arrest.
Was moved to another project after I kept bringing up the fact that the algorithm was biased as the data set was generally representative of a subset of pickpockets, the ones who get caught. My request for video of successful pickpockets that were not arrested to train the algorithm was not viewed favorably.
reply
Did a short stint working on an algorithm that looked for potential pickpockets, trained on video of actual incidents that led to arrest.
Was moved to another project after I kept bringing up the fact that the algorithm was biased as the data set was generally representative of a subset of pickpockets, the ones who get caught. My request for video of successful pickpockets that were not arrested to train the algorithm was not viewed favorably.
reply
crash_course
Should a program be faulted for showing mostly female nurses? 91% of nurses are female. Should it be faulted for recognizing more white people? The United States is 72% Caucasian. It seems silly that we try to tell computers lies, so that their results don-t hurt anyone-s feelers.
reply
Should a program be faulted for showing mostly female nurses? 91% of nurses are female. Should it be faulted for recognizing more white people? The United States is 72% Caucasian. It seems silly that we try to tell computers lies, so that their results don-t hurt anyone-s feelers.
reply
Gravity
Have you guys covered politics much? I know it can be touchy, but I'd like someone in a good position to do so. To explain the issues with things like the party system and gerrymandering, and what you can legally do to change it instead of letting things go until the levy breaks.
reply
Have you guys covered politics much? I know it can be touchy, but I'd like someone in a good position to do so. To explain the issues with things like the party system and gerrymandering, and what you can legally do to change it instead of letting things go until the levy breaks.
reply
Clare
And some are deliberately biased. This is how cultural manipulation through government black projects is done here: let private corporations censor unspecified classes and it's not illegal. Only governments can be called illegal for suppressing speech on line. Nice trick!
reply
And some are deliberately biased. This is how cultural manipulation through government black projects is done here: let private corporations censor unspecified classes and it's not illegal. Only governments can be called illegal for suppressing speech on line. Nice trick!
reply
Ramon
Trash, algorithms tell facts, they ones who are biassed are people with that -equlity- ideology. Races are not equal, peopple from defferent ages have different midnsets, there is only men and women and they also have different nature. The AI shows it by its results.
reply
Trash, algorithms tell facts, they ones who are biassed are people with that -equlity- ideology. Races are not equal, peopple from defferent ages have different midnsets, there is only men and women and they also have different nature. The AI shows it by its results.
reply
Add a review, comment
Other channel videos
