GS Bank CEO: “I think what we're realizing this week is that there's an important conversation to have around people's awareness of what their credit history says, what it shows, and what the facts are in any individual credit history" 😱 pca.st/q19uhpso
8
14
7
89
That is just wild. The bank CEO is basically victim-shaming everyone who’ve had a sexist outcome from their black-box algorithm under the banner of “your credit score and record was probably just shit, honey”. Hear that @stevewoz? Your wife just needs to study harder!
6
7
71
Again, let’s recap our case: 1) Same household income submitted for both applications, 2) Wife has longer credit history, no balance run for about a decade, 3) Wife has higher credit score. ALGORITHM: Wife is twenty times less worthy of credit than husband.
6
16
1
62
The GS bank CEO also goes on to defend the complete lack of transparency into their black-box algorithm because “there’s information out there about how scores are determined”. GS doesn’t have to reveal anything because they paid a consultant to write them a report saying 👍
3
5
33
The hubris here is the stuff of Wall Street legend. They’re so utterly confident in the black-box algorithm that they themselves cannot explain, that it’s simply IMPOSSIBLE that the computer got it wrong. Wow.
5
5
2
39
doesn't it just make you want to unleash a few good exploratory testers against their spreadsheets? (let alone THE ALGORITHM)
3
12
Replying to @martinfowler @dhh
Shouldn’t testing the algorithm for bias be extremely easy? Submit the otherwise exact same data with different genders; check whether outcome is the same

Nov 14, 2019 · 7:09 AM UTC

4
1
6
Replying to @stilkov @dhh
I believe not. Even if you removed or reversed an explicit gender field, it might be inferring from (for example) the name of the person, which would correlate to gender. I suspect gender would show up in various surprising places.
1
8
Point taken with regards to the name. Unsure about other aspects though. I’m sure there’s some analysis that’s been done about this already that I just don’t know about.
1
The twitter feed of @math_rachel is full of great explanations how bias can implicitly and quite subtly creep in. Just because gender isn't an explicit variable in the algo doesn't mean it can't figure out the gender by itself and then discriminate based on that.
2
4