A cautionary tale relating to artificial intelligence surely startled many at SXSW this week. Dr. Joy Buolamwini is founder of the Algorithmic Justice League. Fortune magazine named her the “conscience of the AI revolution.” I will refer to her as Dr. Joy.
Her talk centred on The Coded Gaze, a term invented to describe the biases in algorithms that perpetuate inequity. “The Coded Gaze,” she explained “is really a reflection about power: who has the power to shape the technologies with both their preferences, their priorities and at times their prejudices. I first had an encounter with The Coded Gaze as a grad student at MIT where I was working on an art project.”
When she installed software designed to track her own face, the software literally was unable to detect her! It was, however, able to scan her if she wore a white mask! To test her own conclusion that the face recognition software was faulty, “I took my own TED profile image I ran it through various AI demos and I found that some didn’t detect my face at all, and the ones that did detect my face labelled me male!”
Dr. Joy got even more concerned “when I realized these technologies were entering the real world and messing with people’s lives, including false arrest. Anybody who has a photo posted on social media is likely in a database collected by Clearview AI which has over 30 billion photos online. And now with Generative AI systems, it’s not just the risk of being misidentified or matched with somebody else: your own face can be used in a deepfake. We saw this with Tom Hanks advertising a dental package he never knew about. No one is immune!” Dr. Joy has created a word for anyone who has been harmed by AI systems: excoded.
The TSA
In 2023 the TSA announced it would expand its facial recognition program to more than 400 airports to make it the default way to travel. “You will also see these systems in school, and some hospitals. Since we have some ‘issues’ I decided to start the Algorithmic Justice League to really see how we can build towards a world with more equitable and accountable AI. What got the AJL started was that example of coding in a mask with a white face to be detected, and also this research known as gender shades, which was my MIT thesis work.” She looked at the face recognition software developed by various companies. There was, for example a 34% error gap in the IBM software when it came to differentiating darker females with lighter males. A follow-up study showed that the software used by Amazon was a year behind its competitors in its ability to recognize faces properly.
“Most people don’t even know they can opt out. Even when a TSA officer tells you to step up, you actually have a right to refuse. We continue to push our Dept. of Homeland Security on this as well. This is part of a broader effort around broader biometric rights: not just the right to refuse, but your right to have your data deleted if it’s already been collected. The right to have non-consensual deepfakes created of you as well.”
The Poet of Code
Dr. Joy creates art to illuminate the impact of AI on society. She is now known as the Poet of Code. At her SXSW appearance, she read a new poem entitled Precisely Who Will Die? I found her presentation to be one of the most valuable and powerful of any that I attended at SXSW this year. Here are the first few lines of her poem:
Precisely Who Will Die?
Some say AI is an existential risk.
We see AI as an exterminating reality,
accelerating annihilation,
augmenting destruction.
I heard of a new gospel,
delivering death with the promise of precision,
the code name for an old name.
To target your enemy, the other
reduced to rubble.
Face erased, name displaced
As the drones carry out in a formation
that spells Last Shadow.
I highly recommend her latest book, Unmasking AI: My Mission to Protect What is Human in a World of Machines. It is published by Random House.
Another important SXSW session that drew attention to the risks of AI was by the creators of an Academy-award-winning film. My report can be found at this link: