A feature, not a bug?

Perec’s awkward foray into thinking through categorization of snow using the problematic term “Eskimos” is an own-goal. He highlights the power imbalances and reductionist processes inherent in any classification system even as he muses on the limitations of codification: which comes first, the thought or the classification?

But he writes that it is “so very tempting to want to distribute the world in terms of a single code.” Is he universalizing this tendency? Just as bias is a feature, not a bug of artificial intelligence, can human intelligence, or even human language function without some kind of categorization? Foucault writes of the unease we feel at the disorder of inappropriately linked ideas not just because it is incongruous but “in which fragments of a large number of possible orders glitter separately in the dimension.” Do we all cling to systems of order to stave off the queasiness of unbounded uncertainty? And if so, is this what keeps systems of control in knowledge organization in place—and their unequal human consequences—in place? When we swap higher ideals of justice for (the pretense of) utopian order, what dystopias have we signed up for?

Where Drabinski highlights that critiques of classification and categorizations have a long history and are not new conversations, Crawford situates why this matters so crucially in our current context. Critical theorists may have been pondering these questions for a while, but data practitioners, whose work has huge implications for us all, have embraced codification and run with it. The scale and scope is unprecedented. Machine learning algorithms function through and inevitably embed biases both on matters of allocation and representation, the latter having more insidious and long-ranging impacts. It matters at an epistemological level then that categories are increasingly applied in attempt to naturalize and essentialize that which we consider the social, relational, cultural.

As highlighted in Crawford’s presentation, an example of bias in machine learning, and how it makes its way into everyday systems. Using Google to translate “he is a nurse, she is a doctor” into a language like Turkish without gendered pronouns, then reversing the translation, the AI automatically equates nurse with “she” and doctor with “he.”

One Reply

  • Thanks, Lydia! A couple of your classmates have also wondered if intelligence and/or language could exist *without* classification. I hope we’ll be able to address this in today’s discussion. I also appreciate that you’ve extended Derrida’s psychoanalysis of the archive to the classification realm: what explains the classification drive? What “teloi,” or ideologies underlie these systems? We can ask such questions of our historied ordering systems — and our new classification-based AI systems.

Leave a Reply

Your email address will not be published. Required fields are marked *