There are several wonderful things about the book club I’ve been in for many years: thirteen women get along, we usually quickly agree on each other’s book suggestions, and when we don’t, it’s friendly and amicable. Perhaps the best thing, besides friendships that have grown over years is that we read selections that I’d otherwise never give a second look. Take a recent choice, Weapons of Math Destruction, by Cathy O’Neil. Words are my forte. I’ve always struggled with math and I’m not interested in reading about it. 
However, I opened Weapons of Math Destruction because the purpose of book club is to read the book selection and not just show up for wine, dessert and chitchat (now virtual). I will admit, the book has a clever title, and as an author myself, I know titles are of paramount importance for grabbing the attention of would-be readers. (See her Ted Talk https://www.youtube.com/watch?v=_2u_eHHzRto ) Also, the surprisingly understandable first two pages of the introduction were without mind-numbing numerics. On page three, the author began discussing Washington, DC’s 2007 controversial public school reforms that were initiated by a new chancellor, Michele Rhee. At that time, I was a writing teacher, and I followed the case with great interest.
DC public schools needed improvement and bad teachers needed to be weeded out. But a carte blanche approach that ignored anything to do with the human factor turned out to be greatly flawed. In her investigation, O’Neill follows the trajectory of one teacher and how the new scoring system was supposed to measure her effectiveness at teaching math and language skills.
The algorithm’s results only measured her student’s test scores, eliminating all positive input such as administrative reviews and community/parental input.
Some children, especially those who started way behind, had improved, but the algorithm only produced a number, which was unsatisfactory and didn’t meet the new criteria for satisfactory teaching.
Yes, the new algorithms weeded out some bad teachers, but it also caused the firing of very good teachers who’d been making progress with often formerly struggling students. O’Neil’s book helped me clearly understand how algorithms wield such power, often to great detriment for individuals involved.
O’Neil didn’t set out to be a writer. In 2008, when the economy collapsed, before she began studying algorithms, she was working for a hedge where algorithms figured prominently in her work. It was then that she became aware of how bias and preconceived assumptions built into the framework of many algorithms too often have a destructive result for the individuals involved.
Beyond the DC school revelations, O’Neill gives many disturbing examples of algorithms causing blatantly unfair results that ruin lives. There is often bias in prison sentencing, home purchasing, banking, and other offenses against the truth.