IMPACT COMMUNICATION
AND INNOVATION
AND INNOVATION
Last week, I watched a TED Talk by technology sociologist Zeynep Tufekci, which inspired the title of this piece: We’re building a dystopia just to make people click on ads.
Tufekci discusses how we use dystopias like 1984 George Orwell’s to address issues of privacy, surveillance, and big corporations today. However, this work, written in 1948, no longer fits our current context, as it didn't deal with the specific type of control we are heading toward right now.
The sociologist points out that the problem isn't artificial intelligence itself, and it is highly unlikely that we will see an AI turn against us, as the headlines referencing the film The Terminatorfear. For her, it is the use of the power of artificial intelligence, big data, and machine learning by large companies like Facebook and Google that is concerning.
The author of the book Twitter and Tear Gas: The Power and Fragility of Networked Protest (2017) notes that both programmers and data analysts have already lost track of what is happening with the algorithms they created. There are thousands of lines and thousands of pieces of information that have escaped their analysis due to the massive amount of data being manipulated.
Even so, this technology has been applied, for example, by Facebook. During the 2012 and 2016 US elections, the social network conducted an experiment where users could click to confirm whether they had voted for the new president. Some users had access to data showing which of their friends had already confirmed their vote. As a result, 270,000 more people were influenced to vote in a country where voting is not mandatory.
Profiling
In another case cited by Tufekci, a programmer told her he had created an AI capable of analyzing social media posts, such as on Twitter, to see the type of content and messages users were publishing. This allowed for a pre-diagnosis of individuals showing signs of manic behavior even before they were clinically diagnosed. The predictions, as the sociologist notes, were highly accurate.
Similarly, Tufekci gives an example of how to program online advertising based on user profiles. How are these profiles created? From the information we consciously share on social media and the information we don't realize companies are archiving—such as posts we delete, which Facebook keeps in its database and uses to analyze your profile, according to the sociologist. Beyond that, various other companies specializing in the field collect everything from simple data, like our browsing history, to more obscure information to resell to those interested in turning it into a targeted advertising profile.
In this sense, Tufekci posits a company that wants to advertise tickets to Las Vegas. It can target people who have searched for flights to Las Vegas, or it can target people whose profiles demonstrate certain behaviors and impulses, such as gambling addiction or compulsive shopping, thereby exploiting these personal characteristics and dysfunctions to sell their products and services. The issue here is no longer about the technology itself, but about the morality behind it and its application.
Hacking the system?
Also last week, on November 21, a French fab lab was vandalized and set on fire after becoming known as an “institution notoriously dangerous for its dissemination of digital culture.” According to the perpetrators, the La Casemate fab lab was the result of municipal administration actions intended to “satisfy money-hungry startups and geeks opening fab labs in trendy neighborhoods. These extremely heterogeneous-looking devices aim to accelerate the social acceptance and use of technologies of our disastrous times.”
In his article analysis of the case, French lawyer Jean-Baptist Soufron argues that this is not the first time people have acted violently against computers. From 1979 to 1983, in France, there was a committee known as CLODO, which aimed to "liquidate" or "subvert" computers by planting bombs and burning down technology company buildings.
At that time, Soufron recounts, those responsible declared to the French media that they were "data processing workers and therefore well-positioned to speak about the current and future dangers of data processing and telecommunications." They further stated, "the computer is the preferred tool of the ruling class. It is used to exploit, to register, to control, and to repress."
Later, in 1984, a Belgian group called the Communist Combatant Cells (CCC) bombed and destroyed the headquarters of several technology companies in Germany and Belgium. In London, a group called the Angry Brigade attempted the same, and other similar actions, as Soufron points out, also took place in Asia, South America, and the United States.
The French lawyer insists on emphasizing that it is very easy to call these groups neo-Luddites, when, in fact, unlike the original Luddites, who broke machines because they were taking their jobs, these people have a different focus for their actions.
In 1983, CLODO gave a rare interview to an English publication, in which they provided the following explanation: "It is neither regressive nor new. Looking at the past, we see only slavery and dehumanization, at least if we go back to certain so-called primitive societies. And although we do not share the same 'social projects,' we know it is stupid to try something and [want to] turn back the clock."
In other words, it is as if these tools were "perverse from their origin," such as when we consider that "the majority of computerized sectors are the military, and that 94% of civilian computer usage time is spent on administration and accounting." According to this 1983 opinion, "if microprocessors generate unemployment instead of reducing an employee's working hours, it is because we live in a brutal society, and that is by no means a reason to destroy microprocessors."
New anti-tech critique
Soufron then points out that since that same year, 1983, a film called WarGameshad already established the connection between hackers, terrorism, and violence. By insisting on this theme, the lawyer writes, "it seems that digital politics, protests, and violence only happen in a kind of virtual world, and that they belong to a gray zone where moral values are distant and confused."
Soufron then points out that since that same year, 1983, a film called WarGameshad already established the connection between hackers, terrorism, and violence. By insisting on this theme, the lawyer writes, "it seems that digital politics, protests, and violence only happen in a kind of virtual world, and that they belong to a gray zone where moral values are distant and confused."
As Soufron explains, the vocabulary used there, which still remains a linguistic reference when treating the "good" hacker as white hat and the "bad" hacker as black hatmakes it seem as if the scenario is much more suited to the imagery of, for example, The Lord of the Rings than of Communist Manifesto. “And ever since the seminal 1984 book written by Steven Levy, Hackers: Heroes of the computer revolution, the digital world has been fascinated by these new stories, which claim to be capable of breaking society's rules. Their leaders always want to present themselves as revolutionaries. They always started in garages. They are always former hackers. They all want to change the world,” points out Soufron.
But, for the lawyer, it seems that is no longer the case. After having thrown stones at a Google bus (an act criticized by theorist Douglas Rushkoff and also by Soufronhimself), these new opponents of technology seem to be questioning what is truly revolutionary or prophetic about an industry that relies on antiquated capitalism, monopolies, microwork, state regulations, and has money as its main guiding principle.
“Just as they reject the hacker myth, they also call a renowned place like MIT the ‘temple of technocracy’—people who are surprised, offended, or shocked by this description should remember the way Aaron Swartz was driven to suicide after being mistreated by that institution,” writes Soufron.
Finally, the lawyer ends his text by stating that violence must indeed be condemned, but that this more extreme act is merely a response to the impulse of a “religion of progress and technology that has been sold as an inescapable future for everyone.”
Soufron comments that the members of CLODO were never caught, but that they ended up being dropped from the agenda because a new storytelling of a supposed digital transition revolution had gained more prominence. Despite this, Soufron does not believe there is a new story to be told:
“As the promises of this then-considered revolution now fail to be realized, it is only predictable that people will return to actually protesting and being truly violent as a result. It is not surprising that, after 20 years of obscurity, people are starting to remember this. With the negative impact of the digital industry being felt ever more strongly by citizens, actions will not be restricted to former big tech professionals speaking at elite conferences, or to the indignant articles of social science researchers as they are funded by tech companies. Technology is not neutral and there is no such thing as a virtual world.”
The vicious cycle of dystopia
However, it is this same anti-technology and anti-corporate narrative that has been told by dystopian science fiction. If 80s cyberpunk brought anti-corporate stories, but used technology and the figure of the hacker as a new hero—who is criticized by these new revolutionaries and by Soufron’s text—what happens to the “true opposition” in this 21st-century context?
What we see in the current pop culture imagination are companies like Netflix using the same algorithms highlighted by Tufekci to create series and films that respond to the search and cultural consumption desires raised by their users' habits—and this has been happening since House of Cards to Stranger Things and even Black Mirror.
Although it was originally a Channel 4 product, it was upon arriving on Netflix that the series Black Mirror received a new boostin popularity and influence on the popular imagination, and with it, a new direction for its narratives. The series has always offered a critical and even pessimistic analysis of technological trends, but this time it fell into a vortex of negativity and narrative predictability, where the show's very premise is no longer just pointing to the possibilities of how technology is applied and its potential (not exclusive) disastrous outcomes, but rather that something catastrophic will happen because of technology regardless, all carried by a subliminally moralistic tone.
On the other hand, we also have the case of the series Mr. Robot, which won over not only those working in information security who could relate to that narrative, but also gained the appreciation of cyberpunk science fiction fans without necessarily being a work of science fiction stricto sensu.
In the final seasons, we saw in the figure of Elliot not only the stereotype of the revolutionary hacker, but we also had the opportunity to see that within those same corporations criticized by post-CLODO groups, there are people playing a double game, and that this revolution may not be as simple as a hacked website; it can involve a lot of violence, just as it did in La Cotemade and as the series depicted through the death of the character Susan Jacobs, for example.
In other words, when Tufekci speaks of dystopia, the sociologist is addressing how algorithms applied to advertising are creating a dystopian context for populations due to the immoral way this technology can be used to target audiences unethically, based solely on a goal of profitability.
On the other hand, my interpretation and expectation regarding the title of her TED talk was to understand how algorithms are identifying these very profiles that are hungry for dystopias, and in this way, content production companies, such as Netflix, take advantage of this mindset to reinforce a line of thinking that is not necessarily accurate, but which massages a prejudice long since constructed by dystopian science fiction itself.
And so we fall into a vicious cycle, in which we not only have a devaluation of the genredue to the impoverishment of narratives, as we also see a culture industry based on fear (see commentary by futurist Tiago Mattos on the new filmSingularity), because fear, after all, sells.
The question that remains, then, is whether this strategy doesn't also end up being another offshoot of post-truth, since we offer what people want to see and hear, but not necessarily what "is." The quotation marks here serve as a suggested answer to the question, since when we deal with fiction, we aren't dealing with truth or lies, but perhaps with plausibility in certain genres. In other words, what is meant by drawing this comparison is that art is now being made on demand—a commission that we, the new patrons turned streaming service subscribers, want—even if perhaps only the algorithm knows it. It’s what we want to see and hear, but perhaps not what we need.
At this point, Soufron’s text offers us a perspective that goes beyond the neo-Luddite flattening of anti-tech corporate criticism. Rather than weaving a one-sided, shallow commentary against these companies, groups like CLODO or acts like the vandalism of the La Casemade fab lab show that the problem is far more complex than the fear that machines will take our jobs or that robots will turn against us and destroy humanity.
Given this, perhaps even pop culture is failing to satisfactorily translate this context, whether due to the fear of not capturing an audience thirsty for catastrophe, or simply because of the sheer complexity of the subject.
