While I was deep into drafting a post on AI and the division of labour, my best friend interrupted me with an urgent phone call. She was eager to share that Timnit Gebru had published another article, “The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence.” For those familiar with the field of AI, especially the use of Large Language Models (LLMs) and the controversies surrounding them, you’ll remember Gebru. Her highly-cited co-written research paper, “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” co-authored with Emily Bender and others led to her controversial departure from Google in 2021.

This incident has captured my attention for some time, making me keen to see what Gebru would do next. Her latest work, co-authored with Émile P. Torres did not disappoint.

Gebru, an ethicist whose work primarily focuses on the ethical implications of AI and machine learning technologies, emphasizes the importance of addressing bias and fairness. Her earlier co-authored paper, with Bender, not only led to her departure from Google but also highlighted the limitations of the company’s machine learning protocols, and underscored fundamental flaws in how AI should be perceived and developed. It also raised many questions about the efficacy of Gen-AI. In her work, Gebru noted the significant bias against Muslims and saw that as indicative of the underlying flaws in LLMs.

In the paper that precipitated her ousting, Gebru and her co-authors argued that AI systems often mirror and perpetuate existing societal biases, particularly those related to race and gender. This experience has driven her to advocate for greater transparency, accountability, and diversity within the AI research community. She stresses the necessity for interdisciplinary approaches that incorporate social sciences and other disciplines.

My research sits at the confluence of Computation and the Humanities, specifically focusing on computational rhetoric. I explore the crafting of digital identities and their profound impacts on human conditions within the extensively quantified landscapes of our analogue worlds. An upcoming chapter of my dissertation, set to feature in a special issue of Rhetoric Society Quarterly (RSQ), my co-writer, and supervisor, Randy Harris explore the dynamics of sexuality at the intersection of AI. This work offers a vital critique, exploring how digital technologies continuously reshape and redefine human interactions and societal norms — and by extension, impact them materially.

More recently, Gebru and Torres, in the long awaited seminal publication coined the acronym TESCREAL to critique a convergence of futurist ideologies; they are, namely: transhumanism, extropianism, singularitarianism, cosmism, rationalism, effective altruism, and longtermism.

Their paper examines how the ideological roots of Artificial General Intelligence (AGI) development can be traced back to the eugenics movement through transhumanism and related ideologies, which they collectively referred to as the “TESCREAL bundle.”

Gebru and Torres, himself also an ethicist, philosopher, intellectual historian, view these as embodying a right-leaning bias within Big Tech, likening their proponents to ‘the eugenicists of the 20th century’ for championing projects that, while presented as beneficial to humanity, may actually be harmful.

Let me elucidate on these concepts in a brief sidebar; and, also a brief definition of AGI, in context.

AGI, as defined, collating the works of Stuart Russell, Peter Norvig, or Nick Bostrom, is a type of artificial intelligence that can understand, learn, and apply knowledge across a broad range of tasks, mimicking human cognitive abilities. Unlike narrow AI, which is designed to perform specific tasks (like image recognition or language translation),

AGI can generalize its intelligence across an unrestricted range of activities, solving complex problems and reasoning abstractly. AGI has not yet been achieved but is considered the ultimate goal in the field of AI, representing a future point where machines could potentially perform any intellectual task that a human being can.

Gebru & Torres’ Futurist Ideological “Bundle”

Transhumanism: An intellectual movement that advocates for the enhancement of the human condition through sophisticated technologies aimed at augmenting human intellect and physiology. A perfect example is developing biotechnologies that enable aging reversal, or implementing neurotechnological enhancements for improved cognitive functions.

Extropianism: This is a futuristic philosophy that supports the use of technology to extend human capabilities and life spans, promoting continual improvement and growth without limits. The idea of uploading one’s consciousness to a computer to achieve immortality is also another often themed Black Mirror tale. For example, the episode, “San Junipero,” people can choose to upload their consciousness to a virtual world, allowing them to live indefinitely in a youthful state and experience a variety of eras and lifestyles. This setup reflects the extropian belief in the potential for technology to radically extend human life and improve the quality of living, potentially even achieving a form of immortality. I opt for the finality of mortal death, myself.

Singularitarianism: The belief in a coming “singularity,” a point in time when technological growth becomes uncontrollable and irreversible, resulting in unforeseeable changes to human civilization. Today, AI is reaching a level of advanced autonomy that it starts to improve itself without human intervention. And here alone is a hurdle worth pausing in this stride for speed over ethics.

Cosmism: A philosophy that promotes the exploration and colonization of outer space as a means to secure the future of humanity. Advocating for the establishment of self-sustaining human colonies on other planets to mitigate the risk of extinction events on Earth. As if we have not done enough damage on this planet. This concept is most commonly the theme of wealthy billionaires posturing. For the greater good? I think not.

Rationalism: Perhaps the most often understood of the “bundle” is this philosophy that emphasizes reason and evidence-based thinking as the primary sources of knowledge and truth. For example, using logical deduction and empirical data to challenge traditional beliefs about the mind and consciousness.

Effective Altruism: A philosophy and social movement that uses evidence and reasoning to determine the most effective ways to benefit others, and takes action based on that reasoning. A good example is prioritizing donations to charities that are quantifiably proven to save or improve the most lives per dollar contributed. A very utilitarian ideal, indeed. But what undergirds it all?

Longtermism: An ethical focus on the long-term future and the well-being of future generations, emphasizing actions that can significantly impact the far future positively. For example, Investing in mitigating existential risks that could wipe out or drastically reduce the potential of future humanity, such as artificial intelligence safety research.

Black Mirror anyone?

Together, these ideologies form the Gebru- Torres “bundle.” Furthermore, the authors voiced concerns about research into AGI, critiquing its roots in eugenic thought and argue for a reevaluation of the pursuit of AGI, labeling it as inherently risky. Their critique raises serious concerns about the liberal utopian ideals prevalent in tech circles these days, prompting a pressing question:

utopia for whom?

The unfolding scenarios facing us in the world of AI reveal persistent power dynamics, particularly evident in labour relations, especially. It highlights — no! it begs a question we should all ask:

who performs the bulk of work and how AI is reshaping the world, echoing themes of a latent neoliberal eugenics.

In this context, I want to frame Gebru and Torres’s “bundle,” and how we are witnessing the manifestation of these neo-liberal tech “advancements” specifically in the space of labour. To me, it is clear when the authors discuss the controversial link between the historical eugenics movement and contemporary AGI pursuits. The fallacies of this need for speed to achieve next level digital utopias come at a human resource cost.

Historically, the eugenics movement, which sought to apply scientific ideas to “improve” humanity, often attracted progressives and liberals despite its discriminatory undertones — often racial in tone, recreating a world of servitude to new “masters.”

Gebru and Torres’s critique this pursuit as an extension of eugenic thinking, arguing that it sustains power imbalances and could harm marginalized groups, all under the guise of ensuring safety and benefiting humanity.

Evidentially, each of these ideologies brings a unique perspective that could significantly alter traditional business practices, worker relations, and even the overall purpose of work in society. As these ideologies gain traction, they need to be interrogated. While they are marketed as a shift towards more ethical, sustainable, and technologically integrated work environments, the receipts are in short supply. It is only creating more human and societal fractures. Frankly same ole thing — racialized division.

So, where do we pivot from here as AI power is leveraged to remake our world, and in whose image?

This ongoing transformation inevitably gives rise to critical questions about labour and the dichotomies it creates: who is working, and whose interests are truly being served? As we deploy AI and other advanced technologies influenced by these futurist eugenics ideologies, we are also seeing a division between those who control the technology and those whose labour is governed by it. This split not only echoes historical labour divides but also deepens them, placing disproportionate power in the hands of a few while many others face increasingly precarious and surveilled work conditions.

Reminds you of anything?

In light of these considerations — and the works of Gebru and Torres, among others, it becomes imperative to proceed with caution. We must foster a regulatory and ethical framework that not only keeps pace with technological advancements but also anticipates potential disparities and injustices that may arise from their application. This means implementing robust oversight mechanisms that ensure transparency and accountability in AI development and deployment.

Stakeholders from diverse backgrounds, including workers, ethicists, policymakers, and the public, must be involved in these discussions to provide a holistic view of the impacts of these technologies. Furthermore, it is essential to cultivate a culture of ethical consciousness within tech companies that prioritizes the well-being of the broader society over mere profit and efficiency.

By adopting a more balanced and inclusive approach, we can leverage the capabilities of AI and other advanced technological concepts to foster a labour environment that truly benefits all of humanity. This approach helps prevent the perpetuation of past inequalities under the guise of progressive “neo-eugenics.”

Originally, my blog was set to explore these themes until the announcement of Gebru’s latest publication shifted my focus — yes, for me, it was like hearing about a new episode of Game of Thrones. Works like those of Gebru and Torres represent the future, and while Big Tech is still puzzling over how to deliver less-than-ideal AI futures, I’ll be keeping a close eye on this space.

However, I cannot set aside my initial draft, where I began to address how these issues might evolve into a new form of indentured servitude. In my next blog post, nearly complete — I’ll explore the emerging paradigm where the ideal “worker” is expected merely to execute tasks, with AI taking the lead and critical thinking being sidelined. These, literally, are the days of our lives.

Heaven! help us all.

References

1. Bender, E., Gebru, T., et al. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency.

2. Gebru, T., & Torres, É. P. (2024). The TESCREAL bundle: Eugenics and the promise of utopia through artificial general intelligence. First Monday.

3. Gebru, T. (2023). TIME100 AI 2023. Retrieved from https://time.com

Related posts

About me: Hello, my name is Kem-Laurin, and I am one half of the co-founding team of Human Tech Futures. At Human Tech Futures, we’re passionate about helping our clients navigate the future with confidence! Innovation and transformation are at the core of what we do, and we believe in taking a human-focused approach every step of the way.

We understand that the future can be uncertain and challenging, which is why we offer a range of engagement packages tailored to meet the unique needs of both individuals and organizations. Whether you’re an individual looking to embrace change, a business seeking to stay ahead of the curve, or an organization eager to shape a better future, we’ve got you covered.

Connect with us at https://www.humantechfutures.ca/contact