Overworked AI ? — when AI takes over our human decision-making
  • 02 Apr 2024
  • 12 Minutes to read
  • Dark

Overworked AI ? — when AI takes over our human decision-making

  • Dark

Article Summary

Thank you to Kem-Laurin Lubin, PH.D - C for sharing her insight and knowledge with us.

Click here to read on Medium.

AI determinism & solutionism impacting healthcare futures

“Beware the day when artificial intelligence exceeds human intelligence, for the fate of mankind may hang in the balance.” — Stephen Hawking

Many years ago, in a previous chapter of my life at Siemens Research’s Princeton Lab, I was deeply engrossed in a passion project: integrating electronic patient records into the healthcare workflow. This research aimed to provide physicians with a powerful tool to enhance patient (soldier) care, particularly field surgeons working for the Department of National Defense (DND), with an innovative tool to elevate patient care. During a time when mobile phones and tablets were emerging technologies, we pondered the question: How could we enhance the timely delivery of medical services to injured soldiers? Moreover, we contemplated how such an endeavor could be amalgamated and cultivated within the frameworks of Siemens’ Syngo (now Siemens Healthineers) and Soarian Clinicals’ software suites.

What was unique about this research was enabling patient input and sense making with connection to their medical patient records, not too far removed from what companies like Fitbit are doing today, powered by AI.

This initiative represented a pivotal time in my professional journey, absorbing the generous 30% of time Siemens afforded, us, researchers, for the exploration of “pet projects,” a testament to a bygone era of symbiotic company-employee engagement. This was nearly two decades ago, a reflective point of my career’s narrative.

I surmise that traces of this project still linger in the recesses of my past, perhaps ensconced within a neglected CD, containing my work.

My tenure with Siemens came to an end when I relocated back to Canada in the aftermath of 9/11. This decision was a result of joint deliberation, heavily influenced by the trauma experienced by my Dutch partner, who was deeply shaken by our proximity to the events of that tragic day. Interestingly, during my time at Siemens, my partner was employed by the Dutch competitor, Philips. Despite our differing professional affiliations, we concurred that raising our children in Canada would be in their best interest.

Those memorable days have received new visitation considering, now, the state of Healthcare, compounded by the advent of Gen AI.

Pivoting —from user research and ethnography to computational rhetoric

Today, I find myself on the verge of defending a doctoral where my research spans traditional rhetoric, computational methods, and AI identity studies. My work primarily focuses on examining the relationship between AI and identity characterization. My research extends into computed identities in healthcare, with a nuanced focus on women’s care in the aftermath of Roe v. Wade reversal. My forthcoming joint article in the Rhetoric Society Quarterly’s AI-themed issue, “Sex after Technology,” contemplates the development of digital tools akin to those in abolitionist movements to mitigate the detrimental effects of AI, within this context, and beyond. this is depicted more in an upcoming long article on Surveillance Rhetoric. But more on that later.

I provided back story as context to establish a framework for the substantial changes that have transpired since the conversation surrounding the digitization of electronic patient records and the stringent regulations of the Health Insurance Portability and Accountability Act of 1996 (HIPAA, also known as the Kennedy–Kassebaum Act) that once safeguarded the sacred access to patient records. Times have evolved, reshaping healthcare delivery, and the role of the patient/user as an active holder of their potential health data is now the new norm.

Currently, the rapid advancement of AI in healthcare delivery, notably by entities like Google and its subsidiary, Fitbit, is both evident and at times unsettling, which I won’t get into now. My observation is that amidst this technological evolution, there exists a fervent race propelled by capitalist incentives, with tech companies potentially neglecting the necessary ethical considerations for the access and use of human data.

This trajectory, guided by the twin specters of technological determinism and solutionism, represents a critical concern, suggesting that our collective humanity might be at risk of being overshadowed by the relentless pursuit of technological progress.

But such is the nature of the times when there is an app for everything.

Related post

I really do try to remain impartial in these posts but I am often left asking- Who is Watching the Watchers?

Inthis post, I explore significant announcements of the last couple of years from tech giants Fitbit, and its parent Google, and, using them as examples, examine the implications for healthcare futures. Specifically, I focus on Gen-AI and its potential to revolutionize the healthcare sector, as narrated by the strategies of Google. Within this frame, I also address the necessity of considering the broader context surrounding these technological advancements and the surrounding hype.

It’s crucial to question whether we are relegating the essence of humanity to a mere afterthought in the face of technological determinism and solutionism?

In this analysis, I unpack the nuances of Google’s and Fitbit’s latest ventures into healthcare, exploring how Gen-AI is poised to reshape patient care, diagnostics, and personalized medicine. I will also briefly discuss and scrutinize the ethical, social, and practical dimensions of integrating such advanced technologies into our health systems. As we unfold this technological evolution, and take on new frenzied postures, it’s imperative to assess whether these innovations genuinely enhance human well-being or if they inadvertently prioritize technological progress over the core values of compassion and empathy in healthcare.

Addressing this we can also shed some light on the delicate balance between technological advancement and the preservation of humanistic principles in the medical field. But first, for those who need a quick context and orientation about the two companies affiliations and digital healthcare needs.

External link

And for more historical context, here is a brief about each of the two companies:

Fitbit, known for its wearable fitness technology, has expanded its capabilities to include advanced health monitoring. It utilizes artificial intelligence (AI) and data analytics to offer proactive health management, providing users with personalized insights and recommendations. In 2021, Fitbit was acquired by Google, indicating a shift towards integrated health solutions within the technology sector. This acquisition has allowed Google to extend its presence in the wearable technology market and has highlighted the merging of health technology with AI, potentially leading to new developments in personal and preventive healthcare.

Google, a global technology company, has been active in AI innovation, with applications in various sectors, including healthcare. Prior to acquiring Fitbit, Google initiated “Google Health” programs, which utilized AI to improve healthcare delivery, support clinical decision-making, and offer personalized health coaching and management tools. The acquisition of Fitbit expanded Google’s healthcare offerings, incorporating wearable technology into its product range. This acquisition reflects Google’s integrated approach to healthcare, combining its AI capabilities with Fitbit’s technology to enhance patient care. Google’s research and partnerships in this field position it as an influential entity in healthcare technology, illustrating AI’s role in advancing the sector.

Although not explicitly stated, at the heart of this blog’s inquiry lies the interplay between technology and its expanding array of seemingly indispensable applications, akin to civil utilities, which are gradually supplanting conventional lifestyles. One notable example is observed within the healthcare sector — there are now apps or that.

Specifically, this exploration unintentionally, explores the ramifications of technological progress on societal norms, economic frameworks, and the broader societal fabric. Moreover, it prompts crucial inquiries regarding the degree to which we entrust technology to mold the trajectory of our healthcare destinies. This, by the way, is a growing topic here in Canada, where I live.

Are we overestimating the capability of technology to address complex health issues?

What are the potential risks and rewards of integrating advanced technologies in healthcare, considering the growing reliance and an overt over-indexing of technical solutions at the expense of human-centric approaches. By investigating these themes, I want to open up a conversational space and provide a balanced perspective on the sustainable integration of technology in healthcare, with respect to the lineage of privacy as a non-negotiable. I hope this can foster some dialogue on finding the right equilibrium between technological innovation and maintaining the human touch in patient care.

Outsourcing our health to AI and Tech

Incontemporary society, Gen-AI is emerging as a pivotal force, particularly within the healthcare domain, serving both as a driver and mirror of evolving perceptions of well-being. This phenomenon not only reflects technological advancements but also signifies a profound transformation in our attitudes and behaviors towards health, as well as the privacy surrounding our healthcare data. Today, there is an observable trend towards the quantification of nearly every aspect of human existence — from tracking steps and monitoring diabetes to measuring heart rate and now even capturing breathing patterns for asthma management.

At the heart of this technological revolution lies the conversion of raw data, often collected through mobile applications, which was previously considered private, mirroring the principles of the Health Insurance Portability and Accountability Act (HIPAA) in traditional medical practice. Presently, much of this data is being commodified and leveraged as part of large-scale tech initiatives aimed at providing actionable insights based on personalized health informatics. This paradigm shift represents a departure from mere data accumulation for its own sake towards data-driven decision-making in healthcare.

This evolution marks the advent of self-directed healthcare, empowering individuals to take proactive control of their well-being with the assistance of technology. As a result, individuals are increasingly engaged in managing their own health outcomes, guided by personalized insights derived from the analysis of their health data.

Google’s annual Health Check-Up event, for example, reflects the company’s dedication to leveraging AI for healthcare improvement. Their initiatives encompass a spectrum of endeavors, from enhancing clinical decision-making to providing personalized health coaching, catering to the needs of both healthcare providers and consumers. Moreover, the imperative to streamline administrative tasks in healthcare underscores the importance of technological intervention. AI assists healthcare providers in alleviating administrative burdens, thereby allowing them to prioritize patient care. It’s noteworthy that AI serves as a complement rather than a replacement to medical professionals, emphasizing its role in augmenting medical literacy and providing decision support.

Looking ahead, the company’s advancements in visual search engines and personalized health coaching via Fit Labs, also, indicate a shift towards consumer-centric healthcare. These innovations, spanning from diagnosing skin conditions to simplifying intricate health concepts through visuals, epitomize the amalgamation of technology and empathy, I hope.

Lastly, Google’s Gemini large language model, integrated into medical settings, signifies a significant advancement in AI-enabled clinical reasoning. By contextualizing queries and facilitating meaningful patient-provider interactions, Google’s view of AI seems to hold some promise of transforming healthcare delivery.

While I present Google’s strategic positioning here I also want to say I am a neutral observer of AI as a player. As I have always contended, elsewhere, it is neither good or bad.

What I do support is the responsible use of AI that considers aspects I have mentioned elsewhere and which I will again reference below. All of these factors focused on indexing the human user. They included myriad considerations, touching on areas like policy, governance, morality, privacy, bias, equity, design, and risk, among others. Let me briefly unpack what I mean in this quick list and the AI considerations, in quick form:

1. Crafting policy for AI involves a nuanced approach that must both propel technological progress and uphold ethical standards, setting a stage where innovation thrives within a responsible framework.

2. Governance in AI is about establishing clear lines of accountability and ensuring that AI systems operate transparently and ethically, aligning with societal norms and values.

3. The moral dimension of AI prompts a deep reflection on how technology impacts human decision-making and societal norms, urging a careful balance between technological advancement and ethical integrity.

4. Privacy issues are paramount, as the capacity of AI to analyze and store personal data necessitates stringent safeguards to protect individual rights and prevent misuse.

5. Addressing bias in AI systems is critical to avoid reinforcing existing social inequities, requiring ongoing efforts to ensure fairness and impartiality.

6. Equity in AI emphasizes the need for broad access to this technology, ensuring that its benefits are shared across all strata of society and do not exacerbate social divides.

7. In design, AI must be user-centric, crafted to meet real human needs and adaptable to diverse social and cultural contexts. Moreover, understanding and mitigating the risks associated with AI use is essential to prevent unintended consequences that could arise from its deployment.

In the world of consumer electronics, the emergence of local AI models signifies a fundamental shift in technology interaction. The capability to execute advanced AI algorithms on smartphones opens doors to personalized healthcare experiences. The trajectory of Gen AI in healthcare presents both excitement and challenges. As we embrace AI futures, it’s imperative to ensure that technological advancements are utilized for the collective good. Let us collectively embrace the transformative potential of AI and work towards fostering a healthier and more equitable future.


American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.). https://doi.org/10.1037/0000165-000

DeepMind. (n.d.). Gemini: a conversational AI platform for healthcare. Retrieved from https://deepmind.google/technologies/gemini/#introduction

Hadjiat Y (2023) Healthcare inequity and digital health–A bridge for the divide, or further erosion of the chasm? PLOS Digit Health 2(6): e0000268. https://doi.org/10.1371/journal.pdig.0000268

Related Posts

About me: Hello, my name is Kem-Laurin, and I am one half of the co-founding team of Human Tech Futures. At Human Tech Futures, we’re passionate about helping our clients navigate the future with confidence! Innovation and transformation are at the core of what we do, and we believe in taking a human-focused approach every step of the way.

We understand that the future can be uncertain and challenging, which is why we offer a range of engagement packages tailored to meet the unique needs of both individuals and organizations. Whether you’re an individual looking to embrace change, a business seeking to stay ahead of the curve, or an organization eager to shape a better future, we’ve got you covered.

Connect with us at https://www.humantechfutures.ca/contact

Was this article helpful?