Insights from repeating the same ChatGPT prompt a year apart
  • 25 Jan 2024
  • 18 Minutes to read
  • Dark
    Light

Insights from repeating the same ChatGPT prompt a year apart

  • Dark
    Light

Article summary

Thank you to Kem-Laurin Lubin, for sharing her insight and stories in our knowledge base.

Click here to read on Medium.

Evolving intelligence? comparing ChatGPT’s answers across time

“It is not the strongest of the species that survives, nor the most intelligent, but the one most responsive to change.” — Charles Darwin

InJanuary of 2023, I asked Chat GPT to generate a horror story for me. It was inspired, in part, by my dog walks to a local quarry, as well as my, then challenge, of trying to address the concerns around ChatGPT for student assignments. Back then, I was lecturing a class on Information Design; given that generative AI was a necessary topic of the day and universities were on high alert, I wondered:

“How I can teach my students a balanced view of things without saying — “you may not use AI.”

As many things are best demonstrated, I decided during my early morning dog walk that the most effective way for my students to understand the technology and assess its limitations in essay writing would be through a live, interactive demonstration. We conducted a real-time session with Chat GPT, followed by a critical discussion. My reflections on this experience are detailed in the following blog link:

At the University of Waterloo, we benefit from classrooms equipped to handle diverse technological needs. For my purpose, I required only the class microphone, my laptop, and a minute to generate a story for class discussion. The excitement was tangible, as only about 25% of the students were familiar with what I was about to introduce. At that time, only a handful of academics and professors were aware of this technology, but since it is pivotal to my research, I saw it as an excellent opportunity to engage my class in a discussion about generative AI — its benefits and limitations.

I am a self-dubbed Tech Humanist, currently immersed in completing my Ph.D. in AI Rhetoric studies. My journey intertwines the spaces of technology, human values, and communication. As a working Design Strategist, I have conducted research at the intersections of AI and design, exploring how these domains influence and shape each other.

My insights and explorations have been shared in various forums, including a notable IEEE presentation alongside Dr. Safiya Noble, the esteemed author of “Algorithms of Oppression”. My work orbits around the functional universe of AI and its societal impacts, striving to bring a humanistic perspective to the forefront of technological discourse. Some of my other work in this space includes, the talk, “A conversation at the intersection of race, AI, and technology with Safiya Noble”, a recent presentation and journal article in Springer for International Conference on Human-Computer Interaction, “ Conversations Towards Practiced AI — HCI Heuristics” and even earlier on in 2018, I write about “robots taking over jobs” in a Medium post, “Design Work Futures -The Silent Revolution.”

While my professional focus largely orbits around AI in my specific area of expertise, I also have a keen interest in horror movies, which sparked the idea to experiment with GPT-Chat in a novel context

In this post, I revisit Chat GPT prompt from 1 year ago to explore what if anything had changed in the quality of the outputs by Chat GPT, in generating a horror story. But before anything, I want to understand what had changed in the new version of Chat GPT to warrant the changes in responses.

My analysis will be partly based on a list of updates, applied where relevant, as I compare the older builds of Chat GPT software to the current version, using identical prompt. My exploration aims to provide insights into the direction the company is taking as it continues to enhance the generative texts for which it is renowned.

Chat GPT then and now — what changed

The changes and developments in ChatGPT between January 2023 and January 2024 are noteworthy, highlighting the swift advancements in AI and machine learning. To understand these advancements, I posed the question/prompt to Chat GPT:

“What are the Chat GPT changes from January 2023 to January 2024?

While I’ve applied some human editing for clarity, I’ve maintained the original categories and included examples for better understanding. The major advancements cited encompass:

  1. Model upgrades: OpenAI frequently updates its models, leading to improvements in understanding, response accuracy, and the ability to maintain context in conversations. For example, each new version of ChatGPT tends to be more sophisticated than the last, with better recognition of nuanced queries and more accurate, context-aware responses.

  2. Increased contextual understanding: Over time, ChatGPT has likely become better at understanding and retaining context within a conversation. This means that in a series of questions, ChatGPT can now refer back to earlier parts of the conversation more accurately, making interactions more fluid and relevant.

  3. Enhanced language capabilities: Improvements often include better handling of multiple languages, more nuanced understanding of language subtleties, and reduced biases in responses. For instance, ChatGPT might show an improved ability to understand idiomatic expressions or regional dialects and offer responses that are more culturally aware.

  4. Integration of multimodal capabilities: There may have been steps towards integrating other forms of input and output, like processing images or generating more complex media types. This could mean that ChatGPT can now interpret visual information or produce outputs that combine text with other media, such as creating descriptive narratives from images.

  5. Customization and personalization: Advanced versions of ChatGPT may offer more options for customization, allowing users to adjust the AI’s style, tone, or areas of expertise according to their needs. For example, a user could set the AI to respond in a formal or informal tone, or specialize in topics like technology or literature.

  6. Improved handling of complex queries: Later versions are generally more adept at understanding and responding to complex or abstract queries. This means ChatGPT can provide more detailed and contextually accurate information in areas like technical explanations or philosophical discussions.

  7. Ethical and safety enhancements: There’s a continuous focus on addressing ethical concerns and enhancing the safety features of the AI. This includes reducing risks related to misinformation, misuse, and potential harm, such as by filtering harmful content or flagging questionable requests. This is a definite plus from my perspective.

  8. User interface improvements: The interface for interacting with ChatGPT might have seen upgrades, aimed at making it more intuitive, user-friendly, and accessible for a diverse range of users, enhancing the overall user experience. Again, a plus for my reviews of the platform.

  9. Broader application and integration: ChatGPT’s application scope may have expanded, with integration into various systems and platforms. This broadens its use-cases, allowing it to function in diverse fields like customer service, content creation, and educational tools.

  10. Community and developer support: There’s increased support for developers and the broader community, which includes improved documentation, more robust development tools, and enhanced community engagement. This fosters a more innovative and collaborative environment for using and advancing the technology.

These advancements mirror the overarching trends in AI development, characterized by a relentless pursuit of more sophisticated, versatile, and user-friendly systems.

This progression explains the noticeable difference in the responses generated by ChatGPT in January 2023 compared to January 2024 — just one year later.

Additionally, and alluded to in its image processing abilities, mentioned in point 4, it’s important to mention the capability of DALL-E, another AI innovation by OpenAI, which can generate visuals. Although this aspect wasn’t the focus of this post, its significance and impact on the broader AI landscape are worth acknowledging. I have used this feature to add some visual vibrancy to the stories below.

Chat GPT’s horror story: same prompt — a year apart

I'm an early riser, waking no later than 6 AM every day. One of my morning rituals, come rain or shine, winter storm or sunny day, is taking our 135-pound German Shepherd for his 40-minute walk. Although sometimes my partner does it alone, we usually walk together, especially in the winter months, when our route through an old quarry, which we jokingly call ‘the Sunken Place’, takes on an eerie feel.

The early morning fog often adds a frightening ambiance. I’ve always half-joked with my partner about how this quarry, with its chilling atmosphere, would be the perfect setting for a horror movie about zombie dogs, reminiscent of a terrifying film I saw in my childhood, perhaps “Night of the Living Dead.” I can’t recall the name but the horror remained.

I know I would be petrified at such a prospect especially being so removed from the roads.

Usually, we’re the only ones at the quarry before 7 AM, as we need to return home in time to send the kids off to school and prepare for work. In this desolate setting, it’s easy to imagine that in some alternate, horror-filled universe, the only other souls around would be ghouls. So, intrigued by this spooky idea, I asked ChatGPT 3 to create a horror story for me.

Chat GPT Prompt: “Write a terrifying horror story set in a quarry converted into a dog park, featuring zombie dogs.”

Here is the unedited story generated by ChatGPT 3, complete with the titles it created. Note that the text remains as originally produced by the AI, and I have included the Word Editor scores at the end of each story for reference. I also asked DALL-E, in 2024 to generate images per stories. It will be interesting to visit the images in 2025.

2023 — “A Canine Nightmare” — (version 3.0)

created with DALL-E

As the sun sets on the abandoned quarry, a group of friends excitedly gather to explore the newly converted dog park. But as they let their playful pups off the leash, they soon realize that something is horribly wrong. The once friendly dogs have been transformed into zombies, relentlessly hunting down their unsuspecting owners. Trapped in the quarry with no way out, the friends must fight for survival against the relentless horde of zombie dogs.

The friends, consisting of a group of young adults, had heard about the new dog park that had been created at the old quarry and decided to check it out. They arrived at the park in the early evening, excited to let their dogs run around and play. But as they entered the park, they noticed something strange. The other dogs seemed different, they were acting aggressively and not responding to their owners’ commands.

As they walked deeper into the park, they realized that these dogs were not normal. They were covered in blood, their eyes were glassy and their movements were unnatural. The friends quickly realized that these dogs were zombies, and they were in grave danger. They tried to call for help but their phones had no signal. The friends quickly realized that they needed to stick together if they wanted to survive. They decided to make their way to the park’s entrance, but the zombie dogs were everywhere and were blocking their way. The friends had to fight their way through the zombies, using whatever weapons they could find. One of the friends, a construction worker, had a hammer and used it to bash the zombies’ heads in. Another friend had a baseball bat and used it to hit the zombies.

As they fought their way through the zombies, the friends noticed that they were not the only ones in the park. A group of people were also trying to make their way to the entrance. They joined forces and together they managed to make it to the park’s entrance. But as they reached the entrance, they found that it was locked and there was no way out. The friends and the other group of people decided to make a run for it, and headed to the quarry’s office. Once inside, they locked the doors and tried to come up with a plan. They searched the office and found a map of the quarry and a radio. They used the map to plan a route to the quarry’s exit, and the radio to call for help.

The friends and the other group of people set off into the quarry, fighting their way through the zombies as they went. They were met with many obstacles along the way, including deep water, steep cliffs, and narrow tunnels. But they were determined to make it out alive. As they made their way through the quarry, they came across a group of workers who were also trying to escape. They joined forces and together they managed to make it to the quarry’s exit. But as they reached the exit, they found that it was blocked by a landslide. The friends and the other group of people knew that they were running out of time.

They decided to make a final stand against the zombies, using whatever weapons they had left. They fought bravely, but they were outnumbered and outmatched. Just as all hope seemed lost, they heard the sound of sirens in the distance. The rescue teams had finally arrived. The friends and the other group of people were rescued and taken to the hospital. They were treated for their injuries and were in shock from what they had been through. They were told that the zombie had been caused by a virus that had been released into the air by a group of scientists who were experimenting with a new virus.

The friends and the other group of people were lucky to have survived, but they knew that they would never be the same again. They had been through a terrifying experience and had lost many of their friends and loved ones. But they were alive and that was all that mattered. The park was closed and the virus was contained. The friends were given counseling to help them cope with their traumatic experience. The zombie were put down and the park was turned into a memorial for the victims of the zombie outbreak.

They often think back on that night, and the horror they experienced at the quarry dog park. They will never forget the terrifying sights and sounds of the zombie dogs, and the fear and adrenaline that coursed through their bodies as they fought for their lives. Despite the trauma they went through, the friends have grown stronger and more resilient as a result of their experience. They are grateful to be alive and they know that they will never take life for granted again. They also made a promise to themselves to never let this happen again, they will make sure that the authorities will keep a close eye on any kind of experiments that could put people’s lives at risk. The park remains closed and abandoned, a haunting reminder of the terror that once took place there. But for the friends and the other survivors, it will always be a place of memories, both good and bad. They will never forget that night at the quarry, but they have chosen to move on and make the most of the life they have left.

My word editor gave the above 2023 generated text an 82% — A minus.

Story 2: “Quarry of the Damned” — (version 4.0)

In the quaint town of Eldridge, a once abandoned quarry had found new life as a popular dog park. The Eldridge Quarry Dog Park, surrounded by steep cliffs and dense woods, was the perfect escape for pet owners. But beneath its peaceful facade, a dark secret lurked.

Sarah, a local teacher and dog enthusiast, visited the park one foggy Saturday morning with her loyal Beagle, Max. The park was unusually deserted, the mist hanging low over the ground, giving it an eerie, otherworldly feel. Max, usually playful and energetic, seemed anxious, his ears flat against his head.

As they walked along the gravel paths, Sarah felt a growing unease. The silence was oppressive, broken only by Max’s low whimpers. Suddenly, Max stopped, his nose twitching, as a low growl echoed through the mist. Sarah peered into the fog, her heart pounding.

From the mist emerged a pack of dogs, their eyes glowing unnaturally. Their fur was matted, skin torn in places, revealing gnarled muscles and bone. They were the dogs of the quarry, but twisted into grotesque, zombie-like creatures.

Sarah gasped in horror, grabbing Max and running towards the park’s exit. The zombie dogs pursued, their growls turning into otherworldly howls. As she ran, Sarah recalled the local legends of the quarry — tales of a mysterious illness that struck animals and people in the 1800s, believed to be buried deep within the earth.

Reaching the park’s gate, she fumbled with the latch, her fingers trembling. The zombie dogs were closing in, their snarls filling the air. With a desperate push, she swung the gate open, rushing out with Max clutched tightly in her arms.

Once outside, the zombie dogs halted at the boundary of the park, as if an unseen force prevented them from leaving. Panting heavily, Sarah looked back at the quarry, now quiet once more. The fog seemed to thicken, obscuring the zombie dogs from view.

Shaken, Sarah vowed never to return to the quarry, but the incident at Eldridge Quarry Dog Park became a chilling legend. Whispers spread of the cursed land, where pets transformed into the undead, trapped forever within the confines of the park.

And sometimes, on foggy mornings, the distant howls of the zombie dogs can still be heard, a haunting reminder of the quarry’s dark legacy.

My word editor gave the above a 96% — A plus.

Asking Chat GPT how it did

I used a rubric, not unlike what I would use to evaluate my students, and here is what I gleaned from Chat GPT’s evolution.

Details and description

“A Canine Nightmare 2023” also provides a detailed setting but focuses more on the action and the characters’ reactions to the evolving situation. The description of the zombie dogs and their impact on the park atmosphere is less detailed compared to “Quarry of the Damned”.

“Quarry of the Damned — 2024” offers rich environmental details, painting a vivid picture of the eerie dog park with its fog and deserted paths. The description of the zombie dogs is detailed, adding to the horror atmosphere.

Grammar and language use

Both stories demonstrate a good command of grammar. The older story also employs appropriate grammar but focuses more on narrative progression and action, which affects the language style, making it more direct and less atmospheric. “Quarry of the Damned” uses language that enhances the spooky and suspenseful mood. I definitely found it a bit bone chilling, with language evocative of a slow burn effect.

Flow and pacing

The older story, moves at a faster pace, emphasizing rapid developments and action, leading to a more immediate sense of urgency and danger. The newer story has a slower pace, gradually building tension and focusing on the protagonist’s emotional and sensory experience.

Believability and consistency

The older story, while engaging, occasionally shifts focus between the horror elements and the action-adventure aspects, which might affect its overall believability as a horror story. The newer story maintains a consistent tone of suspense and horror, making it a more believable horror story in a traditional sense. Undoubtedly the generics of the first story output had improved drastically, adding to its believability.

Character development

The older story introduces a group of characters, offering more interaction and diversity in responses, though the development of individual characters is still limited due to the short format of the story. In the newer story, character development is limited, with the focus primarily on the protagonist’s immediate reactions. This, to me, was a marked improvement.

Overall engagement

The older story engages the reader with its fast-paced action and the survival aspect of the narrative. I think this took away from the horror aspect for me. The newer story excels in creating an immersive horror atmosphere, drawing readers into its eerie setting.

My takeaways — still some ways to go

Both stories demonstrate different strengths: “2024 — Quarry of the Damned” excels in atmospheric horror, while “2023 — A Canine Nightmare” focuses on action and group dynamics. The evolution in ChatGPT can be observed in the variation of storytelling techniques and focus elements used in these narratives. It is a marked improvement in the tool, and in some ways, that is its own terror for the human abilities. However, there is still a ways to go. The tool sill needs some human oversight to tell better stories. To me, the best of ChatGPT is seen in the symbiosis with humans.

Broader takeaways

ChatGPT is often unfairly presented as a new technological species that will take jobs from humans. However, this perspective overlooks the potential for a human-machine symbiosis that could revolutionize how we work and create. A human to machine symbiosis offers the opportunity to enhance human capabilities, automate mundane tasks, and free up time for creative and strategic thinking. By working in tandem with AI, humans can focus on tasks that require emotional intelligence, critical thinking, and personal judgment — skills that machines have yet to master.

This collaboration not only preserves the unique strengths of human workers but also leverages the efficiency and processing power of AI, leading to a more dynamic, innovative, and productive workforce. Rather than replacing humans, ChatGPT can be seen as a tool that augments human intelligence and creativity, paving the way for new job opportunities and growth in various industries.

About me: Hello, my name is Kem-Laurin, and I am one half of the co-founding team of Human Tech Futures. At Human Tech Futures, we’re passionate about helping our clients navigate the future with confidence! Innovation and transformation are at the core of what we do, and we believe in taking a human-focused approach every step of the way.

We understand that the future can be uncertain and challenging, which is why we offer a range of engagement packages tailored to meet the unique needs of both individuals and organizations. Whether you’re an individual looking to embrace change, a business seeking to stay ahead of the curve, or an organization eager to shape a better future, we’ve got you covered.

Connect with us at https://www.humantechfutures.ca/contact


Was this article helpful?

ESC

Eddy AI, facilitating knowledge discovery through conversational intelligence