Cookies, Brownies and Chatgpt

A reflection on a primary schooler’s experience using AI chatbots

This article is adapted from the publication InnoEthics by Allegra Cuomo, check out the original featured on her newsletter here.

With the UK government’s national consultation on growing up in the online world well underway, it is more essential than ever to ensure all voices and perspectives are being considered in these conversations.

Therefore, as part of their commitment to supporting youth with tackling digital wellbeing through lived experience, FlippGen hosted the Youth Town Hall, bringing young voices directly into the national conversation on digital life.

As part of my ongoing role as a member of the FlippGen Youth Advisory Board, it was brilliant to work with the team organising and hosting the Youth Town Hall last month. FlippGen’s fully youth-lead event was run in partnership with the 5Rights Foundation and Department for Science, Innovation, & Technology, including an appearance from the Minister for AI and Online Safety, and held at the Lush Studio in London.

The room was filled with youth, some only as old as nine, talking about their experiences of growing up in the digital world. Not only were these discussions filled with the thoughts of those from GenZ and GenA, but the sessions were facilitated and led by young people too.

This set up allowed for attendees to speak honestly about what it is really like to grow up online. Talking about the things they like and don’t like, and the things that they feel their parents, teachers, and adults generally, often miss.

The output of this fantastic event is the collation of all participants’ considerations, thoughts, and solutions, brought together into a formal submission to the Government.

There were so many insightful and thoughtful reflections from those sat on these tables, and if I only I could write about each one! However, there was one comment from a 10 year old that I just couldn’t stop thinking about after the event…

“I asked ChatGPT for a recipe for cookies, so why did it suggest one for brownies at the end of telling me what I asked for?”

Now, I would be lying if I said this comment didn’t make at least a few of the other children, teens, and young adults sat in our discussion laugh at least a little.

However, this comment was following a discussion where we posed the question of ‘What does our ideal digital future look like?’, looking at what children and young people would change about social media platforms or AI chatbots if they could design them themselves.

One of the primary school children raised her hand, a bit of an unimpressed look on her face as she said the full quote: “I asked ChatGPT for a recipe for cookies, so why did it suggest one for brownies at the end of telling me what I asked for? If I wanted a recipe for brownies, I would have asked for one in the first place.”

In order to find a resolution to this child’s complaint, let’s have a look at why AI chatbots do this and use some of the frameworks from the FlippGen Digital Intelligenz tool kit!

This phenomenon of ChatGPT and other AI chatbots asking ‘Want me to…’ or constantly follow up answers with further questions or prompt is a part of the people-pleasing, sycophantic, often over-eager behaviour that can drive users insane or have harmful consequences.

One search on the OpenAI Developer Community forum page brings up articles discussing how annoying users find ChatGPT’s instant prompting at the end of messages. The user that raised this particular complaint said ChatGPT’s prompts for more conversation “feel intrusive and not helpful”, and responses to this post include links to related posts about how to limit this feature, and also how to make ChatGPT more concise and less verbose.

This excessive verbosity and wordy-ness is a common complaint among chatbot usage (though some would argue it is the quality of the model or the result of not prompting effectively that leads to this worse functioning).

In fact, when asked how often students use LLMs or AI chatbots for their university courses, one of my favourite responses was that “it uses a lot of words without saying a great deal of much”. Moreover, a needless use of adjectives accompanying ever noun in a sentence can be an initial way of trying to deduce if text is AI generated.

Anyway, let’s go back to how AI chatbots always want the conversation to continue. A simple Google search will provide articles with a range of solutions to the problem of follow up questions.

While we discuss solutions and ways of limiting this behaviour, it is important to consider the design features and decisions that make this technology the way it is. This comes as a part of digital literacy, with FlippGen’s toolkit giving young people a simple and helpful way to learn more and feel more confident about using digital tools.

The tendency for AI chatbots to be either overly agreeable or uncharacteristically argumentative, and to want to continue conversing with the user for as long as possible, are because engagement equals revenue.

Whether this be through subscription models, premium tiers, or advertising opportunities, increasing user interaction has a monetary purpose, too. Monetising on user engagement is nothing new, with social media as the most pertinent example of recent times.

Of course, this engagement is also an effort by the creators to ‘shelf block’ the user’s time. Any additional moments spent with that tool are time that is not being spent on other ones. Maybe we will coin that one as experience hoarding, or something…

Going back to user experience, sometimes the follow-up prompts are not only relevant, but might provide a useful next step or a different perspective the user may not have considered. However, often they are creating an ongoing exchange from a straightforward query that was supposed to have a quick answer.

In the case of extended and unwanted conversation with an AI chatbot, there are adjustments available in ChatGPT’s settings to alter and limit the feature of follow-up suggestions. It is also a case of personal preference, with some people feeling supported through the extra suggestions, while others find them deeply annoying.

We can let the 10-year-old who wanted a recipe for cookies (and JUST for cookies, not for anything else) know that there is an explanation for why it seems chatbots might only partially be listening to your question, and also ways to fix this feature.

But simultaneously, there is a concern that being provided the information without any sort of friction removes the useful mental exercise achieved by reading through different sources of information, looking through websites for the recipe you like best or trust most, rather than the first one that pops up. FlippGen’s toolkit even notes that studies show ChatGPT might in fact be reducing your brain activity, with its usage leading to weaker neural connectivity, and difficulties with memory and independent thinking.

And from a more sinister perspective, another concern is that the chatbot’s nature of conversation and engagement perpetuates the harmful thought process that leads the brain to mistake chatbots as human-adjacent agents. The continuous engagement reinforces the anthropomorphism of AI chatbots, with our FlippGen toolkit making sure to remind users that while chatting with AI chatbots can feel real, they are just programs without actual feelings.

Both considerations demonstrate the importance of digital literacy when it comes to navigating the online world, whether it comes to social media, AI chatbots, or information on the internet. This focus on digital literacy is a central part of FlippGen’s work, teaching children and young people how to use technology effectively, safely, and wisely.

So, to our 10-year-old sceptic, I applaud her for calling out when a technology that claims to be designed for the benefit of the user demonstrates features that may not necessarily live up to this promise. As young people, we have the power to teach and provide opportunities to others about the importance of digital intelligence, helping shape an online world that puts youth at the centre.