Are LLMs Just UIs?

The LLM Experience

Large Language Models like Chat-GPT have been promoted as all-knowing IO machines. And to the untrained or inexperienced, this is valid. But if you have any technical experience or you've had the chance to build with these tools yourself, your experience is likely completely the opposite.

Most of us have experienced the downsides of LLMs and even agentic workflows: hallucinations, misinformation, or just being verbose and unhelpful. This is due to a number of factors: training data limitations, poor context management, and overly large context windows. As a result, we get frustrated, blame the AI, and thus continue the love/hate relationship we have with this technology.

I believe part of the issue is in our approach or perspective on it. We may be expecting too much out of what in fact may just be a user interface, albeit a very powerful and sometimes unpredictable one.

My Experience

When Celeste and I created Prompt Shorty, a POC storytelling application, I was surprised at how easy it was to implement and use LLM software. You give it context, it gives you responses. But likewise, without context, you're subject to whatever training data the model has available. "Write me a story...", results may vary. In our case, we were using OpenAI's Completions API and passing user input into the "Sytem Prompt" to get more curated responses. We also created a RAG toolchain to add reading level data which further improved the responses. This project made it clear to us that models didn't just "work" out of the box. They need fine-tuning.

Recently, while working with the Sweetwater team, I decided to prototype another chatbot, “Taggy,” to allow contextual search of real product data from Algolia. I trained the bot to receive product questions, interpret them, and repsond in JSON which I could feed into the Algolia request. Then I would use the Algolia response to feed the bot product data. Finally, the bot would use this data in its response, and update the collecion UI. This worked too well.

Click the "Sweetwater" card on my portfolio to see a video walkthrouh.

But what was I actually doing? I wasn't replacing the existing UI with a bot... I wasn't even replacing the search functionality of the page. I was allowing the customer to interact with the same data, business logic, and visual UI in a new way, giving them options. No different than optimizing your site for screen readers. You're not replacing the experience, you're enhancing it.

I'm of course not the only one approaching LLMs this way. Google is working on their own in-browser agentic experience that allows devlopers to expose page data to it through custom data attributes. So in the future, we may end up implementing a shared convention for agentic web pages. We'll see.

TLDR

LLMs take in messages and output responses based on context and training data. That's a good thing. They can be a quick win for your site visitors/customers. But, you have to train and structure them to align with YOUR messaging and business goals.

On the web, there are tons of things working under the hood to deliver you a website, but a website is not helpful in and of itself. You need to add helpful information, structure, context, and intent. Just like an LLM.

Goodbye!