What if my top frequently used Apps are transformed into conversational user interface (Part 3/5)

Gaming world has its own innovative and visually artistic experience that couldn’t be found elsewhere. When a dialogue pops up, it brings all settings and crews ready in their standby positions in order. Similarly the elements shown in the user interface also carry a certain message contributed back to layers of relationships between people and their hidden intentions.

From the article of Luke Wroblewski personal blog, one of my favorites and eminent digital product leaders in the world, has beautifully and well addressed the point that chat user interface interaction empowered by artifitial intelligence nowadays might be able to evolve into more user-friendly chat UI utility visual elements, such as collapes & expand, integrated actions and object experiences powered by large language models, which are some of the fundamental patterns we use all the time.

Collapse & Expand (Source: Expanding Conversational User Interfaces from LUKEW)

Integrated Actions (Source: Expanding Conversational User Interfaces from LUKEW)

I believe all readers can learn tons of insights and inspirations from Luke’s essays, where I will come back to explore more design possibilities in Part 4 extended from the concept above.

This wave of conversation-oriented interaction does aggressively open a variety of applications, once self-owned database has been poured into the training set and generate answers to the questions people ask. Like gasoline vehicles if they all have been plugged in online to collect all the mileage on road, those cars might out-perform competitors in terms of the amount of data and design iterations.

The Apps installed in my Pixel phone have been carried out with a long history of my personal usage logs and data, which naturally provide a valuable off the shelf input of the predecessor of my own personal AI assistant. Apart from the Apps that are used for work or particular purpose (such as those for entertainment, education, transaction or GPS navigation while driving), other Apps that I use a lot include LINE, Facebook, Linkedin, Spotify and Chrome, etc, which I usually spend dozens of minutes to hours per day in average.

Those Apps serve and act distinctively in terms of content and goals provided, while people probably have completely different expectations from those Apps (or we can call it information portals) even if we all download the same one. For example, Instagram might be an outfit inspiration to one person, but more like a restaurant recommendation guide to another.

To illustrate this differences reflected back to my design explorations, I make a simple and small assumption and distill the scenarios from my own experience: we all might have a goal to accomplish, whether the goal is big or small, deliberately or unknowingly, and those giant Apps could take advantage of this conversational UI pattern from the LLM (Large Language Model) to provide unique user experience so we could jump right in and start interacting.

empower messenger app with llms

Chatroom, Voice & Video Calls, Payment and OpenChat (Source: line.me/zh-hant/)

Take LINE for example. LINE App becomes one of the must-have messaging Apps in Taiwan, where almost over 90% of the population has LINE installed in their phones. Like WeChat in China, or WhatsApp in India or Brazil, people use it to send text messages or share videos, and make phone call or conduct payment in this single one platform everyday. From the circle of friends and family, to social groups of community, LINE’s services coverage expand more and more ubiquitous that people cannot live without.

Under the Chats tab of global navigation, there are couple of categories user can choose from, like Friends, Groups, Official Accounts and OpenChats. In each chatroom, user can reply particular old message, and also the system will automatically store photos & videos, links and files that are sent in the messages thread between you and your friends. In other words, LINE knows every information you provide in the chatrooms.

However, rather than figuring out what answer each LLM-based model could deliver, we should figure out what kind of questions we could ask first. Let’s take a look of what these conversational AI models are capable of before we start chatting or prompting (Take desktop version as example).

Dataset training amount and examples of questions to ask on each AI language model platform

Each platform has its own unique personality and relationship with people who use it. While we might be hesitate to type the first question to ask, or start thinking what we know we want to ask, on the landing page it shows you some examples you can start from, as well as connect what you might feel beneficial from the pairs of question and answer, to what those powerful platforms are capable of.

People usually search or browse, two essential behaviors people do for a whole bunch of contents and materials in App or online. As you can see, when you enter those LLMs platforms, apart from the basic menu and tips, it is nearly all empty on the screen, while the input box is waiting for your first interaction. But people might treat messaging App in different way, after all it is text messages of you and your friends and family that drag you over here to close the distance essentially. As being a messenger App that people use everyday, what would people might expect from LINE if it could be as intelligent and competitive as Bard or ChatGPT?

Take one of the real cases of mine to be a target of this new exploratory design. When I plan to kick off the house renovation, couple of homework need to be done and prepared in advance. Merely in the LINE App, a few chatroom groups have been created just for this matter, where each group represents exclusive relationships and goals between me and other people.

Messages and Official Accounts involved for the house rennovation planning

Chatrooms are like a dedicated channel that grabs people into one place and let them talk to each other instantly at anytime. People can ask specific questions, share files or links, or even can have a video call to catch up. In other words, whom you talk to or which topic you are involved decide what chatrooms will be created and associated with multi-media files in that single group. In my discussion and preparation of house make over plan, there are roughly 3 chatrooms with friends, 9 files, 30 photos and over 80 messages and couple of official accounts I join since this initiative has kicked off.

We definitely can keep searching and browsing the contents or certain piece of information arbitrarily or intentionally. However by making use of intelligent models released out there, what kind of new way of interaction could be ahead of these messages layers and gain a new life for people to get the information quickly and easily would be an interesting and a big question. Based on the examples of questions suggested in those LLMs brought up earlier, the questions that people might want to be asking in LINE for this house renovation plan could be:

  • Can you give me the summary and next to-do lists based on all my previous conversations with design contractor I contact with?

  • Can you retrieve any cost estimates or quotations provided by the contractors during the discussions?

  • Can you suggest relevant resources based on the content of my messages?

  • Draw me a new floor plan based on my favorite styles in the latest discussion.

  • Can you identify any potential conflicts or contradictions in the discussions that need clarification or resolution?

  • Can you rename the photos that I provide for reference and highlight the point I like, and send it to the contractor?

  • Can you categorize or organize the chat content based on different aspects of the renovation, such as kitchen, bathroom, flooring, etc.?

  • Are there any specific design recommendations or suggestions provided by the contractors that I may have missed or overlooked?

  • Can you provide me with a consolidated list of the proposed materials, products, or brands discussed during the conversations?

  • Are there any relevant permits, licenses, or legal requirements that were mentioned in the conversations, and can they be easily accessed?

The list could be growing on and on and continue to prompt more questions or instructions based on the topic of interest. Even though jumping out of these groups of chatrooms to allow the system to summarize, extract, create follow-on questions, make exploratory questions, or generate answers are also compelling ways to explore what the App could offer with its available corpus of data and information. Chances are that one day, we all have our own intelligent agents where they talk to each other, and stop by the authority that requires human beings confirmed or involved. Messages are not meant to induce more messages; they just get the job done.

If you look broader, you can always ask more additional questions that are not limited to particular chatrooms, people or information. You can compare the pros and cons of each contractors in regard to their proposals and cost, to fulfill your needs and schedule. Back to the design, let’s see how this new conceptual design might look like:

New conceptual design flow (Image source: https://www.muji.net/ie/kinoie/)

In the new design, the Chats tab has been re-shaped by topics, which represents more solid subjectivity of what context has been involved in one place. For example, user is able to drag several different chatrooms together, to make one subject more complete in terms of data corpus traning if it will. In each chatroom, user can always trigger chatbot AI to ask questions based on the messages in the chatroom. The beauty of this conceptual design relies on what you can get by throwing the questions or asking for further demands. There are always rooms for additional functions like, asking follow-up questions, tweaking the output, sharing the answer to the chatroom, or even giving me surprise!

LINE acts like its own ecosystem, where each chatroom live next to each other and is responsible for the messages or other media in it. LINE App got its own advantage of generating AI talking with you, so does other Apps. Therefore the next question might will be, what if thoses Apps come into ONE AI in my Pixel? Will it be more powerful? WIll the conversations be more thoughtful or full-packed? Will the boundary be broken? Those are what we are going to cover and explore deeper next in Part 4.

In case you miss it, here is the full index of my essays below:

Stay tuned.