Introducing ambient agents

Introducing ambient agents

https://blog.langchain.dev/introducing-ambient-agents/

https://github.com/fanqingsong/executive-ai-assistant

 

What is an ambient agent?

When using ChatGPT (or any other chatbot), they rely on you to initiate the conversation. The agent is kicked off by the human sending a message.

This is great for some use cases, but also severely limiting for others. It requires the user to go into the chat interface and send a message every time they want the agent to do work. There is a lot of overhead in having the agent start work.

An additional limitation is you can only have one conversation at a time. This makes it hard for us humans to scale ourselves - an agent can only be doing one thing for us at a time.

If we think about a UX paradigm that allows us to overcome these limitations, it should exhibit two key characteristics:

• It should not (solely) be triggered by human messages
• It should allow for multiple agents running simultaneously

The characteristics define what we call ambient agents.

💡

Ambient agents listen to an event stream and act on it accordingly, potentially acting on multiple events at a time

Notably, however, we do not think that ambient agents are necessarily completely autonomous. In fact, we think a key part of bringing ambient agents to the public will be thoughtful consideration as to when and how these agents interact with humans.

 

Human-in-the-loop

We use human-in-the-loop to refer to when and how these agents interact with humans. We'll talk about how later, but for now let's discuss the when.

We typically see three common human-in-the-loop patterns for ambient agents: notify, question, and review.

Notify: let the user know some event is important, but not take any actions. This is useful in flagging events that user should see, but where the agent is not empowered to act them. In the context of an email assistant, this could be the agent flagging a Docusign in my inbox - it's not able to sign that Docusign, but I should know it exists.

Question: ask the user a question to help unblock the agent. The agent may be trying to take some actions, but unclear on how best to do so because it's lacking some relevant information. Rather than hallucinate or guess, you just have the agent ask the human what to do. In the context of an email assistant, this could be an agent asking me whether I want to go to a conference. Unless something was in the prompt to instruct the agent on my conference preferences, there's no way it should know that. A human EA would ask me, and so should an agentic one.

Review: review an action the agent wants to take. Some actions are "dangerous" enough that it may be worth hard coding a review for any action the agent wants to take. The human can either approve the action, edit it directly, or give direct feedback to the agent on how to change it. In the context of an email assistant, this could be an outbound email. It might write a draft, but I would have to approve it, edit the message content directly, or tell the agent to fix it in a certain way.

The importance of human-in-the-loop

We think this human-in-the-loop component brings three key benefits to ambient agents:

1. It lowers the stakes, making it easier to ship agents to production
2. It mimics how humans communicate, building user trust and adoption
3. It empowers long term memory and learning

Human-in-the-loop lowers the stakes. If an agent is running fully autonomously in the background, then it really can't make a mistake. You would have to trust the agent immensely before letting it take certain actions (like updating a database, sending an email to an important client, etc). With human-in-the-loop, you can easily gate those actions and require explicit human approval. This way you can be sure that no errant email will get sent.

Human-in-the-loop mimics how humans communicate. A big part of working with someone else is communicating with them. Asking them questions when you're unsure, running ideas by them. If we have "co-workers" that are agents, having them communicate in similar patterns builds user trust and therefor adoption. Consider something like Devin. One of the primary interfaces they chose for users to interact with Devin is in Slack. That's where we interact with human developers, why shouldn't we interact with AI developers in that way? Communication is important.

Human-in-the-loop empowers long term memory and learning. We strongly believe that a key part of AI agents is their ability to learn over time and better align themselves with their human users. In order for this alignment to happen, they need some form of user feedback. This human-in-the-loop component provides this feedback.