How Do AI Agents Earn the Trust to Act on Your Behalf? [Ft. Devi Parikh, Yutori]
If you have agents that are gonna go out and they're actually gonna have access to go buy things, they need to earn the trust. That is part of the reason why scouts is where we started, where we're asking for zero access to your context. Right? And so we focus just on delivering utility and value to you so that we can earn your trust and sort of start going up the staircase. You might start with something that's lower stakes for you. You having control over how quickly you're letting hook control is important. So let's fast forward years 2030. Do you think that most people will be comfortable with AI just listening to their whole day? My guess would be Alright. We're here with Devi Parikh, co founder and co CEO of Yutori. Welcome. Thank you. Thanks for having me. Okay. We got a lot to talk about. I think, turning AI agents from being passive or AI in general from being passive reactive things of us giving it information to proactive chief of staffs, you wanna call it, is kind of what we wanna focus some time around. I think the first thing I want you to clarify for for people at home is what does a chief of staff mean? What do you when you say that, what do you really want these these AI agents to be able to go do on behalf of folks? Yeah. So I think I think most people probably have a sense for an EA or a personal assistant and those kinds of entities, either in your own life or people you may have seen around you. And I think the distinction between that and more of a chief of staff entity is partly related to some of this proactive nature of it, partly also related to sort of someone who you trust to just be your voice and to kind of, over time, as you work together, be able to make similar judgment calls as you would. So someone who you just sort of trust more and more where you expect that they will just notice the things that need to be done to just sort of make your life easier. And you can trust them to do that. You trust their judgment call on it. As opposed to sort of you having to take a thing that's going on in life, having to break it down into sort of bite sized tasks that can then be handed off to more or less anyone, that anyone who's competent might be able to get those tasks done. But not everyone, even if they are competent, unless they have your context, unless they get to know you, unless they know how you make decisions and what your judgment calls tend to be like, those sort of entities wouldn't be able to transition into a chief of staff like Figure four Youth. I think, yeah, those dimensions are the ones that we are trying to push towards. We're not saying we're there yet, that's what aspirationally we are working towards. Great. And that makes sense. And you it's obviously, the the messaging is resonating. You got a $15,000,000 series or seed round Seed round. Recently. That's great. Where do you expect like, obviously you get this future state, however far if it's a month or a year of chief of staff being able to do whatever, where are you today, and where do you want to be in, let's say, a month with this product? So where we are today in terms of what is out there that has already been released that people are using, is a product called Scouts. And so Scouts start to have some of these flavors in them already. So Scouts can monitor the web for anything that you care about. If there's a product you've been tracking, you're waiting for a price drop, you're waiting for it to be back in stock, you can set up a Scout and be like, let me know whenever this happens. If there's any news that you're following, if you want to be the first one to know whenever there's an AI model update or whenever a startup announces a new fundraise or anything of that sort, you can set up a Scout for it. If you're a recruiter or sort of a salesperson, you can set up Scouts for lead generation that anytime a new YC company raises a certain amount of money and so they may be looking to grow, that's sort of a good person that you want to reach out to and so you want a notification for that. So scouts will monitor the web 20 fourseven for more or less anything that you care about. And so it has this proactive nature to it. Like you've set the scout up for something, but then whenever it happens, it will proactively let you know. You don't need to keep asking it. But where it's fairly restricted is the fact that it's a read only task. It's only monitoring the web for you. It's not changing the state of the world. It's not buying anything for you. It's not making any reservations. It just lets you know when something happens, right? So in some ways, it's sort of Google alerts on steroids, if you will. But that future of it being able to go and say, all right, well, know I have to go to San Francisco, San Diego, whatever it is. Hey, let me go look for deals. And when I find the best one, let me just go buy it. Exactly. That's we're not far from that. Exactly. And so that's not far from it very literally. Like, something that you're shipping in the next day or two starts doing exactly that. Well, now it's not just notifying you of something. It can actually get tasks done on your behalf instead of just sort of sending you like a nice digest over email. It can create all sorts of artifacts for your slides and sheets and web pages with visualizations and dashboards and all of those things. Scouts right now can only monitor public information on the web. So anything that needs you logged in. So if you wanted to monitor LinkedIn in general, whatever is publicly available on LinkedIn, Scouts will be able to find. But if you wanted to monitor your own LinkedIn, whenever one of my connections talks about a certain topic or something like that, let me know. Scouts today can't do that. But something that we're shipping will let you log into whatever websites you choose to let Scouts go behind those auth walls. You'll be able to log into those and let them go there. And you'll be able to let them connect let you connect scouts to your email, your calendar, your Notion, your linear GitHub, all of that. So going from scouts that only monitor read only actions only on the public web to now being able to actually complete tasks, intersect with your user context, log in on your behalf whenever you permit them to do so, and actually complete tasks, make artifacts, feels like a meaningful step forward. And then something else that we have coming in a couple of weeks, more or less, starts feeling like a bit of a chief of staff 1.0, if you will. And so I feel like that journey of starting from zero access to user context to starting to become this proactive entity in a person's life in a very meaningful way is happening pretty quickly. So what you're talking about is the difference between whereas now it can it can use APIs to go into certain tools. But you're saying that we will be able to actually just authenticate it. Say, here's my login information. If I have Sales Navigator or something in LinkedIn, it could go use it. Correct. Correct. And it's actually, right now, what it already does is that, yes, when there are APIs, MCP servers, all of those, it will use all of those tools. But for the heavy tail of the web, which is not, doesn't have, like your local tennis court reservation website, most likely doesn't have an API or an MCP server that they've made available. And so scouts will, when necessary, will spin up a remote browser, will open up a web page, will use a browser, use model that we have trained in house to click around. Like for your tennis court reservation, it will click on the date, will click on the time to figure out is this available. And today, scouts will just let you know if it's available or not. But starting as soon as sort of tomorrow, you can ask it to make the reservation. So it will continue clicking to actually make that reservation. And so even today, it's not just APIs. It uses this browser use model, but it can only do it on public websites today. But in what I'm describing that's coming soon, you will be able to log into specific websites and say that you have my permission to go there. And then scouts will be able to use that logged in state to then click around and complete tasks on your behalf. And can it use tools like, you know, let's say, I don't know what LLMs are using behind the scenes, but, Claude can create nice HTML artifacts, but, know, it's hard to get images in there. This you know, Gemini is great with image. Do does it would it allow for, like, a deck to be built? If you said, hey. I'm looking for this, you know, new, I don't know, shirt or whatever it is, and it's going and finding them, not just giving you links, but it's now creating, like, a a mood board or a a PowerPoint slide or something that, like, a you would imagine the chief of staff would be able to do. Yeah. Yeah. It will. Yeah. They they will create, various artifacts, for you so that the information that it has found is sort of presented in a way that's most useful for whatever it is that you're trying to get done, with that information. That's really interesting to me because I've I feel like we've been talking a lot. Whenever I talk to other folks on on this podcast, it's it's always around like where AI is like right on the edge. Right? And it a lot of times comes back to, well, what do we need the Internet for at that point? Like, do we need a website for? Really, the website exists to serve as schema and data points for AI to pull and to verify. Right? And so how far are we from, you know, asking a question and it creates a beautiful website interface for you to answer your question or to pull the the things from disparate sites together to say, here are your options? It doesn't sound like we're that far. Yeah, yeah. Not that far at all. And sort of this idea that when you and I go to a web page, we see more or less the same thing minus maybe whatever personalized ads there are, right, where the information is presented more or less in the same way to everyone. But different people are coming to the website to get different things done. And the fact that information can be presented to you for whatever it is that you are trying to get done makes a ton of sense. And I feel like from a tech perspective, we are not that far from it, at all. Yeah. And these tools and these assistants only get better with more context. Right? So right now, even with the agents, way they're scouts, the way they're set up, you are still giving it context to say, want you to go do this thing. Right? But there's gotta be a a world not too far away where it is listening to conversations you're having. You are allowing it to listen throughout the day. Right? Even maybe it's just when you're at work, but you have something turned on all the time that it can just digest your data to suggest things that it could do for you. Yeah. And even right now with what I was describing with sort of connecting your email, connecting your Slack, connecting your Notion and sort of all of that to what we're currently calling Scouts going forward, we might have a different name for it, that already unlocks so much. It's not even listening to anything else. This is already where a lot of your digital context lives. And the level of useful personalized things that it can suggest that here are four things that I can already get started on on your behalf is just amazing. It's just amazing. And we have a chief of staff at Udore. Her name is Christie. And she loves this product that we have. She's like one of our power users, even internally on the team. And it's just amazing how much useful stuff there is in the context that already exists. Mean, what you were talking about, sort of adding it, just listening in all day at work and all of those things. So, yeah, you you can push that quite a bit. Do you see this as, like, longer term, do you see this being more on the consumer side, product like growth for, you know, individual users? Maybe it's not just on their personal life, but, like, individual user at an organization. Or do you see this as a organizational tool rolled out from, like, top down? We are currently viewing it as a prosumer product that we are building like, we are building your chief of staff. Do think that it makes sense like for your chief of staff to be effective, it makes sense for it to be both across your personal life and work. Like I was saying, that access to your context is very useful, especially when it starts being proactive. And so there, to get access to your work context, it might be a tool that's sort of made available to you through your employer. And so we might be selling to your employer and not directly to you. But it is intended to be like an individual's chief of staff. Okay. Let's talk a little bit about what I, read in your your, website and other places about the trust, ladder. Was it trust ladder? Yep. Yeah. The trust staircase. Yeah. Trust staircase. So trust staircase. This is where, if you have agents that are going go out and they're actually going to have access to go buy things, it, they need to earn the trust to where you're like, okay, they've done mock purchases. I would've, I would've been great with them purchasing that. Now you're good to go. Is, have you rolled that out? Have, have people tested that and how long does that typically take before they're at the top of the staircase here? And they're like, yes, go ahead. Yeah. Yeah. So that is part of the reason why Scouts is where we started, where we're asking for zero access to your context, right? And so we focus just on delivering utility and value to you so that we can earn your trust and sort of start going up the staircase. And we have had people pinging us saying that, oh, it was monitoring this thing. It found it. I can't believe it found it. But by the time I saw it, it was gone again. And so can you just get it to buy it for me? And so we've had people tell us that, like, I wish I could just complete some of these tasks on our behalf. And so that's where we started. That's where we sort of and we have what is called sort of inspect work, if you try the product out, where you can see behind the scenes what was happening, which also helps with people starting to trust this tech more over time. And from there is where we are now starting to take these next steps of completing tasks on your behalf and things of that nature. What I was describing about you being able to send scouts behind an auth wall, the way we've built it is that it gives you complete flexibility. It is not a browser extension where it just has access to all of your logged in states. It is going to be a sandbox thing where you choose which websites you want scouts to be able to log into or which websites you want to log into and then scouts have access to it exactly for this reason that you might start with something that's lower stakes for you, maybe your local tennis court reservation if it doesn't need a card on file, right? Like maybe that's where you start. And then over time, as you see that this is quite reliable, is where you might give it more and more access. I think you having control over how quickly you're letting go of control is important. Yeah, yeah. Everyone's staircase can be longer or shorter. Exactly. And sensitivity to different things, right? Like I might be like, well, sure, whatever pizza, I don't really care. But you might be very particular about the kind of pizza you And want, so it's hard. So yeah, personal preferences there. So speaking of pizza, I was in your background, you have a background in, was it VQA? Yep. And so I, if I understand this correctly, and I'm not technical, but if you understand this correctly, identifying having a picture and having AI or some kind of recognition to understand that it's a pizza is wildly different than understanding if it's a veggie pizza. Because there's a lot of additional reasoning that has to go into that to determine it has to look at all the different pieces of all those pixels to determine do these pixels make up pepperoni or versus something else. So the and and this is was this your time at Meta when you were doing this? Or was a little bit before Meta. I continued while I was at Meta. Yeah. And then you went to Meta, and you were doing video generate or, like, generate generative video? Yeah. Yeah. Over time, I was working on generative models or images, videos, other modalities. What was it like at Meta? It was fun. It was fun. I had How long? Yeah. I was there for about eight years. Eight years. Eight And, it was awesome. I had gone there. I was a faculty I was a professor at Virginia Tech and then at Georgia Tech for a little while before I went to META. So I think just sort of some of the contrast was fun to see. Like at universities, everything happens kind of at a semester's cadence, that anything that you take on is like, oh, this semester, we'll make progress on this. Or the next semester, we'll do something else. And when we first got to meta, everything was at a weekly cadence. Like, why aren't we just getting it done this week kind So of that was refreshing. I think it was also refreshing to see so Mark Zuckerberg used to do these weekly Q and As to the whole company. Like, everyone was welcome to come. I think it was Friday afternoons at the time or time. I think they changed when it was. And so people would just go up to him and ask him questions. He doesn't know what everyone's going to ask. And he used to just directly, at least to me, what it felt like was just direct, transparent, to the point, contentful answers versus sort of a lot of administrators in academia, sometimes in sort of talk for forty five minutes and you're like, what did they actually tell me? And you realize that there wasn't much there. And so that was refreshing. I like being on campus. I like being around students. And so I missed those spaces of academia and being at at META. But, yeah, it was a lot of fun. I used to teach as well. I taught at, University of Central Florida. And I I did it at night for eight or nine years, when I was, you know, worked during the day, and I would go teach a couple classes at night throughout the week. I was single, so it was like whatever I could go, you know, teach at night, I wouldn't care. I I didn't realize how much I would miss that the interaction of teaching. You know what I mean? Like it, and it kind of translates when it's like, okay, you've got big teams and you're leading the team and all that. But I don't know, When you have a really good team, you shouldn't have to teach them Yeah. Too Right? When you have junior teams, yeah, you're teaching a lot, but, what, you know, as I've realized over the years, like the, the real people that you want to hire, the people that are just like, they know this way better than you do, and you are enabling them to go do their job. Do you miss the teaching aspect, though? I think it yeah. Very similar to what you're saying. I miss the actual interactions in the classroom where sort of you explain something and you kind of see the light bulbs go all the out. You get sort of good questions back that you can tell that they are sort of grasping the concepts. I miss that for sure. I don't necessarily miss all the prep that needs to happen before I show up in the classroom. And I have very similar feelings for I also had a research lab and I had my students that I was working with on research projects. And over time, I realized exactly this, that I enjoy mentorship and I enjoy teaching and I enjoy research, but I don't think I enjoy mentorship through research. Well, like, for the research projects, I feel like I enjoy working with people who are sort of already there, maybe even better at me on many dimensions. And it's like more of a collaboration where together we are just getting so much more done. And so I like those separated, yeah, related to what you were saying. And so you you were at Meta for eight years. And did you leave there to start YTORY? Yeah. Two years ago. What was the what was the catalyst? What was the thing that that finally got you to do it? Or were you thinking about it for a while? Or was it like a, okay. Well, I'm ready. Let's go. Yeah. So I think a couple of things. One was over time so right before I left, I was a senior director, which meant I had multiple teams reporting to me. And so very strong teams, amazing work. This was Gen AI. At the time, it was sort of the top priority in the company. Strong team leads, exciting work. But I was missing sort of driving the work itself. I felt like I was more and more in a position of making sure right people are talking to each other and someone's complaining about something today and I'm sort of addressing that. Or like leadership saw something released from someone else and we're trying to be like, oh, it's fine. Like, we'll get there. And so I was missing just being in the details of the work and driving it. So that was sort of a bit of a trigger. And I gave it time because it's like on paper, it's an awesome position to have. And it's like, who walks away from something like that kind of a thing. But yeah, it was very real. And so I was looking for the change. Separately, Dhruv, Das, and I, so the three co founders of Yotori, the three of us have worked together for over a decade, very good friends. We've talked on and off about doing something together going back like six, seven years. We had thought about it very seriously right around when Das was finishing his PhD in 2020, but we didn't do it at the time for various reasons. And Das independently was starting to feel saturated in his role. He was also at META in FAIR. And so he was thinking of viewing and starting something up anyway. It was like, we should And do this yeah, that's how it started. One thing going back to teaching real quick. One thing that I feel like has become a little bit of, like, a superpower for me, I wonder if it's the same for you, is I especially in the AI world where things are are complex and, you know, I've I feel like I got very good at breaking complex things down to be simple to explain to people. And, like, I feel like I've had to have that as a teacher if you really were gonna get those light bulb moments. Isn't it kinda nice having that skill set in the world we're in today where, yeah, sure, you don't you never know exactly someone's literacy of AI, but I'd say the vast majority of the people you talk to in everyday life are nowhere near the conversations that you might have with peers. And the ability to kind of break it down and explain those concepts is really helpful, I find. Yeah. Yeah. I hadn't thought of it that way. You mean in terms of, like, working with AI models and Not only that, just explaining what you do to someone who doesn't understand AI. Yeah. Right? It's there is a skill to that that you hone after, you know, when you've taught for a while. Yeah. Yeah. That makes sense. That makes sense. And actually carrying that forward, I do think I hadn't thought of this earlier, but I do think it probably helps with working with models and agents and coding assistants, being able to sort of break it down and have just the right level of context needed to get this thing done. Yeah, I haven't seen any studies looking at these correlations, but they might exist. Our CTO the other day sent me a screenshot of someone at our job who, he's like, what the hell is this guy doing? And it was like a clawed his clawed usage of like $500 over the weekend. You know, it's like, okay, so either, either, you know, there's a lot going on. They run it on multiple computer, whatever it is, or he is not explaining this in, in a good way, right? It's just consistent prompting over and over and over. So you're right. It probably is a good skill to be able to like speak to computers at this point. Yeah. Okay. So let's fast forward years 2030. Not that far, but, you know, a lot can change between now and then. Where do you think, in terms of where of what Yatori wants to affect in people's lives, like, where do you think what do you think the world looks like at that point? Do you think that we will, by that time, most people will be comfortable with AI just listening to their whole day? My Technology listening to them. Yeah. My guess would be no. My guess would be no. I feel like there will still be people who are and just even looking at the distribution of opinions right now about non AI and other technology stuff, right, I feel like there tends to be pretty heavy tails of extremes. And I think, yeah, I don't see why that would not be the case for AI. I think there is still the only thing I would say to to disagree potentially would be think about how many people a few years ago even were anti smartphones because how do I get this ad if it's not listening to me? And look. Is it listening? I have no idea. But you there are enough data points surrounding that ads ad engines are able to make pretty damn good inferences about what you might want or need, even if you haven't even if you you don't think you've you've done anything to lead it that way. Right? You say, oh, I mentioned it. But, yeah, if you mentioned it, you're probably also doing something online or sending something in email or whatever. Right? So there are ways for it to get that information without listening. However, even the people that I think thought it was listening was like, I'll never have a smartphone. I don't think there's many people without a smartphone these days. Like it is everywhere. And I just feel like with the with the amount of help that it can provide, right, if these if you everyone had their own chief of staff that was just listening and being proactive about what that person may need and being able to just go do it, I have a hard time imagining how people could hold off for too long without just, you know, letting it into their life. Yeah, maybe you're right. I feel like I still and maybe it's like the distribution of friends that I have, but I feel like I still hear people who are like, I'm going to use a different browser for every single thing. Right, That's true, like not on any meta products or things. That's a good point. Like I have people in San Francisco who tell me that they are still not using coding agents because this is just not how you code, right? Like you just code in. And I'm like, this point, it's like an art form. Like, yes, you can code by hand. That yeah. Yeah. Right. Yeah. We had we had a couple folks at at CADRE who engineers, very highly skilled engineers, could code very quickly by hand. They were resistant. And eventually when and, you know, you you show them and you go, you you're not gonna get punished for this. You're gonna get rewarded because look what you're look. The fact that you have done this much engineering and you know what to look for and what's going go right and wrong, you've diagnosed those problems by hand, you will be so much faster at the Vibe code and the speaking to it and using Clog code than somebody who's new to it because they're going to fall into the same pit trap pitfalls as others. Yeah. Yeah. And that person will know how to break the problem down in a way that is digestible for coding agents as they are today. So, yeah. Awesome. Okay. Well, thank you so much for coming on. Really appreciate your time. You. Thank you. Thanks for coming.
Devi Parikh ran generative AI as a senior director at Meta for eight years before leaving to build Yutori, where she's making AI agents proactive rather than reactive. The design choice that stands out: the first product, Scouts, asks for zero access to your context and only reads the public web. Trust gets earned before an agent touches your money, your logins, or your calendar.
Devi takes Keith through Yutori's "trust staircase" and why read-only was the deliberate starting point, not a limitation. She details the near-term shift from notifying you to completing tasks, where you sandbox specific sites behind auth walls one at a time instead of handing over a browser extension with access to every logged-in state. For the long tail of the web with no API or MCP server, Scouts spin up a remote browser and use a browser-use model Yutori trained in-house to click through and complete the task.
Topics discussed:
Yutori builds AI agents designed to act as a proactive chief of staff. Its first product, Scouts, monitors the web around the clock for anything you care about, a price drop, a restock, a funding announcement, and notifies you when it happens. The design is deliberately trust-first: Scouts starts with zero access to your context and only reads public information, earning permission through what Yutori calls the trust staircase before agents move up to complete tasks behind auth walls the user sandboxes one site at a time. For sites with no API, an in-house browser-use model clicks through to finish the task and assembles the results into artifacts like slides, sheets, and dashboards.
What do you really want these these AI agents to be able to go do on behalf of folks? I think most people probably have a sense for, like, an EA or, a personal assistant and sort of those kinds of entities. And I think the distinction between that and sort of more of a chief of staff, like, entity is partly related to some of this proactive nature of it, partly also related to sort of someone who you trust to just be your voice and to kind of, over time, as you work together, be able to make similar judgment calls as you would. So someone who you just sort of trust more and more Yeah. Where you expect that they will just notice the things that need to be done to just sort of make your life easier, and and you can trust them to do that. You trust their judgment call on it as opposed to sort of you having to take a thing that's going on in life, having to break it down into sort of bite sized tasks that can then be handed off.
If you have agents that are gonna go out and they're actually gonna have access to go buy things, it they need to earn the trust. That is part of the reason why scouts is where we started, where we're asking for zero access to your context. Right? And so we focus just on delivering utility and value to you so that we can earn your trust and sort of start going up the staircase. And we have had people pinging us saying that, oh, it was monitoring this thing. It found it. I can't believe it found it. But by the time I saw it, it was gone again. And so can you just get it to buy it for me? And so we've had people tell us that, like, I wish it could just complete some of these tasks on our behalf. And from there is where we are now starting to take these next steps of completing tasks on your behalf.
One thing going back to teaching real quick. One thing that I feel like has become a little bit of, like, a superpower for me, wonder if it's the same for you, is I especially in the AI world where things are are complex and, you know, I've I feel like I got very good at breaking complex things down to be simple to explain to people. And, like, I feel like I've had to have that as a teacher if you really were gonna get those light bulb moments. Isn't it kinda nice having that skill set in the world we're in today where yeah. Sure. You don't you never know exactly someone's literacy of AI, but I'd say the vast majority of the people you talk to in everyday life are nowhere near the conversations that you might have with peers. And the ability to kind of break it down and explain those concepts is really helpful, I find. Yeah. And actually, carrying that forward, I do think I hadn't thought of this earlier, but I do think it probably helps with working with models and agents and coding assistants Yeah. Being able to sort of break it down and have just the right level of context needed to get this thing done.
As, like, longer term, do you see this being more on the consumer side, product like growth for, you know, individual users? Maybe it's not just on their personal life, but, like, individual user at an organization. Or do you see this as a organizational tool rolled out from, like, top down? We do think that it makes like, for your chief of staff to be effective, it makes sense for it to be both across your personal life and work. That access to your context is very useful, especially when it starts being proactive. And so there, to get access to your work context, it might be a tool that's sort of made available to you through your employer. Hence, we might be selling to your employer and not not directly to you.

