How Coinbase scaled AI to 1,000+ engineers | Chintan Turakhia

[00:00] People are skeptical that large, established, highly technical, highly capable engineering organizations can deploy AI at scale and get any effect. But I think you've proven it's possible. It's not only possible, it's adapt or die. It's just been such a huge superpower for the team. How many engineers are we talking about here? A thousand plus. So we're not messing around here. The company tried to adopt other AI tools and we saw this uptick in adoption. People opened it up, [00:30] My biggest thing is how do I make this damn thing stick? [00:32] Because there's something here. I do think that it's really important when you're doing this organizational transformation that you have a single person with incredible conviction at the leadership level who is also hands on the metal. Show the engineers, not just tell. And the worst thing any engineer could do is just be like, I decree you must use AI. Come on, no one's going to listen to you. [00:58] Welcome back to How I AI. [01:02] obsessive here on a mission to help you build better with these new tools. Today we have Chintan Tarakia, [01:08] Senior Director of Engineering at Coinbase, and he's going to show us, yes, it is possible to drive AI adoption and higher velocity in an engineering organization of thousands of engineers. [01:20] He's also going to show us the new expectations for engineering managers and engineering leaders, [01:25] Less meetings and more code. Let's get to it. [01:28] This episode is brought to you by WorkOS. AI has already changed how we work. Tools are helping teams write better code, analyze customer data, and even handle support tickets automatically.

[01:40] But there's a catch. [01:41] These tools only work well when they have deep access to company systems. Your copilot needs to see your entire code base. Your chatbot needs to search across internal docs. And for enterprise buyers, that raises serious security concerns. That's why these apps face intense IT scrutiny from day one. To pass, they need secure authentication, access controls, audit logs, the whole suite of enterprise features. [02:07] Building all that from scratch? It's a massive lift. That's where WorkOS comes in. WorkOS gives you drop-in APIs for enterprise features, so your app can become enterprise-ready and scale up market faster. Think of it like Stripe for enterprise features. [02:37] today. [02:38] Chintin, thank you so much for joining. What I love about what we're going to talk about today is we spend so much time talking about the individual vibe coder or the non-technical person becoming a software engineer and still people are skeptical. [02:53] that large, [02:54] established, highly technical, highly capable engineering organizations, [02:59] can deploy AI at scale and get any effect. There's still so much skepticism, but I think you've proven it's [03:06] possible and you're hopefully going to show us the way. I think it's not only possible, it's, you know, adapt or die.

[03:15] Um, like... [03:17] It's just been such a huge superpower for the team. [03:21] And we've gotten so much efficiency out of it. And there's just like ways to approach it. I think I was reading a-- [03:27] tweet yesterday uh just about a very very long story at microsofting or someone like pulling copilot in to their organization and it was just like just a fun tweet of just like yep we're gonna make graph go up into the right but like the actual adoption wasn't good and and so like i've been spending the last year just absolutely obsessing about it and [03:48] You can do it. People can do it. [03:50] So how can you do it? Because, you know, how many engineers are we talking about here? A thousand plus. Yeah. So we're not messing around here. Yeah, we're not messing around. This is a real team working on real products who know what they're doing, who have built great software. And so where did you start? Either culturally, from a product perspective, from a tools perspective? So I think a lot of it actually just started around this time last year. [04:20] or [04:21] And a big part of that was [04:23] effectively rewriting the entire product from scratch, turning it from a self-custody wallet to actually a social consumer app that just happens to use crypto. And we're using React Native, but we made a lot of decisions for a self-custody wallet. [04:41] But to become a consumer app, you gotta like rethink everything. [04:44] That was one.

[04:45] Two, we needed to do it in like six to nine months. So we were going head to head with like the big social players out there that have... [04:54] multi-thousand person teams. [04:56] that have a 10-year head start. And we were really trying to just do something big and new [05:01] And crazy, like absolutely just crazy. [05:03] And a big part of this is like, how do we rewrite the app [05:09] So that is the best possible app out there, like consumer grade, and do it in this insane timeline. And the team is cracked. They're amazing. But like... [05:19] you know, we became a smaller team as a result of some of these changes. And so I started just looking at [05:27] like [05:27] ways to accelerate. And you know, like, I don't know, my team knows me well, and if you know me, like, [05:33] I obsess about efficiency. And I think that's like so critical to like make teams accelerate their velocity. But in ways that makes sense for tool and using the tool. So around this time, I think Cursor had come out. [05:48] with their sort of initial release. [05:50] It was around like November of last year. We all tried it, right? 2024. [05:55] And it kind of sucked. And it's not like, I love Cursor. I love Cursor. The models weren't there. Just the models weren't there. Like, the models couldn't even, you know, really write a unit test right well. And... [06:06] You know, [06:08] You're an engineer. Um, [06:10] And you understand like [06:13] Once an engineer tries a tool and they're like, ah, this is not so good. Like, it's very quickly and very easy to write it off, right? It happens. And so we kind of went through this like trough of sorrow. It's just like, okay, goddammit, AI tools are not here. The models aren't ready. What are we going to do?

[06:34] And [06:36] You know, for even a year prior to this event, like, [06:40] the company tried to adopt other AI tools like GitHub Copilot. And we saw this like uptick [06:46] in adoption, like people opened it up, checked the box, did kind of like a hello world thing, but it didn't stick. [06:53] Right. And like my my biggest thing is how do I make this damn thing stick? [06:58] Because there's something here. And my mental model was just always... [07:03] The models will, the foundational LLMs will always get better. [07:07] And it's like going to the gym. [07:09] You need to go and build your reps and try it. And that's okay. And the cost of doing it is like nothing. It's just a little bit of wasted time. We're not worried about compute right now because it's so early. [07:20] And so like from basically January all the way to like March or April of 2025, [07:26] Thank you. [07:27] I just... [07:28] change the mindset and the mentality. I [07:31] I was like in cursor every single day, every single hour of the day. And I was like, how do I make this work? [07:37] Right? Like, [07:39] You know, it was great because I was writing code again. It was great because, you know, it was unlocking all these, like, use cases. Like... [07:47] We were doing interviews, like interviewing candidates and just like, I don't want to necessarily write up all the notes, right? That takes a long time. But intuitively, I know, I've assessed, right? [07:56] So I would use it for like tactical day to day paperwork kind of things to accelerate me. But also from like a coding perspective, [08:03] We just pick up bugs.

[08:05] and be like, hey, let's try this, right? What's going to happen? What can I learn? What are the tips and tricks to show the engineers? [08:14] not just tell. And the worst thing any angel leader could do is just be like, I decree you must use AI. Like, come on, no one's going to listen to you. I have to empathize with this because I also running a large, like multi hundred person engineering organization. [08:31] you know, was experiencing even early versions of these tools and had such innate conviction [08:37] That it would, of course, [08:39] Transform how we did work. Like that was very obvious to me. I don't know. It's obvious because of experience or obvious because it was just obvious. But yeah. But then, you know, you just had these experiences as leaders, especially in the, you know, maybe 12 months ago. [08:54] one engineer tries it [08:56] Doesn't work. [08:57] It's not just that engineer throws it away. It's everybody else says, well, I think, you know, I trust their opinion. And if they say it's not going to work, it's not going to work for me. And I do think that it's really important when you're doing this work. [09:11] organizational transformation, that you have a single person with incredible conviction, [09:16] at the leadership level who was also hands on the metal because [09:21] until you can say, well, [09:22] I understand it didn't work for that, but it worked for these three things. Or I actually figured out how to make it work for that because we tried A, B, and C. [09:30] I think it's just the only way. You cannot be in philosophy. You cannot be in, you know, someday in the future, you figure it out. You have to actually get back to it. And then I think like bonus points, so many of us in engineering leadership have like,

[09:45] been pushed away from coding. I know. I was so happy to get back in it. And I'm like, I just want to code again. Like, give me some joy. Give me some time. Yeah. And so I think that's the benefit as well. And you have to show, not tell. And so I did, and like [10:01] I think what I learned very quickly is like, okay, there's something here. There's a there, right? And then we just started picking off like one or two use cases, right? [10:10] And the best way to get to an engineer is just... [10:14] Give them the tools so they stop doing the shit work and so that they can build the stuff they love, right? [10:19] Right? And so like, we would just like pick off unit tests, we'd pick off like linting, all these like little things that just like, [10:25] paper cut and suck the soul out of you as a builder. But the engineers and, you know, like the team, [10:32] just wants to move faster. The team wants to build better things. [10:36] And so we started leaning into like cursor rules for some of these things, even the simplest thing I remember, like, [10:43] I think I remember my aha moment, which was... [10:46] like popping in some bug report, working through it, [10:50] And then I didn't think about it. I just did it. I was like... [10:54] Just create a draft PR. [10:56] Here's the ticket. [10:57] Here's kind of the PR, like, you know, here's the PR description I want. And it just did it. [11:02] And I was like, [11:02] I never need to remember get status, get rebased. Not like, why is anyone doing this anymore? Like, like, what are we doing? And it took, a funny thing is, it took some convincing of me to the team. Like guys, just type, create, draft a PR, like create a draft PR and it'll be done for you. And like, like, well, you know,

[11:23] I kind of have my workflow. It's like, cool, cool, cool, cool. I get your workflow. You can modify it. You can use cursor rules. It's okay. Like no one's getting bonus points for memorizing git commands. Exactly. Exactly. And so like we chipped away. [11:38] And we put in a bunch of rules, like cursor rules, and that helps so much. [11:43] And then, like... [11:45] I was like sensing, I was like, okay, I have enough like folks on the team that are like, yep. [11:50] This is unlocking stuff. And they would post in the team channel, like, look what we had. Literally, a channel called Cursor Wins. And, like, everyone was just posting in the channel, like... [11:59] I just did like, you know, 20 unit tests. [12:02] and then went and had a coffee. This was great. Like, I love it. And so people started seeing [12:08] it in action. [12:09] And then we hit this like point. I was like, okay, how do I speed run now the whole team? There's a little bit of conviction here. [12:16] So you just-- [12:17] And I remember this, like... [12:19] I think I had landed... [12:21] I was going to the East Coast. I landed... [12:23] for my flight, got into an Uber, [12:26] hopped on like an entire team [12:29] all hands, like speedrun. We called it, it was like basically cursor speedrun. [12:32] And I was in the Uber, [12:34] using cursor, putting up the PR. And the goal [12:38] of the speedrun. [12:40] was every single person would just pick up the most trivial thing [12:43] It could be like copy change. [12:45] a bug, whatever, and just put up the PR. [12:48] And we ended up, I think in 15 minutes, I think a hundred people had joined in 15 minutes. We ended up putting up like 70 PRs.

[12:55] And we broke GitHub too, which was cool because we learned like our infrastructure needed improvement. [13:00] So I want to pause real quick because, again, How I AI, a little bit about tactical techniques, and you've used a couple that I have used, which is like [13:07] one high conviction leader with hands on the metal that just says, like, we just got to do this. [13:13] access to tools, [13:14] Focus on toil. [13:16] I think it's very important. You called out linting, you called out tests. Another one I would call out is like design debt. [13:23] where, you know, front-end engineers or designers have just lived with parts of the app they hate. Yes. That is another really great one. And then a shared Slack channel. And one, like, you know, riff I would make on your Cursor Wins channel is... [13:38] we made ours wins and losses. And so we were very clear, like, [13:42] just post what you did and when it worked and when it doesn't because when it didn't people would be like oh yeah but you could try xyz or i have a cursor rule for you or whatever [13:51] But what I haven't heard that I want people to just like perk their ears on and pay attention to is this like idea of a PR speed run. [13:58] which is like do a timed downtime, [14:00] Everybody boot up whatever tool and just... [14:04] speedrun some fixes because how much conviction does an org have to get going from, look, I've been there, like the doldrums of like quarterly planning and this will be in four months and blah, blah, blah, blah, blah, to just like, we just got 70 PRs that we've been sitting on out the door in 30 minutes. I just, that has to be such a transformational moment for an edge team. You know, there was a success rate on those, on merging those PRs and like, it was just like shit.

[14:33] this is possible. Everyone's eyes lit up. And it was really sort of a death to status updates, long live building moment. [14:41] Yeah. And this is the other thing I want to call out because I think you all have a really special culture there. But so often we in product engineering design orgs get like really wrapped around the axle on like the rules of engagement. Like, well, I'm not allowed to build it unless the product manager says it's important. Or like, I can't really make that decision about what color that button is because design hasn't weighed in. And like, I do think these moments where you just break all the rules and you're like, guess what? Remember, you can just... [15:09] Ship code. You can just you can just ship code like put AI aside. AI maybe enables it and makes it like a much less costly, you know, expense. [15:20] But like just doing that... [15:23] is so powerful for velocity and for I also think for quality, like people just take more radical ownership of things. [15:29] So I'm going to 100% steal this. I mean, I want everyone to steal it. Like, [15:35] You know, I really like the way you just put it, right? [15:38] This is a moment where we should be breaking the rules. Because AI is breaking the rules for us, and if we don't adapt to how we can use it, [15:45] We're toast, right? And we is like a very collective, like, whoever's not adapting is going to fall behind kind of thing, right? [15:53] And what all of this like ends up unlocking is. [15:58] is like the reduction in coordination overhead. So like one thing I've been obsessing about a lot is like, okay, cool. Great.

[16:05] Good job on the speed run. [16:07] Yes, we got a lot of stuff done. [16:09] We started then seeing those wins, more and more people adopted. [16:14] Brian then, you know, we were sharing some information with Brian, like how adoption is going. And then we just did a company wide speed run. And at that moment, like there was like 800 engineers on the call. [16:24] And we ended up pushing up for like three, 400 PRs in 30 minutes. And yes, again, we broke GitHub. And that's fine. That's good. Like this is pressure testing. [16:33] we should be designing ourselves to break the rules, right? But the thing, [16:37] I've been obsessing about. [16:39] is like, how do you... [16:40] How do you measure any of this like [16:43] in terms of output, right? There's this like tension [16:47] where, okay, [16:49] The more AI we use, well, does that count as a replacement for people? [16:54] And like, I'm in the camp of absolutely not. AI is an accelerant. [16:58] AI is an accelerator because there will always be more work to do. And so the way I think about it, at least for my team and what I'm pushing across the board, [17:09] is really like [17:11] time from [17:13] Ticket? [17:14] to when the change lands to the user. Like that actually encompasses every single piece you need. [17:21] Thank you. [17:22] And today, even if you go from like [17:26] Ticket backlogs and stuff like that. Like there's, oh, do I, should I, like you said, should I prioritize this? Is this important? Let me ask my PM or let me ask the program product manager, project manager, whatever.

[17:39] And [17:40] Now the whole team, like... [17:42] Fast forward from back then to now, [17:44] We just see... [17:46] Someone give us feedback? [17:48] And literally within seconds, we're like, we built this internal bot, I'm excited to show you. And within seconds, like, [17:56] The PRs [17:57] being authored. [17:58] Right. An agent picks it up and within seconds, that feedback is like acted on. And so we crunch the time to action. [18:05] The time then from ticket to the... [18:07] The PR being ready for review? [18:10] Then the review time, like... [18:12] All my devs complain, review times take too long. [18:15] We found some solutions actually [18:17] I think we are doing average of like [18:20] 150 hours, like was the cycle time for a PR review because there's so much. We reduced it by 10x down to like 15 hours or so, roughly. [18:30] And then [18:31] the last piece is like from that merge [18:34] How do you do like that OTA update? And you squeeze that whole cycle again. [18:39] And then the team is like just literally unlocked with sheer velocity. [18:43] Yeah. [18:44] And then you get stuff in front of customers. Yes. And then you have the velocity of like actual market ideas. Yes. And you get that feedback and like the we're obsessing also about how fast can we take like in real life feedback. Yeah. And then. [18:57] Actually just fix it right then and there. I think there was another aha moment. I was on a call with a user of our product, right? And they're like, hey, it'd be cool if you changed X, Y, and Z. And literally while I was on the call, I just put up a PR and pushed it. And they're like, before the call ended, it was 30 minutes. I was like, just reload the app.

[19:17] It's fixed. [19:18] Okay, before we turn this into an hour of, you know, like two product leaders being like, just ship really fast. We'll go into the merits of reducing PR cycle time, all that fun stuff. Let's actually show a couple things you built. Because I think the kind of meta commentary on like, you can do this in engineering organizations, there are steps to it. There are measures you can take, I think are things that everyone can learn from. But you also have been building. [19:48] THEIR HOME. [19:49] to drive how you drove this into the organization and understand adoption of AI. [19:54] Yeah, for sure. I think a lot of it just comes like from honest [19:59] curiosity and figuring out [20:02] um where the bottlenecks are like why aren't folks adopting how are people using it etc etc i want to show you like i think the the kind of crazy thing i'm about to walk you through [20:11] It's like, I just got this harebrained idea. Cursor has like great analytics, right? And so you go to the admin panel, you look at the analytics, [20:18] And, you know, awesomely, they let you download it into CSV. I was like, what if I just use cursor? [20:23] to figure out [20:25] what my team is doing in terms of using cursor [20:28] But not in just like from a vanity metric point of view of like lines of code. [20:33] committed by AI. I think that's like kind of misleading, actually digging more into, um, [20:40] how they're using cursor and how do we sort of like replicate [20:44] power users so let's see

[20:47] We have some [20:48] Some data. It's in this file here. [20:52] And it's just like a standard CSV from Cursor that you can like download from their site, like your admin panel. And then there's also here a bunch of different sort of fields. So like accepted lines, chat lines. [21:07] Chat lines deleted. Um... [21:10] Various like data elements, but you know one thing like I just sort of started with I want to understand the usage of cursor, right? And I already know we have like light users and [21:23] all the way to power users. [21:26] And one of the things I really wanted to figure out was like, what are the natural clusters of usage? Can you find them across the team? [21:35] What is the best way to cohort them? [21:38] Right. And I'm just gonna pick up [21:42] the standard analytics file here, maybe pop in another one here. And then I love Opus High. I also love plan mode because it gives you a chance to like see what it's thinking through. [21:55] So we can let this cook. [21:57] and see what it comes back with. [21:59] And what I want to call out here for engineering managers or engineering leaders is, [22:04] This is the kind of quantitative analysis that we would all have loved to be able to do. [22:11] across a bunch of engineering metrics at some point, right? Like how often do we get asked by, [22:16] the board or our boss, like what's velocity, what cycle time, which of our engineers are super, you know, like are really on the far edge of the curve in terms of efficiency. How are our junior engineers ramping into the repo?

[22:29] All that kind of stuff. And that kind of analysis is actually really onerous and hard to get at. [22:33] because of the structure of the data. [22:35] and the nature of the analysis. And so what I love about [22:38] just LLMs in general, and in particular using something like Cursor, is you can get to really nuanced cohorting analysis on human behavior and human analytics as a manager in a way that I think has been really challenging to do before. [22:53] Yeah. [22:53] I totally agree. [22:56] The beautiful thing is now with MCPs, with data accessibility, [23:01] Like I think of tools like cursor as just my daily operating system. If I have a question, it doesn't matter if it's technical or not, I just go into cursor and ask it. So it's like super, super powerful that way. Okay, so it's asking me a little bit about like, [23:17] What outputs do I want? I do want it in root CSV, just it makes it easier. [23:23] I do want a static dashboard just for fun. Like I'm not really trying to create a brand new dashboard right now. But my main goal here is just honestly, honestly, like [23:33] fine natural cohorts, right? And so it's going to kind of try to do light, moderate, active, power, super user. It's going to look at lines suggested, so volume, sophistication, agent mode, model preference, acceptance rate, and breadth. What features are they using? [23:47] I'll spit out, you know, [23:48] A CSV dashboard, you know, likely generate a Python script too that it can reuse. [23:53] So I'm just gonna kick off build mode. While that's cooking, I do want to just maybe bop over to like

[23:59] It's going to create all this stuff in Python, create the scripts for me. Awesome. But we can look here. [24:06] at some of the the information right [24:09] So like this is all sort of random made up data. It's like sample data. But what it [24:15] did was in a previous run, it looked at all the data, [24:19] generated the Python script, which is great, super simple. [24:23] And it sort of just did some like high level status metrics like AI code percentage again on all this made up data, AI lines per week, composer lines. This is when you're using the agent mode in cursor or tab lines, right? When you're hitting tab. One of my team members actually got the cool cursor tab award, which is great. [24:41] And so it sort of breaks all this down and then... [24:45] What it really segmented around was like agent heavy users, which is folks who [24:51] really lean into agent usage. [24:54] There's also tab heavy users. This is like a different cohort. They just lean into tab usage. And they maybe want really just a bit more control and [25:01] Maybe they haven't gotten yet used to. [25:03] like how to let go with an agent. You have balance users that try both. And then you have sort of like, maybe cursor curious or maybe not cursor pilled or, you know, LLM pilled right now. [25:15] And so I generated this whole script [25:17] It's great. And now let me show you sort of a bit more analysis I want to do here. [25:24] So let's do this. Run the analysis on, I have a sample user set.

[25:32] And generate the HTML as well. [25:35] And let's, we actually like, this is sort of the output of the analysis script that was generated in Python. [25:42] Which is already cooking in parallel. [25:44] Got it. So what you've done here is you've taken some raw data from Cursor. You've asked one kind of agent to do a cohort based analysis and generate a enriched CSV essentially with some data and then you're kicking off another agent. [26:00] to actually do the analysis on that and generate sort of an HTML view of it so you can... [26:07] visualize the data. That's right. [26:09] That's right. [26:10] What it did was... [26:12] The Python script that was generated, right? It found these natural cohorts, these natural cohorts of super user, regular user, power user, light inactive. Again, this is just... [26:22] Honestly, sample data, but based on real information, real schema, real cursor data fields. [26:31] And it came up with like 70% are an agent heavy in the sample data, 20% are minimal, 4% are balanced. [26:38] We have some room to improve here on the sample, right? Like not enough people are using it. [26:44] and [26:45] So it does a bit of a breakdown, which I kind of like, you know, [26:49] Kind of a recap of metrics. Yeah, we have a lot of lines of code. [26:52] In this data, we have 520 power users. [26:55] again made up names but like this person is crushing it i want to know [27:01] what this made up person Gabriel Diaz is doing, right?

[27:06] Awesome thing here, it generated a little [27:08] a visual dashboard, [27:10] Nothing fancy, something just really simple to look at. [27:13] right total lines composer lines tap completion a little bit of breakdown [27:18] some structuring on the tiers and usage, right? But what I really kind of want to understand is like, what is... [27:28] Gabriel Diaz doing, right? This made up user who's just like crushing it. Yep. [27:33] How about based on the data, generate guidance for each user cohort? [27:37] What? [27:38] you know, they should do to advance and graduate to super user. I'm looking for explicit guidance. Effectively, like, [27:43] I want to turn this into some type of playbook, right? So let's let this cook. And then in parallel, [27:51] What I also want to do is I like visuals and [27:55] There's something intuitive here where like, [27:58] As we look at the data itself, [28:00] Thank you. [28:01] Right. We [28:03] We know that the path... [28:05] to this super user over here. [28:07] It's not like you go inactive to light to regular to power to super. We know it's not linear like that, right? [28:13] Right? There may be like forks from light to straight to power user. Regular user seems to be like balanced on the tiering. But what I want to know is like, what are the special things these folks are doing? [28:25] And how do I sort of shift the curve? Right. And so [28:29] I'm also going to throw another question in parallel, like create a mermaid diagram for all the different sort of paths a user can take from light to power. And it's I'm assuming it's not linear.

[28:40] And let's just see what this cooks up to. [28:44] Okay. [28:44] This is really working hard. Really working hard. Opus 4-5. Yeah, Opus is... [28:51] I hope this is really working hard on this, but-- [28:54] Yeah, let's see where it goes. [28:57] Well, you know, it's really interesting. I'll give you a shorter hack on this one. So I think what this is generating is, [29:03] is like an HTML playbook that you could share out that has things. I will tell you what I would do in this use case, and I've done this a couple of times with customer QBRs, is I say write a Slack post that I can put in my engineering channel [29:18] on a couple of these stats and how we can get people to move from [29:23] A to B, and it will write me like a short little slack post. So I love this idea of going from something like a CSV [29:31] to a really deep analysis, to an HTML-like visualization, to like three bullet points I can send in Slack. And as a manager, each one of those steps would have taken just forever to do. And now you can get them all done in Cursor. [29:46] Yeah. You know, [29:47] That is, that's like kind of the awesome thing is... [29:51] the power of something like a workflow markdown file is huge. It's absolutely huge. And it's exactly like the thing you're describing here. [30:00] Meet Rovo, your AI teammate, connecting knowledge, people, and workflows so teams can work smarter and move faster. It helps people find answers, make decisions, and automate work, securely and with context, through Search, Chat, Agents, and Studio.

[30:18] Rovo runs on the Teamwork Graph, Atlassian's intelligent layer that unifies data across your first and third-party apps so no knowledge gets left behind. And you always get personalized AI insights from day one. And the best news? It's already built into Jira, Confluence, and Jira Service Management paid subscriptions, so the power of Rovo is already at your fingertips. Know the feeling when AI turns from tool to teammate? [30:48] you [30:48] If you rovo, you know. [30:51] Discover Rovo, AI that knows your business, powered by Atlassian. Get started at rovo.com. That's R-O-V as in victory, O.com. [31:05] Let's see, let's see what it came up with. Right. And like, [31:08] You know, the thing is, like, no one should expect all this information is going to be perfect. Like, if anyone is thinking, oh, wow, what is going to be my job as a leader if Cursor can do all of this? It's like, well, your job as a leader is to lead. [31:22] right and to make change and impact and this accelerates so [31:27] Inactive users, like, yeah, kind of true. You have it installed. You haven't really used AI features yet. [31:33] The hardest part is getting started. [31:35] I kind of like this. It gives like... [31:38] Just some very simple prompts. [31:40] Try the agent mode for your next task. [31:42] Something very, very simple, something like, wait, type, try a tab completion flow. [31:47] I kind of feel like

[31:48] The LLM really wanted to just turn this into a game. Like a little quest or something. Yeah, it's gamified a little bit. Yeah, it is a bit gamified and it's kind of fun. All right, so this is cool. It's kind of given me like... [32:02] This would be my Slack post, TLDR. 16x more AI lines, super users versus other users. Let me zoom in just a bit more. More agent requests for super users. I love this. Stop typing, start shipping. It's dark mode, so the engineers will just love it. Yes, right? It's kind of perfect. And then... [32:22] You installed cursor, but you haven't used AI yet. We talked about this. That's cool. Lightmob. Okay. You know, this resonates. Stop saying fix this bug. Actually, like... [32:33] talk to it like you would maybe a junior engineer right um cursor just did release bug bots i love [32:40] I love BugBot. [32:44] Agent isn't for hard stuff. It's for everything. These are like motivational quotes now, but I think like we should just make posters for and put them up on the wall. [32:54] Write unit tests, actually read the comments. Okay, cool. Now power users. [32:59] You're good to be great, think bigger, and to have the harder. [33:03] if cursor is listening i think this is like gonna be your new merch line guys i need a hat that says tab harder [33:14] Yes. Okay. So just to recap again.

[33:20] We're doing free product work for Cursor here. We took, you know, your ultimate problem was like, how do I drive up adoption of these tools? And you're like, of course, I'm going to use the tool. [33:30] to understand adoption and then figure out ways to drive adoption. [33:34] We did analysis, we created a visualization of the [33:39] the data itself, you identified cohorts and power users, which would have been very tedious to do if you were going to do manually. Yeah. And then you created a [33:49] hosted playbook as well as a series of motivational statements which we can either give to our friends at cursor for free or trademark right now and make a little money. Agent everything tab without thinking bug bot always on. [34:09] Iterate prompts. Love it. It's kind of fun. And this, you know, again, what I think is fun... Let me talk about what I think is fun about this. One... [34:18] And everybody who has been in engineering leadership knows this is the kind of stuff you get asked to put in a board meeting, you get asked by your boss, like, [34:27] What percentage of our engineers are using cursor? Do we have power users? Are we actually getting value? And we're talking about an AI use case right now. But again, across management, there are actually measurable. [34:39] things you can do about the performance and efficiency of your team. Yes. And I think it's been so impossible to get before. [34:46] Two. [34:47] It would be no fun. [34:49] If you didn't get to do it with code, which you get to do with code. Actually, that is the thing. You can solve problems with just code now, right? You can just do things. You know, you're so right. I think this...

[35:03] I underappreciated exactly what you're saying right now, and I just want to repeat it. [35:08] Because normally, [35:10] You would be asked this, and then you would have to go pull an IC to do that. And like, what? What? [35:16] Yeah. Come on. Like, no, you can just do things right now. And again, it's like not that I think people under appreciate the velocity creation of a fun task. Yeah. Which is like at the end of the day, like this is silly, but also the like little fun bits of it. You're like, great. I want to go to the next level because I got like a little dopamine hit from this dark mode playbook. That's kind of funny. And I think people under appreciate like that iteration speed that can just come with like a fast feedback loop. [35:46] Yeah. [35:47] When you're building something and the fast feedback loop, when you're building something that has high quality against it, which like something designed like this does so much more fun to look at than a Google Doc. Yeah, I totally agree. Or a spreadsheet or a dashboard. So we did it. We did it. Again, you and I are... [36:05] twin stars i think here and so we probably go all day on the things that we find fun but let's go to a second use case that i think people are going to see and let's see how fast we can do this use case which is [36:16] You were talking about the speed of feedback to feature. [36:19] And you said some fighting words out there. You're like, we're really compressing the time from feedback to feature. So how does that actually... [36:27] work. [36:28] Those were some fighting words. [36:31] And... [36:32] You know, I think...

[36:34] You know this, right? You want... [36:36] You want to build this for your users, right? And you want to create the best damn product out there as fast as possible. And the way to like make that cycle work really well is genuinely how fast you can move. [36:49] feedback okay [36:51] But I wanna start from, [36:53] how does like feedback even normally come in, right? So, you know, normal like teams and culturally, like you'll have dog fooding or bug bash sessions, right? You'll get on a meet or get in a room, keep using the product, blah, blah, blah, all that jazz. And then, [37:10] someone has to like collect the bugs in a Google Doc and then take those bugs in a Google Doc and put them into [37:16] a ticket system, right? Okay, and then there's a whole discussion around... [37:23] Is this important? Is this not important? Okay, should we pick it up in this sprint? Should we wait for another sprint? And by that time, your user has turned out. [37:32] They're like, [37:33] You guys didn't fix this. I kind of hate it. Moving on. Right. Everyone's attention is like so, so, so short. And right now, like the whole team, we're all preparing for a big launch. [37:45] And [37:46] we wanted to get together and do this thing called a surge. And this is where we just bring the team together. And we do very, very long days using all this AI and just shipping massive amounts of code. [38:00] uh and fun fact like during these surges we end up shipping like more than three to four x more pr volume in the same time

[38:07] But... [38:08] The other thing we wanted to do [38:10] was bring people into the office. And we set up this thing called like a feedback cafe. [38:16] So we'd invite externals, internals, etc. And we'd dog food with them and we'd show them the app. And like, here's just like a couple seconds of, you know, what it looks like. [38:28] We're just standing there. [38:32] collecting information, doing all this like live dogfooding. And the hard part though is especially in real life. [38:42] How do you actually capture that information? Because it's voice, it's video, [38:46] How do you translate it into a system? [38:49] So. [38:50] I just spent like [38:52] Have a weekend. [38:53] and built a tool to capture feedback live. [38:57] Let's just pick something. I'm gonna pick... [39:00] I'm gonna... [39:01] Pick a new thing. How I AI. Testing... [39:06] with clear. Awesome. [39:09] So let's do that. It's going to create a little session. Perfect. Very simple. And we have two modes. You can use this on your mobile phone. That's what the team did when they were in real life. But for this, I'm just going to capture some audio and... [39:24] Let's see what's what's actually maybe I can just hear from you like a fun little bug. [39:29] or something. [39:30] of a product that you think you want to fix. So we're going to start capturing audio. [39:35] Thank you. [39:36] There is a AI chatbot that I use where my account when switched to business account,

[39:43] forces me to clear all my chats. And I think we should fix that bug so that I can access my existing chats. [39:50] We're going to start capturing audio. Okay, cool. We captured it. [39:55] It's basically taking the audio... [39:58] I did a system prompt, sent it to an LLM. And then what we do, [40:04] is the prompt is basically saying, go and identify the bugs. And then I'll create it. I'm going to do one while it's processing. [40:11] Right now, I'm using the app, I'm on the trade tab, and I'm clicking the "from" field and I'm typing in numbers, but the numbers are not showing up. So that's not letting me make a trade. [40:22] So I think in our first example, [40:25] the audio is a little hard to capture just because it's going through the system but let's look at the second example [40:30] It calls it out really clearly. On Trade tab, typing into from field does not display enter numbers. User cannot initiate a trade. Cool. Really clean. Yep. I hit create linear ticket. [40:42] It even gives like a suggested title. [40:44] The user journey I care about for this is trade. Boom. [40:48] I create the ticket itself. [40:51] Awesome. I pop over. [40:53] The ticket is all here. [40:55] The file is there. [40:57] Linear is an incredible tool, it's doing some triaging. But the thing I want to now hop over to is... we're gonna just create the PR. [41:06] So we have this tool we built in-house. We call it CloudBot. It's actually like using all sorts of underlying models. It's not something that is specific to

[41:15] to Claude. So Claude bought CreatePR. I know the repo for this is Wallet Mobile. And here's the ticket. [41:25] Oh, that's not the ticket. The ticket is boom here. Great. [41:31] Cool. So... [41:33] I just went from [41:35] A bug report? [41:36] to ticket. [41:38] Thank you. [41:38] To a PR. To the PR is cooking. [41:42] Okay, so I have to pause because if you are new to How I AI, you have not seen my signature move when I really love something, which is this. [41:50] And I was doing this because I was just thinking about... [41:55] This little micro app that you have on the left side, which is... [41:59] you know, live user feedback. [42:02] totally unstructured, right? Video or audio. [42:05] Run a little baby LLM on it. Get not only a summary of the issue, but a good recommendation on how you might fix it. [42:15] Very quick beep boop to linear. We love our friends at linear. I think it's a great platform for agents and then a little custom agent in your Slack that can read those linear tickets and just execute on them. [42:26] And [42:27] Again, so traumatized by the past, maybe, which is like, this process would have been... [42:36] You know, somebody... [42:38] manually summarizing what came out of a research session. [42:42] Some document being written. Somebody actually making explicit decisions about what to include and not include. The decision making is gone.

[42:50] Yeah. [42:51] Like no filter anymore. You don't get that like, well, you know, if I make this five pages long, no one's going to read it. So I'm really going to focus on the top 10 things. It's like, [43:00] Let's capture everything. [43:02] And then just burn through it. And then, [43:04] I have to ask you, why did you all build your own little bot to do this? What was the advantage? [43:10] of building the bot. [43:12] So this is like in-house and we built it. You know, it all started... [43:18] Around like middle of this year I created this like [43:21] I was just obsessing so much about AI. And I was like, how do I... [43:26] How do I create better tooling for the team, for the company? [43:29] So everyone can be accelerated. So I invented actually, like I put a call out on Twitter, I invented this role called super builder. [43:35] And the single job [43:38] single most important job of a super builder is to create more super builders so we we hired our first super builder [43:45] And they... [43:46] We talked about some ideas and one of the biggest things because [43:50] Most of our company uses Slack. We're all in Slack. And Slack, you know, I'm a strong believer. It's just a bunch of humans pretending to be systems. [43:59] Right. And the cost of writing something in Slack is zero, but the cost of answering something in Slack is enormous. And most of it is noise. [44:07] right and so one of the things was just like how do we bring the workflows [44:13] that we are all so used to, [44:16] And how do we capture that and then add AI on top of it? So we had various reasons. We know lots of companies have background agents, cursor, etc.

[44:27] We just have like different sort of security requirements right now that we just couldn't launch with and that's fine. So we built this in house and [44:35] we have these like feedback channels, right? Hey, there's a bug here, there's a bug here. And so now all we just do is like cloud bot [44:42] go and do something with that. Or if someone is like, hey, we just got out of this meeting. Here's a summarized transcript. We're like, awesome. At linear agent, go break this down into tickets. And then just like, you know, you know, the look you showed like, [44:57] Right? Like everyone is just doing that emoji of like the head exploding, right? Because then now we have like 20 tickets and then we do fun things like this, which is just go like bonkers where we just fire off... [45:17] Tons and tons of calls. [45:20] Right. [45:21] to just... [45:23] Thank you. [45:24] And so we built this plan mode. So this bot has a create PR. [45:30] which it's cooking. It has a... And also the cool thing about CreatePR is when it's done, [45:37] it will respond back it will show you a link to like the cursor branch using cursors deep link [45:44] And when then the one-off build is ready, it will show the QR code so you can just scan. [45:51] and start playing with the fix. [45:54] There's a plan mode, which is very much like cursor's plan mode.

[45:59] It just comes up with like, [46:01] a plan, [46:02] And then we also have explain as well. [46:08] where it's like, "Oh, I want to debug something." So like, why is... [46:14] Chintan's app not working right now. [46:19] Chantan.base.e as an example. Right. And it has like all the skills, all the MCPs. And so... [46:29] The thing. [46:30] The thing I realized is context is the most important thing. So the place where we capture all of our context is linear. [46:37] And then [46:38] this agent that we built, we added skills and MCPs. So if we can capture context through linear, then we can trigger the agent [46:47] using all the context from linear. And then it goes off into all the MCPs like Datadog, Sentry, Amplitude, our internal Snowflake databases, etc. [46:57] and it has the ability to pull context from the rest of the company, [47:00] and it can work across multiple code bases, [47:03] And then boom, like, [47:06] It's a super builder. This is awesome. And so before we move on, I think what I want to call it here are a couple of things that I hope people didn't miss. One is... [47:17] right now, if I can give people career advice. [47:19] Thank you. [47:20] You want to be like [47:22] The... [47:23] the top three most AI people in your engineering organization. I'm sorry, I just have to say it. Like,

[47:29] I, you know, I, whenever I told you, pulled an engineering leader aside or someone aside who's like maybe a little AI skeptical. [47:36] And I said, like, I want you. [47:39] to lead this. I wasn't doing it. Yes, of course I want to do it because I think it has high impact on the company, but I felt like I was doing people a career career. [47:47] favor. [47:48] by giving them this role. And so if you can find companies [47:51] that are hiring super builders that will put you in the role of driving AI across an organization, where you can learn these skills, I tell you, it is... [48:00] incredible benefit to your overall career. And I don't think people appreciate how much [48:07] that is pretty still rare right now. So if you can find it, I would just [48:11] beeline directly directly to it i think the other thing and we've seen this a couple times we saw this amplitude actually did it [48:18] Building your own agents is not impossible for organizations. And so if you do have security compliance, data access restrictions, [48:27] you can't use cloud agents you can't use these things it is not impossible to build these things yourself and there are lots of like really great sdks out there too that you can use um to do so and then you know three like i do think some of these platforms linear and slack are just friction reducers to access to ai and so [48:47] If you are thinking about driving adoption in your organization, like figure out how you can get the right platforms in place. [48:53] that can unlock access to agents because if you ask somebody to open or learn a new tool, it's just going to create too much friction. [48:59] to move forward. I think there's like one super important thing like this. This is a channel where we call cloud bot playground and I'm scrolling through fast just to show you like how much

[49:12] people are using this was one night i was up at like 1 a.m just pushing this we got like 200 bucks right from this tool i showed you and i just kicked them all off in like one solid go and [49:26] just to get things cooking. [49:28] And like, it was great. [49:30] uh let's see if a plan came out here yeah so like there's a there's a plan that comes it actually creates the plan in the linear ticket yep [49:37] The trick here, why slack? [49:41] is because Slack is how things go viral within your company. Totally. If you have pulled out the magic into some separate tool that others can't see, [49:52] It doesn't happen. And so by getting things into Slack, people just like, holy shit, this is possible. Let's go. And it's like, it's really cool. [50:01] I completely agree. Okay, so we have just seen about... [50:06] Everything I wanted to see from the engineering side, but before we get out of here, I want you to spend just a couple minutes... [50:12] On a personal use case. Okay. [50:14] Thank you. [50:15] Let's go. I think the one that resonates probably for everyone is getting, if you have kids, getting the school emails that it's like, "Oh, here are 50 events that are about to land [50:26] Here are the dates. [50:28] I've just started taking a picture of it. [50:30] And then throw it into chat GPT and say, create the calendar invites. 100%. Right? It's like, it's the dumbest thing, but oh my God. And then the shared calendar dance happens. And it's like, it's so great. Another thing though, like I love food and wine. I really do. And like, I've done like Somalia training, et cetera, et cetera. And I realized like...

[50:48] I went to New York recently with one of my buddies. [50:51] he's learning about AI, but he's like, what are some of the real use cases that would resonate with me? And I was like, well, [50:58] Like one of the biggest sort of anxieties people have is when they go to a restaurant, they're handed the wine menu. [51:04] And they're like, what do I pick? What if I pick the wrong thing? So with my friend in New York, we went to some like champagne tasting. And so like I just took notes. There's like this whole notebook. Right. I just did this like an hour ago. And I was like, oh, here's a great producer. Single star means like, yeah, it's good. And then. [51:24] Here's another one. Oh, see, I wrote amazing by like, this is someone I've actually never tried before, but I loved, loved their champagne. It was just super yummy. [51:35] Here's another one. [51:36] Right. [51:37] Effectively, then I just like pop this right in. [51:40] And I said, "Here are a bunch of champagnes that I tasted. Figure out from my notes, like, what are my taste preferences?" [51:48] Well, [51:48] Really simple. Because you know like when I did like Somali classes [51:53] The biggest thing that it teaches you is the vocabulary to describe the stuff you like. [51:59] Right? And then, so it just took the images, it figured out the producers, [52:03] And this is actually like spot on. The fun thing I did with my friend while I was in New York was like we were just he was he's. [52:09] He actually is the real life version of ChatGPT. And it's what inspired me to do this, which is he's always trying to figure out my taste preferences. And so, you know... [52:18] This is like my strongest signal. I love...

[52:21] like these wines that have very little sugar that are like really rip-roaring acidic. I love some aging. I love growers, right? Grower Champagne, not like the big houses that are like very sweet. It even went into like a certain subcategory of like, [52:37] you know, the chalky style. [52:39] this specific producer that I wrote amazing by for. And it also called out something I learned in real life, which is like, I do like Pinot Meunier, but only like, [52:49] with this sort of characteristic, right? Kind of crazy. All right, fine. And so then it came up with like a little bit of like a champagne profile. [52:58] cool and if I'm buying stuff [53:00] you know here's here's what i would buy [53:03] All of that's fine, okay? Like, why on earth would anyone do this, right? Like, people must be listening and be like, okay, maybe just drink a little less champagne, dude. But like... [53:13] The fun thing is, let's say you took, you went to a restaurant, right? And I just did this for this, like, example here. And you just, like... [53:20] dropped in, took a picture, [53:23] of [53:24] the wine menu [53:25] Right. And it's like a big old menu. Some of them are like size of a dictionary. Some of them are simple, but like you don't want to make a choice, especially you just want to be with like talking to the company that's in front of you and not like staring at the wine Bible. [53:38] You drop it in. [53:40] And boom, what it actually comes out with, and I think the prompt I asked is, what would I like from this list? What are good values? [53:49] And it kind of just went through this really fast based on my preferences.

[53:54] And it's right. Like, I would love this. I have had it. And it's great. And it's fun. It shares the price. Absolutely no-brainer. [54:01] Another example, [54:03] Another example, [54:04] And then it kind of gets into a bit more detail, like categorically. Like, look, if you want to value one, [54:10] and just like want a bunch of bottles, go for this. Like everyone's gonna love it. If you want something a bit like more splurgy, try these. [54:19] Right. And very much like it kind of talks about what why you'll like it. What I love the most always says this is the stuff just to stay away from. [54:28] Right. [54:29] And, you know, if it's a big night, then just go get these six bottles and call it a day. And so, like, that's the fun thing here for me. [54:36] So what I have to call out for folks is we've actually seen not this particular use case, but this flow before, which is like how you reverse engineer your own taste. So we saw Hillary at Whoop show how to reverse engineer her own taste on slides. We saw, I forget, somebody else reverse engineered photographic styles. [54:57] Ravi reverse engineered photographic styles and said like here's a photo like tell me explain to me how to how to describe this but you are the first person that has reversed engineered their own taste in wines and I love this and now you can. [55:12] pick yummy stuff to get for you know what [55:15] Six bottle cart. I'm going out with you. [55:18] Next time. I know. We'll celebrate AI adoption or something like that. This has been so great. I have one, two lightning round questions for you. We'll keep them very short and then we'll get you out of here. My first one is,

[55:32] If you look back two years ago to now at work, [55:35] how are you how are you spending your time differently like how has all this changed how you [55:40] personally spend your time [55:42] My calendar is empty, like almost empty. We love it. And the reason why is because the coordination overhead of like, hey, let's prioritize this. Let's change this. Let's change the roadmap. No, you just do things. That's one. Two, I'm writing way more code. The team knows like if their contributions fall below mine, like that's, we got to like help on the AI. But like, look, like I'm also jumping in. The team is doing incredibly hard work. [56:07] I... [56:08] I'm spending way more time in the code base, fixing bugs, trying things, coming up with like technical approaches. I am not a replacement for like, [56:17] the insane amount of talented, cracked engineers on my team, but [56:21] I'm able to move things forward much faster and cut through the bullshit. [56:26] If AI has done anything for us, canceling meetings would be the gift that I want. Okay, my last question is... [56:35] When AI is not listening to you, when it gives you a really dumb playbook for your engineers, what is your prompting technique? [56:43] It depends on like how many times I've tried to convince it. But generally, it's like... [56:49] okay one you're clearly not listening to me this is what i said two yeah i know i'm absolutely right but like stop being stupid i need your help and three i i like the nuclear option is i threaten it and i say

[57:03] Um, [57:04] Claude, if I'm using Claude Opus 4.5 High, like, okay, I'm going to stop using you, Claude, and I'm going to switch to Gemini. And then it gets its shit together. [57:14] I love it. I don't know what that says about either parenting or management style, but I think it is effective. Well, this has been great. Where can we find you, your team, and how can we be helpful? Yes. So I'm on Twitter, at Chinthan.com. [57:31] Tarakia. We are building the base app. It used to be known as Coinbase Wallet. And I think by the time that when this episode airs, it will be live to the general public. Use it. It is a consumer social app that happens to use crypto and it's enabling creators to earn and be valued. And we're excited to launch it. And we think it's like a real big paradigm shift in crypto consumer apps. So give us a feedback. Give it a shot. Post. [57:59] see the magic happen. [58:01] And we are hiring to Cracked [58:04] front end back end design engineers ml super builders super builders i have two super builders happy to bring in a third one but like it is it is really really fun uh to work here on this team and [58:19] And it'll be awesome. [58:21] So come join us. [58:22] Well, thanks for joining us. [58:24] Thank you. This was such a great way to cap off the week.

[58:37] You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. [58:48] You can see all our episodes and learn more about the show at howiaipod.com. See you next time.