Claude Cowork came out of an accident.
Felix and the Anthropic team noticed something interesting with Claude Code: many users were using it primarily for all kinds of messy knowledge work instead of coding. Even technical builders would use it for lots of non-technical work.
Even more shocking, Claude cowork wrote itself. With a team of humans simply orchestrating multiple claude code instances, the tool was ready after a brief week and a half.
This isn’t Felix’s first rodeo with impactful and playful desktop apps. He’s helped ship the Slack desktop app and is a core maintainer of Electron the open-source software framework used for building cross-platform desktop applications, even putting Windows 95 into an Electron app that runs on macOS, Windows, and Linux.
In this episode, Felix joins us to unpack why execution has suddenly become cheap enough that teams can “just build all the candidates” and why the real frontier in AI products is no longer better chat, but trusted task execution.
He also shares why Anthropic is betting on local-first agent workflows, why skills may matter more than most people realize, and how the hardest questions ahead are about autonomy, safety, portability, and the changing shape of knowledge work itself.
We discuss
* Felix’s path: Slack desktop app, Electron, Windows 95 in JavaScript, and now building Claude Cowork at Anthropic
* What Claude Cowork actually is: a more user-friendly, VM-based version of Claude Code designed to bring agentic workflows to non-terminal-native users
* Why “user-friendly” does not mean “less powerful”: Cowork as a superset product, much like how VS Code initially looked simpler than Visual Studio but became more hackable and extensible
* Anthropic’s prototype-first culture: why Cowork was built in 10 days using many pre-existing internal pieces, and how internal prototypes shaped the final product
* Why execution is getting cheap: the shift from long memos, specs, and debate toward rapidly building multiple candidates and choosing based on reality instead of theory
* The local debate: why Felix thinks Silicon Valley is undervaluing the local computer, and why putting Claude “where you work” is often more powerful
* Why Claude gets its own computer: the VM as both a safety boundary and a capability unlock, letting Claude install tools, run scripts, and work more independently without constant approval
* Safety through sandboxing: why “approve every command” is not a real long-term UX, and how virtual machines create a middle ground between uselessly safe and dangerously autonomous
* How Cowork differs from Claude Code: coding evals vs. knowledge-work evals, different system-prompt tradeoffs, longer planning horizons, and heavier use of planning and clarification tools
* Why skills matter: simple markdown-based instructions as a lightweight abstraction layer for reusable workflows, personalized automation, and portable agent behavior
* Skills vs. MCPs: why Felix is increasingly interested in file-based, text-native interfaces that tell the model what to do, rather than forcing everything through rigid tool schemas
* The portability problem: why personal skills should move across agent products, and the unresolved tension between public reusable workflows and private user-specific context
* Real use cases already happening today: uploading videos, organizing files, handling taxes, managing calendars, debugging internal crashes, analyzing finances, and automating repetitive browser workflows
* Why AI products should work with your existing stack: Anthropic’s bias toward integrating with Chrome, Office, and existing workflows instead of rebuilding every app from scratch
* Computer use one year later: how much better it has gotten, why vision plus browser context is such a superpower, and why letting Claude see the thing it is working on changes everything
* Why many “AI verticals” may get compressed: specialized wrappers may matter in the short term, but better general models and stronger primitives could absorb a lot of narrow use cases
* The future of junior work: Felix’s concerns about entry-level roles, labor-market disruption, and whether AI can compress early-career learning into denser simulated experience
* Why Waterloo grads stand out: internships, shipping experience, and learning how real teams build products versus purely theoretical academic preparation
* The agentic future of the desktop: what it means for Claude to have its own computer, whether AI should act on your machine or a remote one, and how intimacy with personal data changes the product design space
* Why Electron still mattered: shipping Chromium as a controlled rendering stack, the limits of OS-native webviews, and why browser engines remain one of the great software abstractions
* Anthropic’s Labs mentality: wild internal experiments, half-broken future-looking prototypes, and the broader effort to move users from asking questions to delegating increasingly long and valuable tasks
* Why the endgame is not just more capability, but more independence: teaching users to trust AI with bigger scopes of work, for longer durations, with fewer interventions
Felix Rieseberg
* X: https://x.com/felixrieseberg
* LinkedIn: https://www.linkedin.com/in/felixrieseberg
* Website: https://felixrieseberg.com/
Anthropic
* Website: http://anthropic.com
Full Video Pod
Timestamps
00:00 — Cheap execution and building all the candidates00:44 — Intro in the new Kernel studio02:47 — What Claude Cowork is04:18 — Why user-friendly can be more powerful05:33 — How Anthropic built Cowork07:09 — Prototype-first product development08:00 — Why local computers still matter09:20 — Skills, primitives, and platform leverage12:13 — Cowork’s architecture: VM + Chrome + system prompt15:38 — Felix’s own bug-fixing Cowork workflows17:38 — Local-first agents20:16 — Evals, planning, and knowledge-work optimization23:14 — What Anthropic means by evals24:21 — Scaffolding, tools, and why skills matter27:44 — Demo: YouTube uploads and self-generated skills31:03 — Calendar automation and cleaning your desktop34:47 — Browser context and why DOM access matters37:47 — Skills portability and plugins44:36 — Which AI categories survive?46:19 — Junior jobs, simulated work, and labor disruption52:00 — Gradual takeoff vs big-bang takeoff53:42 — Finance, taxes, and enterprise verticals56:24 — Vision and the improvement in computer use57:31 — Why Claude writes its own scripts58:06 — Should Claude have its own computer?1:01:26 — Windows 95 in JavaScript1:03:19 — VM tradeoffs and sandbox design1:07:23 — Approval fatigue and safe delegation1:11:18 — The future of Cowork1:12:27 — What comes next for agentic knowledge work1:15:13 — Electron, Chromium, and desktop software lessons1:22:16 — Multiplayer agents and coworker-to-coworker workflows1:26:05 — Anthropic Labs and closing thoughts
Transcript
Alessio: Hey everyone. Welcome to the Latent Space Podcast, our first one in the new studio. This is Alessio, founder of Kernel Labs, and I’m joined by swyx, editor of Latent Space.
swyx: Yeah, so nice to be here. Thanks to, uh, TJ, Alessio, Allen helping to set everything up. It looks beautiful. We even have the logo outside.
Yeah, kind.
Felix: It’s like really nice, right? When you walk in here as a guest, you’re like, ah, this is a serious production. You’re like, feel it immediately.
swyx: Yeah. Felix, you’ve been, you’re, you’re currently a product manager of Cowork or,
Felix: uh, really Technic
swyx: Eng. Yeah. The, the identities are kind of vague member technical staff.
Felix: I know member staff is like, the official title will carry around forever.
swyx: Yeah. I basically kind of wanted, like we’ve been. Kinda obsessed. I, I’ve been using it a lot, even for managing latent space. Like, uh, cowork helps me upload videos and like title things and like edit and everything. It’s, it’s like really amazing.
Alessio: Cool. He said multiple times Cowork has said gi in the group track.
swyx: Yeah, yeah, yeah. So, so we have a second, uh, we have a second channel, uh, for latent space tv. Uh, and I, uh, and uh, we basically, this is our Discord meetup. Um, and I I, we have like Claude Coworks, it might be a GI, I don’t know if we, we have, uh, uploaded it yet, but one of the sessions was like a, like a Claude cowork thing.
Felix: I, you have to see, I would love to see it. Like, I’m so curious, like one of the most fun parts of my job is like constantly see the weird things people use Cowork for because it’s obviously like very hard for us to actually design for specific use cases we do. But like every single person who’s like most amazed is usually amazed about a thing that I didn’t even expect cowork would be good at.
Um, we have a new designer and it’s one of the first small tasks. I was like, Hey, we need like a new emoji for cowork for our internal stock. It’s like a pretty small thing. I like, can you please do it? And he drew an SVG and just gave it to coworker was like, can you animate this emoji? And now it has like this beautiful loopy animation.
Um, and I mean, I think obviously this goes down to like, it turns out you can do more things with code than you expected, but it, it’s like that kind of stuff that is really fun to me. So, long story short, I would love to see like, the kind of things you’re doing.
swyx: I’ll pull it up. I’ll pull it up.
Felix: Yeah. Yeah.
swyx: Uh, but before we get into it, I, I think always wanna start with like a top level. What is Claude Cowork for people who haven’t heard of it? Haven’t tried it out.
Felix: Okay. Uh, real quick, Claude Cowork is a user friendly version of Claude Code. So the way it basically works is we have Claude Code and for us, fairly impressive agent harness that over December we noticed more and more people are using either, even though they’re not technical, they, they’re not at home in the terminal or they are at home in the terminal, but they started using Claude Code for non-coding workloads, right?
Like managing expenses or like filling out receipts or organizing a knowledge base. Like there was a big obsidian moment that a lot of people liked and we wanted to capitalize on that, but also bring, bring this capability to people who are not terminal native and who might not know how to like brew and store something.
So cowork is Claude Code running in original machine with a little bit of padding, a little bit more guardrails, making it a little safer and a little bit more convenient for people who don’t wanna first open up the terminal when they go to work.
swyx: It’s interesting, uh, that is kind of. Pitch that way as a more user friendly thing because I always feel like it, it, to me, I I treat it as like why I’m familiar with Claude Code.
Like we, we did a Claude Code episode Yeah. A year ago. But this one is like even more power user tools ‘cause it, uh, it kind of integrates much better with like clotting Chrome and, uh, in all the, all the other tooling. But like, maybe, maybe that’s like a perception thing, right? Like
Felix: No, honestly, I don’t think you’re wrong.
This is like a, a thing I’ve been thinking a lot about for like the last two weeks. So,
swyx: but when they say user friendly, it’s like, oh, it’s the dumb down version. But no, actually this is the superset.
Felix: Yeah. Like, I think a similar thing happened, A similar thing happened to me about 10 years ago, like maybe 12 years ago when I was at Microsoft and we started working on, on Electron and like browser-based technologies and cross-platform stuff.
And one of the first use cases was Visual Studio Code, which used to be a website. And the initial narrative was, or Visual Studio Code is, is like a more user-friendly version of Visual Studio. But in a similar vein, I think there was some voices saying, oh, this is. For serious developers, like, we’re not gonna use this.
Right? For like anything. And I think in the end what happened is people have different stories about why Visual Studio Code became such a big thing. But my personal, my personal belief is that the Hackability and the extendability has like played a pretty big role, right? You can hook in Visual Studio Code that like almost any workload, it’s so easy to hack on, so easy to put extensions for it.
And I think cowork might be hitting a similar thing where it’s very easy to extend and it’s very easy to bring into your workflows. Uh, so the convenience I think is a bit of a, it’s obviously the thing we strive for as developers, but I think the way people find value in it then is by probably mapping it onto whatever they actually have to do in their job.
Alessio: So end of last year, you see the spike of like non-technical usage and clock code. What’s the design process to say we should make clock code work? Because I mean, you built it in only 10 days. Um, I’m sure there was some discussion before on whether it’s easier to use mean. You know, like making, making like a desktop GUI is obviously one way to do it, but like there’s a lot of nuance in the product.
Like maybe talk people through what was like the trigger of like, we should build a separate thing. We should not build like a different plot code thing. And then maybe some of the more interesting design decisions that maybe you didn’t take.
Felix: Yeah, I think philanthropic, we’ve been thinking about ways to move people who are comfortable with using Claude to answer questions and bring more of the power of like this thing to now like, execute tasks for you.
I can like solve problems for you can like build things for you. How do we bring that capability to people who are currently mostly comfortable with like a like question answer paradigm within the chat. And we’ve had a lot of prototypes around that. Just going back as far as like easily a year and a half.
Like we had a lot of people working on that. Um, and internally philanthropic is a very prototype demo, first culture. We have a lot of like internal prototypes that don’t reach the public. What Cowork actually became is like we sort of picked the right pieces out of the many prototypes that we had.
Right. And that’s, that’s maybe also like, I think an important qualifier whenever people mention this like 10 day number. I do think it’s important to me to mention that within Double Scratch there was like a lot of stuff already happening, right? Like, and I think it’s important for people to remember that when you build a website, you use React, you use like a bunch of other things.
And this is like a similar scenario with like a lot of pieces we already had. Um, and in terms of decision path, I think we live in like an interesting new world where execution is actually quite cheap.
swyx: Mm-hmm.
Felix: So maybe, maybe what you would do That’s so crazy. The year. I know it’s wild.
swyx: You should be, ideas are cheap.
Execution is the hard part. I
Felix: know. And like the, we, we used to live in this world maybe where you would take a product manager and the product manager would go to a number of potential customers and in this like very low bandwidth way, would try to. Try to like tease out what are the problems they’re having, what are they willing to buy?
Um, and then maybe what can you build to like drive out that need and then you go back and you like draft a spec and you think about it and then like you make a design and you execute it. We internally philanthropic app, not pretty much closer to the point where we’re like, don’t even write a memo, just like build, like let’s build all the candidates very quickly.
Let’s just build all of them and then pick the best ones. I think the, the decision that is most impactful both for the product as well for the users right now is like the way we put value on your local computer. I think that’s a big decision point a lot of people have thought about. Should this thing, whatever it is, should it ultimately run into computer or should it run in the cloud?
‘cause they’re big trade offs, right?
Alessio: I guess like if we solve auth, it would be easy to do in the cloud. But I think like the fact that I can just download any file from anywhere and then put it and cowork there, it’s like a big unlock. Um, I mean it’s interesting you mentioned reusing certain pieces. I think this is something I’ve been thinking about even with Claude Code, right?
The price of like writing code is going to zero, blah, blah, blah. But it actually seems like the value of having some sort of platform substrate is like increasing because as you build these new things, you can kind of plug them together.
Felix: Yeah.
Alessio: So I almost feel like when people are saying, oh, the value of a lot of software is gonna zero because you can recreate it, to me it’s almost like the opposite.
It’s like having an existing platform to build on top of. It’s like even more valuable because you can kind of bolt things on.
Felix: Yeah.
Alessio: You have obviously mcps, you have skills, you have like obviously the models, which is a big part. All these things kind of come together. Do you feel like that’s a valid way to think about it, where people should invest even more in kind of like primitives.
To rebuild on or are you like recreating a lot of it each time because like things change and it’s easier to rewrite than reuse?
Felix: You know, I think, I think you’re right. I think you’re right that the holistic platform is really useful. And this is maybe a whole like a somewhat contrarian view to a lot of people in ai.
I actually don’t think that the future is going to be hyper personalized software down to the point where everyone is running their own version. Like, I actually think it’s going to be quite hard for all of us to have our own internal chat tool and like, if I wanna talk to you, like
swyx: how
Felix: is that gonna work, right?
In the, in the context of cowork and how we build it, I think it’s a bit of a combination. Like what the, the execution that gets cheap is not necessarily rebuilding all the primitives. I think our priori, there’s also not a lot of value in it. So for instance, my team did not think about rebuilding clock code.
We’re like very much started with the. The core thesis of this should be Claude Code.
Mm-hmm.
Felix: And then we’ll like build things on top of it. The part of the execution that gets a little cheaper is like, how do you take all of these Lego pieces and put them together in a way that makes sense for users?
It’s like actually valuable. You have so many different approaches now in terms of what kind of, what kind of things do you actually elevate to a primitive, do you strongly believe that all your products should be built by just combining primitive that the public also has available? Do you keep some things internal?
Um, and I think that’s still evolving, but I think what’s probably gonna go away is like, I’m not sure if it’s gonna fully go away, but I’m gonna say, I think for me personally, I will probably no longer try to come up with a really good product without testing up with people. This is not a new concept, but wherever you used to have to make costly decisions around, do we pick technology A or technology B, or do we like, um, build it this way, build it the other way.
I really strongly believe now you just build all of them and try them out with a small focus group and then whatever, whatever is better is what you go with. Right. And that, that is probably quite different even from how we maybe worked a year ago. Right. Like, I think, I think this happened very recently.
Alessio: Yeah. I started building something in on Electron since you’re here. Coincidence. Uh, but then Electron and like SQL Light are like, there’s like some issues that like between development and like, uh, building anyway. And I was like, let’s just rebuild the whole thing in Swift and just recreated the whole thing in Swift.
And it’s like, I. It’s done.
swyx: You know, I didn’t take any effort. I, I, I don’t even know Swift.
Alessio: Yeah, exactly. I was like, I’m the, I’m not reviewing it anyway, whatever. You can write in whatever language you pick, but the important stuff that I did was not write the electron bindings. Yeah. It was like the logic of what happens in the app, you know, and then the model is like, yeah, I can just recreate the same thing as with
swyx: Yeah.
I, I think you still want, especially for people who are doing like high performance software or like very complex software, uh, you still want like, some view of the architecture. Uh, but you can use markdown for that,
Felix: right? Yeah.
swyx: Uh, you don’t actually have to read the code again. I, I’m still like on a sort of like a definitional thing.
Um, can we build a good mental model of Claude Cowork? Um, this is what I have, right? Like you you said it’s like fundamentally cloud co. We don’t wanna touch it. There’s the cloud app, there’s clouding Chrome. I think you guys do something different in planning, but, uh, I’ve been talking with Tariq who is on the cloud co team, and you guys are, he’s like, no, we just exposed planning.
Maybe we can clarify like, what are the major pieces. That people should be aware. It goes into cowork, like,
Felix: okay, I think you basically have them. So really, um, you can, you can take planning more or less out. I think there’s a few things that are really valuable in cowork. Um, the virtual machine is probably the most powerful thing.
So we currently run like a, we currently run like a lightweight VM and we put clocked out into the vm and we do that for, for, um, a number of reasons. Safety and security is a big one, but even if you, even if you ignore for a second safety and security and you’re just like, okay, Yolo, I want this thing to do whatever.
It is quite powerful to give Claus on computer that is like generally a good idea. And in terms of architecture and UX and everything else that we’ve been working on, philanthropic, it often is quite useful for you to like anthropomorphize, um, clot aggressively and just be like, this is a person. What will you do if you give a, if you had a person, right?
Yeah. And the analogy I’ve given my dad this morning who is still like quite insistent on using chat even for like coding things, is if you were a developer and your employer told you that you don’t need a computer, they’re just gonna like, send you emails with a code and you send emails with code back like that, maybe work for Patrick Miles in the back, but that it’s not very effective.
Um, so what we can do with the VM is because it’s a, it’s a Linux system, Claude Code has more or less free reign to install whatever needs to install. It can install Python, it can install no js. We do have strict network ingress and egress controls. So you can still, as, as a user in like plain human language, make it clear to, to the entire system what you’re okay with and what you’re not okay with.
But at no point do we have to ask a real person, like a, like a person who might be in marketing or a lawyer. I’d have to go to a lawyer and be like, are you okay with me installing Homebrew?
Alessio: Yeah, yeah.
Felix: Right. Because the implications of the question and the answer are complex and nuanced and like, not, not easy to reason about.
This gives us a lot of distraction that makes Cloud very powerful. Now then around it, we, we do probably have a number of things that also keeps growing almost every single week that you’re probably noticing that make cowork maybe better for certain tasks than just cloud. Cloud on its own. Yeah. But most of those actually live in the system prompt.
They’re about like, what can we infer about the work that you do? What can we, what can we intru in the system prompt to make that more effective? It’s of course the like very tight integration with Cloud and Chrome. You’re noticing that a lot of people, especially as the models get better, a lot of people throw up their hands when it comes to MCP connectors in this area.
I’m not gonna, I’m not gonna go through like 25 M CCP connectors, click off everywhere and then like half of them don’t let me do the things anyway. So Cloud and Chrome is quite powerful because we can just talk to the cloud and Chrome sub agent and that will just do things for you.
swyx: Yeah, so, so one example right in MCPI, honestly, I think that the state of MCP is kind of, kind of.
Really hard to integrate. Um, I need to, I needed to add, uh, Figma MCP to the coding agent that I use.
Felix: Yeah.
swyx: Uh, and, but I didn’t wanna read the docs, so I just had caught to it. And it’s, it’s great at reading docs and the same, same way I had to set up like a Google Cloud, um, account for some project I was working on and get some API keys somewhere.
And Google Cloud is famously super hard to navigate, so I just didn’t wanna deal with any of it. I just used Claude Cowork
Felix: within the first week of developing on Core. This happened very, very quickly. Um, I caught myself by starting to use cowork for coding tasks, which is not ostensibly what we built it for, right?
We don’t need to. But I found myself, um, I found myself like on our internal, internal tool that we have for, to collect crashes and just like debugging information and I found myself sort like picking out the ones that I think we can easily fix versus the ones that might be like kernel corruption or something else on the operating system.
And I found myself sort of picking these out and then just telling Clark, go fix this bug. I was like, what am I doing here? Go one level up, tell a cowork, I want you to go to all these crash tools. I want you to find all the bugs that you think are fixable and not like an operating system crash. And then I want you to tell another cloud to like fix all of that.
Um, and that’s, that’s, that’s sort of another cloud,
swyx: just so it can spin up another instance or,
Felix: uh, it, currently what I do is, um, and this is a bit of a hack, but I tell it to use clockwork remote to which website itself? Yeah, that’s interesting. So you basically take, if you, if you imagine like a dashboard with like 20 bucks, you, this is remote control or clock or remote, or, sorry, I just wanted to confirm what, the way I’m using it is.
I have cowork running and I’m telling cowork, here’s where I normally go every morning to find the latest bugs. Go read the entire bug list, separate out which ones are fixable, which ones are, are fixable, and then for the fixable ones, four is this almost loop. For each bug, write a markdown file with a prompt.
And then for each markdown v, that is a prompt. Start of a cloud set. So natively Claude Code has
swyx: this concept of subagents. Mm-hmm. And this is basically a subagent, but you’re not using the subagent functionality.
Felix: I’m not using the subagent functionality. And the reason I’m not is because I’m firing that off as a Claude Code remote
swyx: task.
Felix: Yes. That’s kind of nice. ‘cause then I can just fire it off. I can go to my next meeting and in Claude Code remote. Now the work is happening.
swyx: Mm-hmm. Yeah. You, you see like you’re already starting to use the cloud over your local machine. And I think this is one of those things where like. Shouldn’t just everything just be cloud first, right?
Felix: Ah, this is such a good group. I’m like solely bad about this. I have so many thoughts about that. Okay. So I generally believe that Silicon Valley overall is undervaluing the local computer. And my default argument for that is always how come we’re all using MacBooks and not like an iPad or a Chromebook?
Um, that there is like still value in, in having a local machine. And now when I think about Clot, it’s this entity that is supposed to be very useful to you, like it tremendously useful to you. I think that entity needs to have access to all the same tools you have access to. Otherwise it’s gonna be hamstrung in like all these complex ways.
And there’s, there’s sort of two approaches we could take. We could say, okay, we’re gonna like one by one chip away at everything that is at your computer and move it into the cloud. That’s, that’s one way to do it. Um, and I think other products have taken that path. I personally, this is a very personal opinion, but I personally, for the amount of tools that I use.
Just don’t have the patience to give another tool like permissions to every single thing and keep those permissions up to date. The second thing that I’m still grappling with, and I don’t have a good answer for anyone just yet, but the second thing I’m still grappling with is what does it look like for someone to slurp up your entire work and put that in the cloud?
Like if I, just as an example, like if you could click a button and it just clone your entire computer into the cloud, is that something that you would want? I’m not totally convinced yet that all everyone will. Mm-hmm. And that is sort of like upstream of all the technical issues we’re gonna have. ‘cause like in general, I think the world is not ready for this kind of stuff.
Like, I’ll give you one quick example that would probably be very easy for us. So as a desktop app, we in theory with your permission, can do a lot of things on your computer, including reading your Chrome cookies. If we really want to do right, we could take your Chrome cookies, you would have to decrypt them for us.
We could put those on the cloud if we really felt like it. Pretty easy solution. That would be super cool. We could just be like, oh, we can do all your tasks in the cloud now. Um, a lot of websites, thanks, include it. If, if they see the same authentication from like two different locations, we’ll just lock down your account and now you have to go to the branch and be like, okay, I, I’m here with my passport.
You actually know that. Wow. Yeah. As tired as well are of the term agent for the age agent future, I think there’s a lot of stuff that sort of slowly needs to catch up and until that’s the case, the way I, as someone’s working on clock and make Cloud most effective is to like put it where you are working.
swyx: Anything else? I thought with our mental model, so like, basically like, uh, part of me also just want, like the more I understand how it works, the more I can use it to its full potential. Right?
Felix: Yeah.
swyx: And so what I’m get hearing from you is you told me to delete the planning thing. You’re not doing anything special on, on the, that’s only exclusive to Qua cowork.
Felix: We have some tricks for this sort of like change week over week. We eval cowork maybe against different use cases than he would evil clock code, right? If you think about it this way. Okay, so like clock code is our eval clock cowork. Yeah. So clock code is like quite optimized for coding tasks and we mostly value it whether or not we’re getting better or worse depending on how good it is at like a typical suite job.
And Clark Cowork on the other hand, we evaluate more against typical knowledge work, the kind of stuff he would find in finance or in like maybe a, like in like a legal office. Um, my personal use case is always like managing my things, like managing my personal mortgage or something like that, right? Or like wealth planning for me and my family.
Those are the kinds of use cases we eval, clock cowork on. And what you might be picking up on is like the subtle changes we make to the system. Prompt what we put in the system, prompt how we steer, clot with the tools we give it. Um, like either it’d be better in one or the other direction and whether there’s a trade off, try us exist a lot.
CLO code will be better of a code and Claude Cowork will be better. For non-coding tasks, will those gaps still exist in the next three generations of models? It’s like a little unclear to me though.
swyx: Yeah,
Felix: because right now these like hyper optimizations we make, I’m not sure for how long they’re still be relevant.
swyx: I think what I was referring to was also, it, it just, uh, it qualitatively felt different when I probably, it’s just all prompting and I’m reading too much into it, but like the, the fact that it comes out with like a nine step plan, I can edit the plan and give feedback and, and, and see it execute the plan.
Yeah. It felt more long range than in Claude Code, but maybe that already existed in Claude Code and you just build a nicer UI for it.
Felix: It’s kind of both. Um, like if the Clark Code people who build the planning functionalities would city, they probably say yes, we have all of those things in Clark code and they do.
Um, I think people tend to give cowork. Tasks that are maybe of longer time horizon, I thought is
swyx: so long. Yeah.
Felix: That’s like one thing, right? It’s just like that the, the chunk of work tends to be maybe a little bigger. And then the second thing is that because the work, when it gets longer, it gets a little bit more ambiguous.
We do tell co-work to make heavy use of the planning tool or to make heavy use of the ask user question tool, right? We do want it to come up with like. Different scenarios of, okay, tease out what the user actually wants. Don’t go off to work for like four hours and then come back with the wrong thing.
And you’re probably picking up on that.
swyx: Yeah.
Felix: Um, I wish I could tell you I like built this magical thing and it’s like, there’s some secret sauce,
swyx: but No, no, no. I mean, it’s, it’s just clarity is good that, you know, engineers just want to know. Yeah. They can, they can plan around it. And then I think also for me, um, I am realizing I have to switch to my, my other machine because this is a new machine that doesn’t have my session.
But, uh, yeah, the, the, the planning is really important for, for me to like approve or like to see whether it’s like, it’s right. The ask is, the question is so beautifully presented. I mean, it also, it also available in like cursor and, and in Claude Code. But like, I, I think like it’s so nice to see that it, like it’s kind of for me like to understand that it gets me, it gets what I want to do.
Felix: Yeah.
swyx: Yeah.
Felix: It probably very hard
swyx: just on the topical evals. Mm-hmm. When you say eval, I think people are very vague about what it means. Is it just like vibe testing or do you have like automated programmatic evals of Claude Cowork?
Felix: When we say eval, uh, what we really mean is that we essentially take the entire transcript, including all the tools that clot has available ultimately to it, and we then measure what are the outputs, depending on what we tweak, right?
So we do run that a lot. We use that in training. Um, we use that in, in like, if you sort of separate out post training from like the scaffolding around it. Cowork sort of exists in the scaffolding space, but obviously we also train on it a little bit. Um, so when we say eval, we mean given the certain transcript, what do the outputs look like?
Including the file outputs as well as like the actual token outputs, like the ones that you see in the chat window.
Alessio: I’m curious, um, how much of the failure modes are the model intelligence versus like the usage of the end tool to put the intelligence in? Like the well planning is like a good example, right?
It’s like one thing is to come up with a plan. The other thing is like make a nice spreadsheet. Yeah. That kind of runs you through the plan. Like how have you seen that? Well,
Felix: the thing that I grapple with a lot is that whatever scaffolding you come up with, I think we still have a bit of sort of like model overhang where the model is dramatically more capable than right.
Users end up using it for. And I think part of that is that we’re just not getting the model all the tools to do all the things that’s theory capable of, right? There’s like one thing, um, however, whenever you do build the scaffolding, I’m sort of wondering at what point, at what point will that scaffolding go away and like how much you invest in figuring out what the right scaffolding is.
It’s kind of up to, it’s a little bit of a bet. And one thing that I as an NJ quite enjoy is that like working in philanthropic and working at a frontier lab, I maybe have a little bit more insight into what’s coming, coming down the chute in terms of like, what’s the next model, what is the model capable of?
What is good at, what is it bad at? And I’m, I’m increasingly wondering, is the right thing for us to like really invest too much in sort of these like scaffolding corrections where the model might otherwise not misbehave, but just not do the thing that you want?
Alessio: Yeah.
Felix: Or is it to just like give it as many capabilities as possible, try to make those safe so there’s the worst case scenarios, likeno status might be otherwise.
And then just simply wait a second for the next model drop. I’m personally, currently more leaning into the ladder. I think we’re gonna see a lot of like applications and companies that do very impressive things with ai that in the short term might seem very effective ‘cause they’re very specialized to individual use cases.
But I think once models get better generalization and get better at like those specific use cases without being super guided on those, I’m not sure how long that’s gonna stick around. And you can kind of, kind of already see this in like skills and NCP servers, right? Mm-hmm. We’ve, we’ve already seen sort of this like slow shift from MCP service to skills.
And like, maybe a good example is Barry who made skills. He was initially hacking on something that honestly looked a lot, looked, looked a lot like what Cowork does today. It was sort of thinking about what if cowork, but for like people who don’t wanna build code. Mm-hmm. And, um, he too did that as a prototype inside the desktop app.
One of the first use cases we thought of were, okay, what, what are like coding like use cases that could really benefit from graphical interfaces and like from being a little separated from the actual underlying code. And everyone comes with the same answers. Data analysis,
Alessio: right?
Felix: Yeah. Or saying how many users do we have today?
How many, like, it’s always data analysis. And I think the thing that ultimately led to skills is that we wanted to connect this little prototype to our data warehouse and. The team very quickly discovered that like instead of building a custom tool for the thing to talk our data warehouse, they just like meet and embarked on follow like mm-hmm.
Dear Claude, if you want to get data, here’s the end point. Here’s what the API looks like. You’ll figure it out.
swyx: Ah.
Felix: And then it be hand over control. Yeah, yeah. Also just like maybe go one step up in the layer of abstractions, right. Just, yeah. Instead of, instead of telling the thing, here’s ACL I, please call the CLI, or here’s an MCP.
Please call this ECT shape. Just like this is the end point. If you wanna know something, if you post here, maybe you can do post sql. It’s gonna be okay. And that ended up being so effective that they started trying the same pattern of like just giving the model a markdown file that describes whatever it needs to do.
That the whole thing eventually became skills and we’re like. We should package this up. This is a good idea.
swyx: Yeah. Um, we’ve had Barry Mahesh, uh, on, on our conference and uh, he’s uh, definitely got a good idea there.
Felix: Yeah.
swyx: I wanted to show you the, how I’ve been using Claude Cowork.
Felix: Uh, this is was my favorite part.
swyx: This is this. So this is like me, uh, this is how we run the Discord. Uh, we literally, uh, at first I didn’t trust Cloud Core. This was my very first usage.
Felix: Okay.
swyx: Right. So then I was like, okay, I will just try to manually download from Zoom all my recordings and upload it to YouTube. Yeah. Because this is a very laborious process.
I got a click, click, click YouTube, um, isn’t super user friendly. Uh, and it just did it. And then I was like, actually, you know, even the download from Zoom part, I should also. Put into Claude Cowork, and then I did it right. Here’s a bunch of, and it starts compacting here, and it, and it, it starts to even be able to do things like look through the individual frames of the video to name the video so I can upload it auto automatically.
Oh, that is, and this replaces my job as a YouTuber. We will forever appreciate your creative Yes. You know, and so that’s great. Uh, but then by the way, it compacts and makes, makes like a new thing, right? So I, I don’t, I don’t have the initial, initial thing, but then I asked it to make its own skills so that it, so that something that’s repetitive and one-off and human guided becomes more automated and I can use the skills independently and reuse them.
Uh, and it obviously you can write skills and that goes into context and skills at the bottom here, which is, which is so nice. Um, so I have all these skills that, that I now sort of do on a weekly basis. Uh, I know you’ve released scheduled Coworks, which I haven’t done yet, but
Felix: course I should try them. I, I think this is like so wonderful and fun for me to see because.
One thing that is very fun for me about skills in particular is that they’re so easy to make. Like anyone can make a skill, like a text message, could be a skill, and they can be so hyper personalized to you. And this is like sort of the subtraction layer, right? Like, um, I, I’m just guessing, but I assume, heck, you are very good at your job.
You’re probably given this thing some guidance about how to do it, right? I,
swyx: I just said, wrap everything up into, into a skill, right?
Felix: Yeah.
swyx: And then, uh, and then I was like, actually, sometimes I might need to break, uh, things apart because some parts fail or some parts might be needed in individually. So I told it to split one skill into three skills.
So it’s like a skill splitting thing, and then there’s like a parent skill that just orchestrates all of them if I want to use that. You know, like, um, I think that’s, that’s like really good. Uh, and, and, uh, there’s, there’s one more part, which is the, uh, Google Chrome thing that I told you about.
Felix: Yeah.
swyx: Where I’m like, okay, you know, what’s better than uploading, using Claude Coworks to YouTube?
Like actually. Looking at the docs to like programmatically upload to YouTube and then putting that in a skill. And I’ve never done that before. I don’t want to deal with Google Cloud. Yeah. So Claude Cowork does it for me.
Felix: That is really cool.
swyx: So, so I, I just, I don’t care. I just, like, I do a thing. I don’t, it doesn’t really matter.
Felix: That is really cool. And then you’ve, I assume paired the skill just with the script that it’s built.
swyx: Yeah, no, I just update, update the skills.
Felix: Oh, that is beautiful. Yeah. That’s wonderful.
swyx: It’s kind of like a skill, like, uh, uh, basically I think like the way that people ease into Claude Cowork is like take a knowledge work task that you would normally be clicking around for and then, uh, try to turn, turn that, and then you do the, okay, well what if you went further?
Okay. And then when, if you went further, when, if you, and it sort of expand the scope of cowork as you gain trust with it and, and also teach it how to replace you.
Felix: Yeah. It’s like a little bit like playing factorial, but for your own life. Uh, like you say, you start really small.
swyx: Yeah.
Felix: You start automating something really tiny and like.
Once it clicks, you keep adding onto this like automation empire. Just like make your life easier and easier. My favorite skill has been, um, every single morning Kohlberg starts looking at my calendar and make sure that there’s conflicts because people tend to schedule a lot of meetings, sometimes last minute, sometimes miss it soft and painful.
And a lot of products have existed like that A lot. I’ve written in the custom prompt there. I haven’t made it a skill, um, honestly should.
swyx: Yeah.
Felix: But I’ve given it like pretty clear instructions about okay, here are some people, if they book over other meetings, I’m probably gonna go to their meeting. Like if Dario schedules a meeting.
swyx: Right.
Felix: Not try to reschedule down. Right. Um, and I think there’s some other rules in there about like what kind of meetings I care more about what kind of meetings I care less about. What is okay to like, maybe pun like when I want to be, when I want to be working, when I don’t want to be working. And it’s those really small things that I can think kind of click with people.
Right. When we launch co-work, I think one of the US races that went most viral on Twitter. X was clean up your desktop, which is stuff, because silly, that’s such a smart thing, right? Like you don’t need to model to clean up your desktop. Not really. Um,
swyx: like this, like clean up my desktop.
Felix: Yeah, exactly. Yeah.
swyx: I need to, I need to choose my desktop, right? I guess give it access to my desktop.
Felix: Yeah.
swyx: Okay. Uh, okay. This is very scary. Oh, we’ll do it.
Alessio: I did, I did it with my downloads folder. It was like, you have so many term sheets and there’s like eight copies of your rental lease for your office. I was like, all right.
Like, don’t yell at me.
Felix: It’s like, it’s not such a small task. And then like, I, I would never go out there and normally otherwise and tell people I’ve pulled a product. It can organize your folder. Right. Um, because it feels small. But I think to your point like,
swyx: oh, here’s, here’s the, here’s the ask user questions.
Felix: Yeah.
swyx: Uh,
Felix: beautiful. Right. Elite obvious junk. You probably shouldn’t click that.
Alessio: No.
Felix: If he’s not done right.
swyx: As long as it’s reversible, I don’t
Alessio: make up blend to,
swyx: yeah. Uh, yeah. No, I, I have a, I have a typical, everything is super messy folder. So, yes. I think this, this is super helpful. So this is a pretty simple task.
Mm-hmm. But I’ve, okay, here it is. Right. Here’s the progress. I don’t see this in, that’s why I’m like, this gotta be something different than, uh, than Claude Code, because I’m like, we
Felix: do. Yeah. That’s, we do system prompt that. We’re like, all right. We want you to think about like, this task Yeah. Methodology.
Yeah.
swyx: And then I can, I can, I can do like little suggestions for, for, for these things. It’s beautiful. Look at this. I, I can, I can like say like, oh, don’t do that. Don’t do this. It’s amazing.
Felix: I’m so happy. You like it. Um, I mean, the other way around, like we’re part of the Clark core team, if you would like this in Clark COVID.
swyx: Yeah. Yeah. Yeah. Uh, so, so yeah, I mean, uh, this is really good. Obviously I, I’m like kind of raving about it. Uh, you know, I have other things like sign up for pg e so if you can do phone calls for me, that’d be great. Um, I, I do, people
Felix: have done that. Obviously you can’t do that natively, but people have done that with like, various other providers.
swyx: Yeah. Uh, and then this is like signing up for the Figma MCP. Um, I, I really am trying to do like everything, um, data analysis as well. I do think, um, oh, design to code, uh, very, very good. Right? So like, here’s a Figma file, take it. And then this is where like a lot of other tasks is like knowledge work, like replace my manual clicking, but this is no, I would normally use Claude Code or uh, Claude Code for this, but because I perceive that you have better Chrome integration
Felix: mm-hmm.
swyx: I, I think you can actually do a better job of this. And I, this, this is one shot at my, uh, conference website.
Felix: That’s pretty cool. Like at some point I would love to like, hear how you feel about code. In the desktop apps, which is like I never use, which is the, the same team. Same team.
swyx: So I use the call code in terminal, which I, I perceive to be the default way of cloud coding.
Felix: So one thing this has,
swyx: sorry, I’m just like, I’m not
Felix: here, I’m not here. All products. Can I talk about other stuff? Like I, I’m not sure if people out there wanna like hear me advertise my stuff for like an hour. Please do that. Um, this thing is like a builtin browser, which is a thing a lot of products have said.
Yeah, it’s a builtin browser. And I think giving cloud eyes into like what you’re actually working on makes it so much more effective. And that’s probably what you’ve seen in cohort because it can see Chrome, it can like debug the dom, it can like see things. Um, that does make it more powerful.
swyx: Yeah. So, so I think, uh, my mental model was kind broken.
‘cause I only use this cowork because I thought it had a, a browser thing in it. But I understand that the Claude Code app. The app version of Claude Code does have a built-in browser. I’ve seen, I’ve seen this preview thing.
Felix: Yeah.
swyx: I just, I’ve never used it.
Felix: But in the end, in the end, you sort of have it by hard.
Yeah. You basically get the same thing. Right? Like the, the, the additional skill that you’re describing is chart is better if we can see what it’s working on. Right. That’s, that’s sort of like the summary here and like whether it’s using your Chrome
swyx: Yeah.
Felix: Or it’s just like making up its own little like browser.
It doesn’t really make a big difference because either way it’s gonna see what it’s working on and that just makes it much better. And then you don’t have to run QA for your cloud.
swyx: Why doesn’t it pick up my existing Claude Code sessions? ‘cause I, I mean, obviously I’ve used Claude Code, but Excellent question.
Um, don’t have a good answer other than like, we’re honest. Just haven’t Yeah. This is what the Open AI team does. Okay. Uh, cool. I I I don’t have other, like, I, I just, I, I do wanna expand people’s minds and also maybe show people if they haven’t really done it, but like, I, I think it’s very interesting how I sometimes use this more than I use, I mean, I use dia, right?
Yeah. Um, I, and I use, uh, I’ve used like all the other agentic browsers and philanthropic didn’t have to build an agentic browser because you just had Claude Cowork and that’s enough.
Felix: Yeah. I also think like maybe integrating with number of excellent browsers out there, it’s like currently on my personal priority list, a little higher than like trying to rebuild a browser from scratch.
Yeah. You know, never say never, but I think going back to this idea of like, we wanna plug this into an entire existing workflow, I think our goal is actually to not replace any of the applications we have in your computer. But instead of like, work really well within a new workflow,
Alessio: make the new one. Yeah.
Are, it seems that nowadays, especially on the browser, most of the innovation is like user ergonomics. It’s not really like the underlying browser engine. So I feel like to call it, it doesn’t really matter if it’s like the, uh, or Chrome or Alice, whatever.
Felix: Yeah. We wanna, we wanna meet you wherever you are.
Which is like, like obviously I would say that, but it’s also just generally true because I don’t wanna shrink my potential user base artificially by saying, okay, like, I’m gonna start building for the people who are willing to switch browsers.
Alessio: Right.
Felix: That’s such a, like, you know, like many lawsuits have been filed over who gets to review the browser and like a lot of money has switched hands over the question of like, which browser is default and which search engine is default within the browser.
Um, I just wanna build for, yeah, I wanna build for swyx essentially. Like, I wanna, I wanna, I wanna build for people who have a number of annoying tasks that they feel like. Maybe clock could do it. Could do it for them.
Alessio: Yeah. What do you think about skills portability? I think there’s been one thing, I use another thing called zo, which is kinda like a cloud computer plus agent.
And I have a skill to add visitors to the office. Yeah. So whenever somebody has to come in after hours, they need to check in downstairs. Um, but I wanna like text the thing, so it doesn’t really work in, in cowork, but now that skill is in the zone harness and it’s not in my cowork thing. And then if I make a change, it’s gotta, I gotta sync them.
How do you see that going? Like I see memory as like. Cloud personal, kinda like, I don’t necessarily want my memories to be cross thing.
Felix: Yeah.
Alessio: But I do want my skills to be cross agent that I use. I think with MTPs, people do the same thing. It’s like, oh, Mt. P Gateway. Mt P registry. I don’t really know if that’s like a business.
So I’m curious like if you’ve had any thoughts in the area.
Felix: I think for me, this is sort of where I go back to the really basic primitives for our skills are file-based instead of like this complicated thing that exists inside a place somewhere that is like super proprietary. I’m really leaning into the idea of like, it’s all just files and vultures, and that makes it very portable on its own.
Right. We do have skills as part of this container format, which was just called plugins.
Alessio: Mm-hmm.
Felix: And plugins are available both for Claude Code and Claude Code work the same format, and you can install plugins. This works in cowork today. You can basically say, I’m gonna add a whole, like just a GitHub repo as a.
Skills marketplace or like a plugin marketplace. And that’s how we’re doing portability. I think we have a lot of room left to grow in. How do we make it easy for people to know that they can write skills? How do we make it easy for them to just like, share a skill with you? Because obviously all the words I just said, right?
Like I’m losing most of the knowledge worker base out there, right. And start by saying, oh, you can connect to GitHub repo. It’s not exactly how most people will end up working in like a general knowledge worker space. Um, but I think there’s something there. And another thing that’s there that I think has not really been properly explored is the, the, the combination of which part of the skill is very portable and then which part of the skill is like very personal to you.
Right. And I think that’s something we haven’t really solved as an industry. Hmm.
swyx: It’s like, which, how you wanna introduce more structure to the skill or have always have like. Public skill, private skill, you know, pair. Yeah, yeah. Kind of. I think there’s
Felix: like a, like the easiest way to do this, which is we do like use string interpolation or something.
Right, right. Yeah, yeah. Insert username here, insert like phone number, insert, like known folder, locations, that kind of stuff. Um, that’s probably clunky. That’s why we haven’t built it. Um, but I do think someone is going to come up with like an interesting way to keep everything we like about skills. The portability is just a file, it’s just marked down.
It’s just text, honestly. Right. Like a text file words. The complete lack of structure, which means you don’t need any kind of tutorial to write a skill. Just like explain it to Claude the way he would explain it to me and Claude will probably get it before I work. Mm-hmm. Right? You’re just like, for booking a flight, tell Claude how to book a flight the same way we tell him somewhere.
I just started working here today. But combine that with a very like, personal thing. Um, maybe we’ll stick with a booking a flight example. I don’t actually think. AI should be booking flights. I think the tools we have is yes.
swyx: Yeah. Finally, somebody says it. It’s the default demo that everyone’s making.
Felix: I’m
swyx: like, I even against like booking demos, it is not a good showcase.
Felix: Yeah. I’m like, I just wanna book my flight myself. But, um, I think there’s a lot of things that have a personal and a non-personal component and that’s maybe why people reach for flight booking because some things are very universal. Yeah. Super flight is usually better, right? Like few people try to book the most expensive flight.
And then some things are quite personal about like what times you prefer, which seat you prefer, which airports you prefer. Combining that and like a skill format that is actually portable, compatible, easy to understand for people. I think that would be very exciting. We just haven’t figured it out yet.
Alessio: Yeah, I think the text part every, I think everybody by now has some sort of like cloud file thing. Either Dropbox, Google Drive, whatever. So it feels like in a way it should basically like sim link. My skills into all my agent harnesses. Yeah. Just keep those ing like we have internally this like valuable tokens repo, which is like all the commands sub agents.
It’s good. Uh, and then I build like a TUI where you can start it and be like, you know, install this command and this three sub agents into this agent in this folder and just copy paste this. It doesn’t do anything. It literally cp the file into that. But I feel like there should be something similar where like whenever I go into a new thing, it’s like, hey, here’s like the link to exactly the cloud folder and just bring down these skills into this.
Yeah. Like today it doesn’t quite work like that. Like if I install a new agent, I cannot, I have to like copy paste all the skills and I don’t even know where they are.
Felix: Yeah.
Alessio: That’s like the big problem. It’s like where do I find them?
Felix: Yeah.
... [Content truncated due to size limits]