Since server actions became stable, I've watched teams flail between "use them for everything" and "they're a leaky abstraction, avoid." Neither is right. Here's the line I draw in my own code.
Use server actions for
Form mutations triggered by your own UI. This is what they're built for. Progressive enhancement works, revalidation is tied to the mutation, and the type safety between server and client is automatic. There is no better option for this case.
Simple optimistic updates. Paired with useOptimistic, server actions give you the happy path of mutations with rollback handling for free. Implementing the same shape over an API route is three times the code.
Authenticated user-scoped writes. Because actions run in the same request context as the page, session/cookie access is a direct read. No token passing, no header plumbing.
Use API routes for
Anything a third party will call. External consumers — mobile apps, other services, webhooks — need a stable, documented URL. Server actions don't give you that. They're an internal RPC, not a public API.
File uploads beyond trivial sizes. Server actions work for small uploads but streaming large payloads is simpler over a route with direct access to the request stream.
Anything that needs to be cached at the CDN layer. Routes are addressable. Actions are a POST that can't be cached.
Webhook receivers. These are inherently public URLs from an external system. They belong on a route.
The overlap zone
Reads triggered from client components are ambiguous. You can use a server action, but a server component or a direct RSC fetch is usually cleaner. If you reach for an action to do a read, ask whether the component could be a server component instead. Usually it can be.
The real failure mode
The mistake I see most is using server actions as a generic RPC mechanism — passing arbitrary shapes, building abstractions over them, layering validation and error handling until you've rebuilt a worse version of tRPC. When that happens, stop. You wanted routes. Use routes.
Server actions are a mutation primitive with great ergonomics inside a closed Next.js app. They're not a replacement for a public API layer. Keeping those two ideas separate saves you from fighting the framework later.