My team is in the agentic orchestrator phase. I like this software factory pattern in concept, but our biggest challenges in development are acceptance testing of anything UI-related. Mobile app testing in particular is still a huge bottleneck that requires a human. AI models really suck at identifying poor usability and jank, particularly because they only typically process snapshots of the app from an instance in time.
I know there are traditional testing frameworks that can detect jitter and frame drop to a certain level. We could potentially start having agents build that in.
If we had concrete designs and specs on every project, that would also be helpful, but in a fast-moving startup, that gets delegated to the builders. That puts a human back in the loop every time.
Curious to hear what anyone else does to fully adopt a software factory pattern.
I totally agree with this. Agents seem tremendously bad at UI to me. Maybe it's just because I am a back end guy.
Right now I'm working on a declarative UI framework which can help me along here. My thought is that if I sacrifice a little control for sane primitives, that will make that spec /build loop easier.
I think ClayUI is a really interesting "reduced instruction set" for UI. I don't know that immediate mode UI is the right call for anything web related (that's how you get React lol) but his reduced primitive layer is very interesting to me
Are there known examples of software where the "software factory pattern" (an incorrect term, given that it's not a pattern but a workflow. Or rather an idea. A hope. A wish.) proved to work long-term?
Preferably ones that I could validate myself instead of just having to take someone's word for it.
I don't believe the "software factory pattern" can even be established as a pattern. I wonder if it's considered normal these days to call a Silicon Valley management style a "pattern."
Fundamentally, a pattern is a reusable solution to a recurring problem. But what the author is describing here is simply an operational loop and a management strategy.
I think current agent methodologies are practically indistinguishable from human developer management theories. Isn't this just a feedback process? I would consider this an agentic workflow.
Also, people casually use the metaphor of a "factory" when churning things out, but a factory fundamentally operates on "orders"—it has a specific objective and a target production volume. The entire approach of just trying to dynamically respond to everything under this label feels somewhat contrived.
Furthermore, I always have this underlying question: why are we building software factories in the first place? Where are we manufacturing the people who will actually buy all this?
I've heard stories of people finding success by running "app factories" in the early 2010s when apps were scarce. But in the agent era, I believe we are already drowning in AI slop. We have to remember that back then, producer friction was high, and simply submitting an app was a difficult hurdle.
Now, AI handles most of the basics by default. Things that were once highly praised are now just the baseline. To create something actually worth selling today, you have to break the existing grammar entirely, build much more complex architectures, and offer deeper features. Given this new standard, I seriously question whether mass production is the right answer.
Furthermore, I always have this underlying question: why are we building software factories in the first place? Where are we manufacturing the people who will actually buy all this?
You say this but then
Now, AI handles most of the basics by default. Things that were once highly praised are now just the baseline. To create something actually worth selling today, you have to break the existing grammar entirely, build much more complex architectures, and offer deeper features. Given this new standard, I seriously question whether mass production is the right answer.
To do something much more complicated, you need to rework the fundamental ways of working which is exactly what the author is proposing. I think you are already sold that this is the base line (but it doesn't seem like it based on your post).
Will has trouble with this in a relatively nimble org, where he has good control of what is adopted. Imagine how much fun this is in more ossified enterprise environments, where getting work done efficiently means bypassing process with dev lead and line management blessings. The differences in performance among individuals has never been wider, and not just gains in productivity, but performance losses as some people who have iffy judgement cause trouble a lot faster.
Maintaining quality when an organization just lacks the muscle to make changes and nobody has the mandate to try to make sure uptime is in good shape is a challenge. A lot of things break, precisely because there's just as much change as Will sees, but it's very unevenly distriuted.
Has anyone demonstrably gained market share over a competitor who is vocally _not_ adopting agentic software development flows? This article outlines a lot of process churn without a clear through line to how it is impacting feature development or revenue.
vocally _not_ adopting agentic software development flows
I suspect that the vendors out there who have not adopted AI development and are losing market share to competitor who has are not vocal about not being an adopter, they are just complacent.
Writing about these things in public and putting yourself out there is greatly appreciated. Hats off for that!
However, as far as what's being pursued, it seems more like wantonly trying to ride a hype cycle without strongly questioning the end-to-end value of new software development approaches or vetting their immediate suitability.
Personally, I think it would be more sensible to take a few individuals or a smaller team(s) and do more isolated/skunkworks experimentation and adopt as justified based on what those people report/experience. The smaller group can adjust faster and iterate/advise the larger dev org about the good approaches/techniques/strategies, and avoid more broad damage/chaos for things that aren't that well thought out.
For more conservative AI use cases like adding to code review, writing low stakes PR summaries, or beefing up security checking, a more global, but still not off-the-rails, approach would be the kinds of things that would make more sense to push more broadly.
I find it a bit suspicious that they don't talk about the amount of human intervention required (apart from writing the RFC, which itself can be done by the agent).
I think this can work if you've set up your codebase with proper AGENTS.md with all the guarantees and low level design you expect. I think some human intervention is required here so that you keep the codebase maintainable - you as the human know the domain and future plan well so your addition is valuable.
I also think its necessary to have some garbage collection - scheduled jobs that look at the codebase and think of ways to tighten it and come up with better design.
Lastly, even if one doesn't take away the full factory, I still think that deployments should be fully automated. In the companies I have seen, deployments are still a cognitive burden - one must look at 100 different dashboards and test in staging, look at logs and so on. This is something the agent can do very well and is best automated. Very few orgs have done this!
Why would they use Notion in a system like this?
Seems like setting yourself up to have a massive pile of janky cruft.
managers that love notion are the cruft you're speaking of
My team is in the agentic orchestrator phase. I like this software factory pattern in concept, but our biggest challenges in development are acceptance testing of anything UI-related. Mobile app testing in particular is still a huge bottleneck that requires a human. AI models really suck at identifying poor usability and jank, particularly because they only typically process snapshots of the app from an instance in time.
I know there are traditional testing frameworks that can detect jitter and frame drop to a certain level. We could potentially start having agents build that in.
If we had concrete designs and specs on every project, that would also be helpful, but in a fast-moving startup, that gets delegated to the builders. That puts a human back in the loop every time.
Curious to hear what anyone else does to fully adopt a software factory pattern.
If your UI is created for humans to use, you should still have humans involved in testing it.
I totally agree with this. Agents seem tremendously bad at UI to me. Maybe it's just because I am a back end guy.
Right now I'm working on a declarative UI framework which can help me along here. My thought is that if I sacrifice a little control for sane primitives, that will make that spec /build loop easier.
I think ClayUI is a really interesting "reduced instruction set" for UI. I don't know that immediate mode UI is the right call for anything web related (that's how you get React lol) but his reduced primitive layer is very interesting to me
Missed the part in July when the token bill starts rolling in
May I ask what you’re making that somehow is improved by turning everyone into chat bot controllers?
Are there known examples of software where the "software factory pattern" (an incorrect term, given that it's not a pattern but a workflow. Or rather an idea. A hope. A wish.) proved to work long-term?
Preferably ones that I could validate myself instead of just having to take someone's word for it.
It’s all experiments at this point
Have you produced anything shippable AND maintainable?
I don't believe the "software factory pattern" can even be established as a pattern. I wonder if it's considered normal these days to call a Silicon Valley management style a "pattern."
Fundamentally, a pattern is a reusable solution to a recurring problem. But what the author is describing here is simply an operational loop and a management strategy.
I think current agent methodologies are practically indistinguishable from human developer management theories. Isn't this just a feedback process? I would consider this an agentic workflow.
Also, people casually use the metaphor of a "factory" when churning things out, but a factory fundamentally operates on "orders"—it has a specific objective and a target production volume. The entire approach of just trying to dynamically respond to everything under this label feels somewhat contrived.
Furthermore, I always have this underlying question: why are we building software factories in the first place? Where are we manufacturing the people who will actually buy all this?
I've heard stories of people finding success by running "app factories" in the early 2010s when apps were scarce. But in the agent era, I believe we are already drowning in AI slop. We have to remember that back then, producer friction was high, and simply submitting an app was a difficult hurdle.
Now, AI handles most of the basics by default. Things that were once highly praised are now just the baseline. To create something actually worth selling today, you have to break the existing grammar entirely, build much more complex architectures, and offer deeper features. Given this new standard, I seriously question whether mass production is the right answer.
You say this but then
To do something much more complicated, you need to rework the fundamental ways of working which is exactly what the author is proposing. I think you are already sold that this is the base line (but it doesn't seem like it based on your post).
Will has trouble with this in a relatively nimble org, where he has good control of what is adopted. Imagine how much fun this is in more ossified enterprise environments, where getting work done efficiently means bypassing process with dev lead and line management blessings. The differences in performance among individuals has never been wider, and not just gains in productivity, but performance losses as some people who have iffy judgement cause trouble a lot faster.
Maintaining quality when an organization just lacks the muscle to make changes and nobody has the mandate to try to make sure uptime is in good shape is a challenge. A lot of things break, precisely because there's just as much change as Will sees, but it's very unevenly distriuted.
Has anyone demonstrably gained market share over a competitor who is vocally _not_ adopting agentic software development flows? This article outlines a lot of process churn without a clear through line to how it is impacting feature development or revenue.
I suspect that the vendors out there who have not adopted AI development and are losing market share to competitor who has are not vocal about not being an adopter, they are just complacent.
Writing about these things in public and putting yourself out there is greatly appreciated. Hats off for that!
However, as far as what's being pursued, it seems more like wantonly trying to ride a hype cycle without strongly questioning the end-to-end value of new software development approaches or vetting their immediate suitability.
Personally, I think it would be more sensible to take a few individuals or a smaller team(s) and do more isolated/skunkworks experimentation and adopt as justified based on what those people report/experience. The smaller group can adjust faster and iterate/advise the larger dev org about the good approaches/techniques/strategies, and avoid more broad damage/chaos for things that aren't that well thought out.
For more conservative AI use cases like adding to code review, writing low stakes PR summaries, or beefing up security checking, a more global, but still not off-the-rails, approach would be the kinds of things that would make more sense to push more broadly.
I find it a bit suspicious that they don't talk about the amount of human intervention required (apart from writing the RFC, which itself can be done by the agent).
I think this can work if you've set up your codebase with proper AGENTS.md with all the guarantees and low level design you expect. I think some human intervention is required here so that you keep the codebase maintainable - you as the human know the domain and future plan well so your addition is valuable.
I also think its necessary to have some garbage collection - scheduled jobs that look at the codebase and think of ways to tighten it and come up with better design.
Lastly, even if one doesn't take away the full factory, I still think that deployments should be fully automated. In the companies I have seen, deployments are still a cognitive burden - one must look at 100 different dashboards and test in staging, look at logs and so on. This is something the agent can do very well and is best automated. Very few orgs have done this!