The question now comes up in almost every first conversation: will AI soon be writing our software for us?
A clear yes or no would be convenient, but it misses the point. Part of what still sounded like a brochure two years ago runs in everyday work today. Another part still doesn’t. And a third part was never an AI problem at all, but one of data and interfaces.
What has actually changed, where the work has moved as a result and which items regularly slip through in projects: that is what this is about.
Three stages, and only one of them is new
People talking about AI in development often mean very different things. It helps to separate three stages. They differ not in the tool, but in how much context the AI itself has to grasp.
At the first stage it makes suggestions in the editor: it completes functions, writes test cases for existing code, explains unfamiliar source. GitHub Copilot has done that reliably for years. Everyday equipment, not news.
At the second stage it works across several files. A refactoring right through a module, a new interface including tests, migrating a framework. Tools like Cursor or Claude Code read through the project on their own, change several places and run the tests. That holds up today, under one condition: the project needs a comprehensible structure and tests the AI can measure its own work against. Where both are missing, it produces a lot of damage very quickly.
The third stage is the one this is really about. From a cleanly formulated requirement, something clickable appears within hours: data model, interface, basic functions. Not a wireframe, but something you can operate.
That changes the early project phase above all. Which is why we now build a prototype together with our clients in the very first workshop instead of drawing wireframes. A sketch is hard to argue about, something operable is easy to argue about, because misunderstandings land on the table immediately. Anyone who has watched an approved concept fall apart after the first click test knows what that difference is worth.
And still: a prototype is not a product. Between “it runs in the meeting” and “it runs in production, with real data and real liability” lies the larger part of the work. Blurring that difference sells an illusion.
The work doesn’t disappear, it relocates
You often hear that AI makes developers more productive. That falls short. More precisely: the effort leaves one activity and turns up again in three others.
It moves from writing to describing. An unclear requirement now leads faster than ever before to a plausible-looking wrong solution. The bottleneck no longer sits at the keyboard, it sits in the question of what should actually be built. That clarity comes out of the conversation with the department that will use the thing.
It moves from writing to reviewing. Generated code looks good. It is cleanly formatted, sensibly named, tidily commented. Even when it contains a flaw in reasoning. That is precisely what makes it trickier than bad human code, where you can see the mistake. Review time is no longer somewhere to economise, it is the actual work.
And it moves into operating. Where three times as much code is written, three times as much has to be maintained and, at some point, understood again. A prototype nobody can see through any more is a mortgage. Regardless of who wrote it.
Responsibility disappears nowhere in all this. It just sits in a place where it is harder to delegate.
The switch-off test
The most common architectural mistake in AI projects has nothing to do with models. It looks like this: company knowledge gradually migrates into the AI tool. Product details in the chat history. Process rules in the system prompt. An assistant that somebody in sales clicked together themselves and that by now knows things nobody has written down anywhere else.
There is a simple test for this. Ask yourself what happens if you switch the tool off tomorrow or swap it for another one. If knowledge is lost in the process, it is sitting in the wrong place.
Authoritative data belongs in the authoritative systems: in the CRM, in the ERP, depending on the organisation also in the CMS or the document store. There it has a history, a permissions model and somebody who is responsible for it. The AI reads from it and writes results back, but it is never the place where something comes into being first.
Technically this pattern is called RAG, Retrieval-Augmented Generation. The question goes to your own data first, and only then do the retrieved sources travel to the language model together with the question. The practical gain is not just that the answer is current. It can be traced back to a source. For information that goes out to customers, that is what decides between usable and useless.
A model doesn’t compensate for bad data
It only phrases it more elegantly.
If the same customer goes by a different name in three systems, if product attributes are half maintained, if nobody can say which system wins when they contradict each other, then even the best model hands out wrong information. In complete sentences and with a convincing tone, which doesn’t make things any better.
That is why in practice an AI project almost never begins with a model. It begins with an unspectacular inventory: which system is the source for which information? Once you have answered that question, the largest part of the groundwork is behind you. And you benefit from it even if no AI is used in the end.
What proposals usually leave out
Development costs get calculated. Three items rarely stand next to them.
First, the running model costs. Unlike classic software, every use costs money, because every request is billed. With an internal assistant and modest usage, that barely registers. But if the function sits in every page view of a shop, the item grows along with your success. It is perfectly manageable through caching, smaller models for simple tasks, hard upper limits. You just have to do the sums beforehand and not after the first invoice.
Second, the review effort. Every output that goes outside or triggers a decision needs a defined way of dealing with errors. Either a person looks at it, or the system limits the damage itself by claiming nothing when it is unsure and passing the case on instead. An assistant without that mechanism is not a product. It is a risk with a user interface.
Third, the attack surface, and it has two sides. Whatever flows into somebody else’s model leaves the building. The question of processing location and data processing agreements belongs at the start and not in the acceptance test. And as soon as a model reads external content, meaning emails, uploaded documents, web pages, those can contain instructions that it follows. Prompt injection is what this is called. It is not a laboratory problem, it is the reason an assistant with write access to production systems needs a very tightly drawn scope.
The obligation that already applies
One point that regularly gets lost in the discussion about tools and models: since February 2025, Article 4 of the EU AI Act has required companies that deploy AI systems to ensure a demonstrable level of AI literacy among their staff.
That affects more organisations than most people assume. It is enough for employees to use AI tools in a work context, and the company counts as a deployer. There is no prescribed curriculum. What is required is a level of knowledge that fits the use: how does the tool work, where are its limits, which data may go into it, how do you recognise a wrong output.
In practice this mostly means keeping a record. Who was trained on what, and when. Not because a fine is looming immediately, but because missing training can be treated as an organisational failure when damage occurs.
A lot of this isn’t an AI problem at all
An observation somebody rarely says out loud: a good share of what gets requested as an AI project is in truth an interface problem.
If data moves between two systems regularly, formats are converted or approvals passed along, that needs no language model. It needs an interface or a rule-based automation. That is predictable, traceable, cheap to run, and it surprises nobody.
The sensible division of labour: rules take the unambiguous, AI takes the ambiguous. In customer service that looks like this: the enquiry is read in and routed by rules, because that part is unambiguous. Summarising it and drafting a reply is not, and that is what the AI is there for. A person approves it, and the result is documented back in the CRM. Every step does what it does best.
Where to start
The worst opening question is: where can we use AI? It reliably leads to solutions that go looking for a problem afterwards.
More useful is a question that is easy to answer in-house: what does somebody here retype regularly from one screen to another? Those spots are the best starting point, because the effort, the source of errors and the responsibility are already known there. After that comes the classification: is this a data problem, an interface problem or really one for AI?
A viable first use case repeats itself often enough for automation to pay off. The data it needs already sits in a system. A mistake is annoying but not dangerous. And there is somebody in the building who can judge whether the result is right. If one of those is missing, it turns into a difficult project.
Starting small is right, but starting measurably matters more. What makes no difference after eight weeks won’t make one after eight months either.
Conclusion
AI writes considerably more software today than many expected two years ago, and considerably less than the advertising suggests.
Working prototypes within hours are real and they change the early project phase. The road from there into production has not become shorter because of it, if anything it has become more demanding, because describing, reviewing and operating have gained weight. What decides the benefit in the end is the architecture: knowledge belongs in the authoritative systems, not in the tool. And no prompt optimisation replaces well-maintained data.
What belongs in the budget and is usually missing: running costs, review effort, security. What already applies, however small the undertaking: the competence obligation from the AI Act.
Anyone waiting for the autonomous development team is waiting for the wrong thing and is missing what has long been possible today.
