Duplicate data entry: three routes out of retyping

In most companies the systems were bought one after another rather than planned together. First came the merchandise management system, because stock and prices had to be managed. Later an accounting system was added, because the tax adviser needed a particular format. Then came time recording, because hours had to be booked against orders, and at some point a spreadsheet for costing, because none of the existing systems produced the report the way management wanted to read it. Every single purchase was right at the time and solved a concrete problem for one department. The handover between the systems, by contrast, was never the subject of a decision, because it belonged to nobody.

On top of that come boundaries of responsibility. Sales maintains customer data the way it needs it for quotations, with contact person and terms. The warehouse needs a delivery address that may differ from the billing address. Accounting needs a customer account number, payment terms and a description that works for tax purposes. These three views of the same customer are each justified in their own field. As long as they are created in separate systems, the same information is created three times, and nobody is responsible for making sure it matches. When an address changes it is usually corrected where the error was noticed and stays as it was in the other two places.

The vast majority of companies in Germany are small and medium-sized enterprises (Federal Statistical Office of Germany), and that is exactly where a role responsible for processes across departmental boundaries is usually missing. It is therefore not helpful to treat duplicate entry as a discipline problem. It is a structural problem, not a matter of diligence. Anyone wanting to remove it has to start with the structure: with the question of which system holds the truth for which field, and how that truth reaches the other systems.

Pure typing time is the smaller part of the bill, but the only part that is easy to measure. Sit down with the people who carry out the workflow every day and measure on five real cases how long the re-entry takes. Do not estimate, use a clock, because estimated figures tend to come out too low when the activity feels self-evident (project experience). That measurement together with the number of cases per week produces a sound figure per case type.

The larger part of the cost is not in that calculation. It arises from discrepancies between systems: a line item reads differently in the order than on the delivery note, the invoice does not match the customer purchase order, the costing differs from the accounts. Each of these discrepancies generates queries between departments, search time across several systems and the occasional credit note run. There is also an effect that is hard to quantify but easy to observe: when reports repeatedly fail to match, management stops believing them and goes back to deciding by instinct. At that point the company loses the actual benefit of its data.

For the decision this level of accuracy is entirely sufficient. You do not need full cost accounting, you need an order of magnitude. If the annual effort for a case type is in the low four figures, an elaborate interface is rarely economic. If it is clearly higher, a closer look pays off. It is exactly this figure that is missing in many projects, which is why people then argue about technology although the question is a commercial one.

The first route initially costs no development work, only a decision. For every data field it is defined which system holds the binding version. Customer master data is led by the merchandise management system, payment terms by the accounting system, working hours by time recording. All other systems may display and use the value but not change it on their own. That sounds like a formality but changes a lot day to day: anyone wanting to correct an address then knows without asking where to do it.

The level matters. A system leads only for particular fields, never for an entire customer record. Otherwise you end up in debates about which system is the most important one in the company, and those can neither be won nor put to good use. Deciding per field, by contrast, can be settled factually: a value belongs where it is created in the specialist sense and where it is checked in case of doubt. Put the decision in writing, in a table with three columns: field, leading system, reasoning.

The leading-system route is the cheapest, but it does not remove retyping entirely as long as the systems are not connected. What it removes first is the argument about which version is correct, and it gives the remaining transfer a clear direction. That is the precondition for both of the following routes: a transfer whose direction has not been settled cannot be modelled cleanly either as an interface or as a file. If you want to put the system landscape in order more broadly, the wider framework is described under data integration.

The third route is often underrated because it looks unfashionable. Yet it solves a considerable share of cases in mid-size companies at a fraction of the cost. The leading system produces a file at a fixed interval and the target system reads it in. Almost every merchandise management system, accounting system and time recording tool can export and import data, even where no programming interface exists. For nightly runs, month-end closing or the handover to the tax adviser this is frequently the appropriate solution.

For this route to hold, it needs few but binding agreements: fixed field order, fixed separator, defined character set, unambiguous date format and a clear storage location with file names that show date and content. Equally important is the rule that a file, once produced, is not modified afterwards. That keeps it traceable later which data was handed over and when, which fits the German principles of proper bookkeeping. For invoices, the European standard for electronic invoicing provides a defined format that allows exchange between systems regardless of the vendor (European Commission).

The limits of this route are just as clear: there is no return channel, so the target system cannot report that a record was rejected. The data is only as current as the last run. And if nobody checks whether the file was produced and read in at all, an outage is noticed only weeks later. A simple check that reports daily that a file with a plausible number of rows has arrived is therefore part of the route and costs little.

Not every instance of duplicate entry has to disappear. There are cases where manual entry is simply the more economic and the more robust choice. That includes cases that occur rarely: if a case type comes up twenty times a year and takes three minutes each time, one hour of manual work per year stands against a development effort that will never pay for itself. It also includes cases with a high share of judgement, where the person entering deliberately checks, adds or decides. Automating that point does not remove the work, it removes the check.

Another case concerns systems that are foreseeably going to be replaced. If you know that the merchandise management system will be replaced in eighteen months, do not build a permanent interface to it, bridge the transition with a file or with manual work and plan the connection properly in the new system. The same applies here: as long as data in a system is not maintained reliably, an interface only distributes the mess faster. Clean-up comes before connection, in that order.