Monitoring interfaces properly: beyond the server being up

It helps to think of monitoring not as one thing but as four levels building on each other. Each level answers its own question, each catches its own type of failure, and none replaces another. Leaving a level out creates a blind spot — not a problem as long as you know about it, dangerous when you believe it is covered.

Effort rises from level to level, but so does the benefit for day-to-day operations. The first two levels usually come with the technology or can be set up within a few hours. The third and fourth levels have to be described by someone from the business side, because that is where operational expectations are put into words. That is exactly why they are missing in many companies — not because of the technology, but because nobody writes the expectation down.

In most companies whose support we take over, level 1 is in place, level 2 partly, level 3 rarely and level 4 almost never (project experience). Yet the fourth level is the only one that supports decisions: it shows which connection permanently creates rework and where a rebuild pays off. Which figures are suitable for that and how they are produced regularly is described under metrics and reporting.

Once the checks are in place, the harder question follows: at what point should an alert be raised? Fixed numbers fail regularly. Setting a rule that fewer than fifty orders a day triggers an alert produces a message every Saturday, on every bridging day and during every shutdown period — and the check is switched off after three weeks. Setting the threshold so low that it never triggers leaves a check that is nothing but paperwork.

A usable threshold has three components. First an expected value derived from your own history, for example the average of the same weekday over the past eight weeks. Second a tolerance band around that value reflecting normal fluctuation. Third a minimum duration: a deviation only becomes an incident once it persists for a defined period — around thirty minutes for a transfer running every few minutes, a single missed run for a nightly job.

In practice it works well to start wide and tighten the thresholds after two to four weeks, once the real fluctuation is known. That includes an operating calendar: public holidays, shutdown periods, stocktaking days and announced maintenance windows belong in the configuration, otherwise monitoring reports on exactly the days when nobody is available anyway. A threshold that has never triggered is not a quiet connection but an unverified assumption.

Logs are the part that seems least important during setup and is missed most when an incident is reviewed. The question is never whether anything is logged at all — almost every system writes something. The question is whether the log contains what you will need later, and whether it still exists at that point.

For a business check, six pieces of information belong in every line: timestamp, the quantity checked, the expected value, the actual value, the assessment and the recipient of the alert. Without the expected value it is impossible to say afterwards whether the check should have triggered at all. Without the recipient there is no way to trace who was informed. Both are regularly omitted because they look redundant in normal operation.

The second half concerns retention. In many environments logs overwrite themselves after a few days because that is the default setting. If an error only surfaces during the monthly close, exactly the period in question has disappeared. What works is an explicitly defined retention period per connection, separated into technical logs and business check logs — technical lines may live shorter, business lines should outlast at least one completed closing cycle.

Where these decisions are recorded is not a matter of taste: they belong in the description of the connection, together with direction, interval, ownership and restart procedure. Monitoring that is only configured in tools and described nowhere is lost with the next system change. How to keep such descriptions short instead of turning them into a binder is shown under process documentation.

The usual mistake during rollout is insisting on completeness. Anyone wanting to monitor every connection on four levels at once needs a project, a budget and coordination — and therefore never starts. The reverse order works better: begin with the connection whose three-day outage would create the largest backlog, and set up the simplest effective check there.

The following order can be worked through alongside daily business. Each step delivers value on its own, and you can stop after any step without leaving a half-finished state. The organisational part dominates: points one to three are mostly agreement, real implementation effort only starts at point four.

For companies without their own IT department this still means noticeable effort, especially in points four and six. It can be handled as a one-off setup followed by ongoing support; recording all connections with direction, interval and ownership is part of a process analysis, and how setup and ongoing support are charged is set out under pricing. The decisive point remains the same regardless of who implements it: monitoring without named ownership is a technical indicator. Only with a name, a deadline and an escalation path does it become part of operations.