Best Time to Modernize Your Data Infrastructure: Before or After It Starts Failing?

One need only look at a calculation carried over in the migration for a likely culprit; perhaps some refunds have been put in the wrong reporting period, or an amended transaction has not made it to the warehouse. There is little point in having quick queries if the figures are off to begin with.

Such is what makes for hard choices on infrastructure. An investment is dictated by such considerations as the cost of maintenance, the accuracy of reports, how well the database performs and the work entailed in doing away with the systems in place.

Look at the Cloud Bill a Little Closer

It is not uncommon for a database bill to be the subject of much scrutiny, even if the cost is not in proportion to the volume of information on hand. With certain analytical databases, one is put to the charge for computing resources used by queries. A dashboard that has to make repeated passes over a huge transaction table in order to put a handful of monthly totals on screen can run up an unexpected tab, all the more so with multiple teams running like reports.

To put a finger on where the money is going, there are query histories to consult. One will see from execution plans any number of superfluous scans, joins that are anything but efficient or calculations done over and over by the system.

Before parting with funds for a new platform, there are three things to look at:

• Queries that are reading too much data. See if a report is scanning a whole table when it could be making do with some aggregated numbers or a few records.
• Processing that is being done again and again. There may be pipelines in separate departments or for different applications doing the same sort of extraction and transformation.
• Data in storage for no good reason. Have a look at intermediate tables, duplicate exports and temporary datasets, bearing in mind what has to be retained.

Such an inquiry might result in a reworking of the partitioning or the way queries and processing are scheduled. Summary tables are an option for those figures that are in high demand, provided they are refreshed as they should be.

Then there is the matter of the platform itself. Any estimate worth having will factor in everything from support and engineering to put the move in place to the storage, transfers, and resources needed. Do not be misled by a low per gigabyte rate; it is no indication of what the final bill will be.

Follow a Refund Through the Reporting Process

When a customer puts in an order and later asks for his money back, the sales application will put down both. How it does so is a matter of the software in question. Some will have a refund transaction of its own; others alter the original order. Then there is the reporting pipeline: if it is only set to import finished sales and not any changes made after the fact, the figure for that refund will be tallied as revenue.

You can see the issue in other circumstances too, say with an accounting correction or a change to a product identifier, or when two customer accounts are merged. Move the records to a different database, and the calculation in the report will be no different.

Engineers have to trace the records from extraction on through to storage to get to the bottom of it. What is the source application doing to signal an update? Is a deletion noted? And will the reporting system put new values in place of the old ones?

For receiving such changes, some databases have change data capture; elsewhere an API or incremental export is used by the application. There are requirements to be met with each one for dealing with the odd interruption or a record gone missing, or having to run the process again.

As for the published numbers, it is up to finance to determine what bearing a correction has on them. The engineers may put the treatment in place once agreed upon, but the rules of reporting are a business concern.

What an Uptime Percentage Leaves Out

While one can make use of availability statistics, they do not tell the whole story when it comes to the hands-on work of putting an entire reporting environment back in order.

Then there is the matter of cost. An analysis of outages by the Uptime Institute for 2025 has some telling numbers: 57 per cent of those surveyed put the price tag on their last major outage at $100,000 or more, with a fifth or so saying they were out more than a million dollars. Such research into the sort of infrastructure outages that are commonplace is instructive in gauging the financial impact of any service disruption.

For recovery planning, there are two metrics to go by. The RTO, or recovery time objective, sets the limit on how long an interruption can be tolerated; the RPO defines what amount of data loss is acceptable in terms of time.

Demanding recovery point requirements might be satisfied through replication, but an inadvertent deletion will find its way to a replica as well, so having separate backups that are recoverable is a must.

It is not a question of simply restoring database files. One has to have applications with the proper permissions and connections, and the services they are reliant upon. To get the data back to where it should be, transaction logs could be called for.

An exercise in recovery ought to put these dependencies to the test in conditions that are representative. What is uncovered may show that the environment as it stands would be better served by a redesign or some alteration of backup procedures.

Moving Everything at Once Creates Its Own Problems

It is not uncommon for a number of disparate applications to draw on the very same database, each with its own agenda. One might have a dashboard that is content with periodic refreshes; another application in front of the customer will not tolerate anything but low query latency. Then there is financial reporting, which is beholden to precise decimal arithmetic and the need to replicate past calculations.

To put all these workloads under one roof is to invite a good deal of testing overhead. And should a deployment go awry, the impact is felt across more business functions than would otherwise be the case.

Before deciding what belongs in the migration, the team needs answers to three questions:

• Where does the data go? Trace the applications, reports, exports, and external consumers connected to the database.
• Which details matter to those consumers? Record expected field formats, calculations, update frequency, and treatment of historical information.
• Who will verify the replacement? Assign responsibility for checking results and agree on acceptable discrepancies.

Application settings, query logs, scheduled tasks, and conversations with system owners can reveal dependencies missing from the documentation. Some workloads may be suitable for separate reporting pipelines, whereas others rely on shared transactions that complicate separation.

Record counts and checksums help detect certain transfer errors. Reports used by finance and operations also need reconciliation against agreed business rules.

Companies can involve external data engineering services in database engineering, ETL/ELT development, migration, and DataOps. The scope should establish who handles production access, reporting approval, and the removal of obsolete connections.

When the Existing Database Becomes Difficult to Defend

An unsupported database version deserves attention for reasons that have little to do with query speed.

Security vulnerabilities may remain unresolved after vendor maintenance ends. Driver compatibility, operating-system requirements, and application dependencies can further restrict the available options.

Assessing the exposure requires checking vendor support conditions, security obligations, and the feasibility of upgrading the existing installation.

Capacity can present a separate problem. Consistently high resource consumption, unacceptable latency under representative workloads, or availability requirements beyond the platform’s capabilities may justify replacement.

Even then, the workload deserves investigation. Poor query design can exhaust substantial computing resources. Additional hardware may increase the bill without addressing inefficient processing.

A proposed replacement should demonstrate how it will handle the requirements the current platform struggles to meet. Performance tests, recovery exercises, and compatibility checks provide a much firmer basis for investment than general dissatisfaction with ageing technology.

Put Both Options Into the Budget

The expense of moving databases extends beyond the replacement platform. Engineers must adapt applications, transfer and verify records, test dependent systems, and prepare staff for changes in reporting procedures.

Running both environments during validation adds another expense. Retiring the previous system requires its remaining consumers to be identified and disconnected.

Keeping the existing database also has a price, including support contracts, maintenance work, recurring reporting corrections, and measures needed to address known technical risks.

The financial assessment becomes easier to examine when costs are separated into three groups:

• Routine operating costs: Infrastructure consumption, software support, monitoring, and maintenance.
• Engineering work: Repairs and upgrades for the existing system, alongside development, migration, and testing expenses for the replacement.
• Exposure to disruption: Documented security weaknesses, recovery limitations, and the potential financial consequences of unavailable services.

Estimates should reflect measured workloads and the organization’s actual requirements. Assumptions about processing demand, infrastructure consumption, and migration complexity deserve particular scrutiny.

A supported database that meets recovery and performance requirements may have considerable value left. Persistent security exposure, failed recovery objectives, or technical limitations affecting essential operations can make replacement a stronger investment.

The decision ultimately depends on what the business needs the data environment to deliver, which shortcomings have been demonstrated, and how the proposed work will address them. A credible budget should make those connections visible.

This is a sponsored article. Opinions expressed are solely those of the sponsor, and readers should conduct their own due diligence before taking any action based on information presented in this article.