Public-sector digital transformation

Data Migration and Data Quality When Moving to New Systems

A new system is of little use if incomplete, duplicated or inconsistent data is moved into it. We handle data migration in documented steps: inventory of sources, cleansing rules, field mapping between old and new, repeated test migrations, and reconciliation reports that prove what was transferred and what was not. The institution's data owner then signs off the result before go-live.

Send a project request WhatsApp

What we offer

Data in public institutions is usually spread across spreadsheets, databases of legacy systems, paper registers that have been partly entered, and files held by different departments. We begin by taking inventory of these sources and describing their content, volume and quality: fields in use, empty values, duplicates, and differences in how names, dates and codes are written from one source to another.

Together with the data owners we then define written cleansing rules, for example how date formats are standardized, how duplicate records are handled and what happens to incomplete records, and prepare a mapping document showing, for each field in the new system, its source in the old systems and any transformation applied. We never delete or change a record on our own initiative; every rule is approved by the institution, and unclear cases are referred back to it.

Next we run one or more test migrations in a test environment. After each run we issue a reconciliation report comparing counts, totals and samples between source and target, and listing rejected records with reasons. Once issues are resolved we plan the final migration, the cutover window and a fallback plan. Live operation only starts after the data owner has signed off the final report.

Who it is for

Institutions replacing a legacy systemWhen years of data must move into a new system without loss or distortion.
Departments working with scattered spreadsheetsWhen the same data is spread across files and departments in inconsistent formats.
Directorates consolidating branch recordsWhen records from several branches need to be unified in a central database.
Archiving and digitization projectsWhen scanned documents are complemented by metadata that must be accurate and searchable.

Problems we address

Duplicate records for the same person or entity, with names or numbers written in different ways.
Empty fields or implausible values nobody noticed because the old system never validated them.
No documentation of what fields and codes in legacy systems mean.
Migration done in one go without testing, so errors only surface after go-live.
No evidence that all records were transferred, which undermines trust in the new system.
Unclear responsibility for deciding whether data may be corrected or removed.

Tasks and deliverables

Data source inventory: A list of sources describing content, volume, format and the unit responsible for each source.
Data quality report: Analysis of empty, duplicate and inconsistent values in each source, with illustrative examples.
Approved cleansing rules: A document defining how each type of issue is handled, approved by the data owner.
Field mapping document: Links each field in the new system to its source and the transformation applied.
Migration tooling: Repeatable scripts and procedures that perform extraction, transformation and loading the same way on every run.
Reconciliation reports: Comparison of counts, totals and samples between source and target after each run, with a list of rejected records and reasons.
Cutover and fallback plan: Timing of the final migration, freezing input in the old system, and fallback steps if a serious issue appears.
Sign-off record: Documented approval of the final migration result by the data owner before go-live.

Who does what

What IAG does

Taking inventory of sources, analysing data quality and documenting findings.
Proposing cleansing rules and preparing the field mapping document.
Developing migration tooling and carrying out test runs and the final migration.
Producing reconciliation reports and following up on rejected records.
Planning cutover and fallback together with the institution's technical team.

What specialists, partners and authorities do

The institution appoints a data owner who approves cleansing rules and the result of each phase.
The institution resolves unclear cases and decides which records are corrected or excluded.
The institution or the vendor of the legacy system provides access to data and explains field meanings.
The institution sets data protection rules and decides who may view data during migration.

Practical example

Illustrative scenario

Consolidating records from three branches into a central system

An institution has three branches, each keeping its records in spreadsheets with different layouts, and wants to move them into a new central system.

Inventory the spreadsheets in each branch and describe their fields.
Analyse duplicates and differences in how names and dates are written.
Approve cleansing rules and the mapping document with the data owner.
First test migration and a reconciliation report listing rejected records.
Resolve issues and run a second test migration.
Final migration and data owner sign-off before go-live.

This is an illustrative scenario explaining the approach, not a client reference or an existing project.

How we work together

Inventory and analysis
We list data sources, analyse their quality and document findings without changing any data.
Rules and mapping
We draft cleansing rules and the mapping document, which the data owner approves.
Build the migration
We develop repeatable migration tooling and prepare the test environment.
Test migrations
We run one or more tests and issue a reconciliation report after each until issues are resolved.
Final migration and sign-off
We carry out the migration according to the cutover plan, and the data owner signs off the result before go-live.

Intended results

Data in the new system whose origin and transformations can be traced.
Documented evidence of what was transferred, what was rejected and why.
Fewer surprises after go-live thanks to prior test runs.
Written quality rules the institution can keep applying to new entries.

What we need from you

A data owner with authority to approve rules and results.
Access to data sources or copies of them in line with the institution's instructions.
People who know what fields and codes in the legacy systems mean.
Timely decisions on unclear cases.
A time window for the final migration and a freeze on input in the old system.

Basis for cost and timeline

Cost and duration depend on the number, size and current quality of sources, the complexity of the transformations, the number of test migrations and how quickly the institution can make decisions. We usually suggest starting with a fixed-price inventory and analysis phase, then quoting the migration itself once the actual extent of the issues is known.

Frequently asked questions

Do you correct the data yourselves?

We only apply cleansing rules approved by the institution. Any case not covered by a rule is referred to the data owner for a decision.

How many test migrations are needed?

That depends on data quality and complexity. We repeat test runs until the data owner accepts the reconciliation report, and agree the expected number in the quote.

What happens to the old system after migration?

The institution decides. We usually recommend keeping it read-only for a period the institution sets, and retaining a copy of the original data.

Can data be migrated from paper registers?

Yes, but it first has to be entered or scanned. We plan this as a separate digitization project or an additional phase.

Who may view the data during migration?

Only the people the institution designates, under the access and data protection rules it approves.

Request this service