Data platforms where every number has a source.
A figure in a report is only as useful as the path behind it. We build the ingest, storage and query layers so a number can be traced to the records, transformations and versions that produced it.
Where it fits
This work fits data that several teams depend on but nobody can fully explain: a metric that differs between two reports, a pipeline only one person can rerun, or source files loaded by hand before each close. The useful starting point is one dataset with a known consumer and a figure that can be checked against its sources.
Example workflow
Consider a figure assembled from two operational systems and a file delivered by a partner. The platform ingests each source with its load time and version, validates schemas and required fields, and records which transformation produced each derived table. When the figure changes, the lineage shows whether the cause was new source data, a late file or a change in the transformation. A load that fails validation is held back rather than silently replacing the previous result.
Agree the definitions first
We identify the consumers, the figures they rely on and how each one is defined, including the edge cases that make two reports disagree. Source ownership, acceptable freshness and retention are decided before the pipeline is built, so the checks have something concrete to test.
Make lineage part of the pipeline
Every load records its source, time and version, and every transformation is code under version control. Checks on schema, volume and key figures run with each load. Late or failed inputs are visible, and a rerun reproduces the same result from the same inputs.
What the delivery includes
We agree the scope and acceptance criteria around the selected workflow. The delivery covers the working service and what your team needs to take it over.
- Documented sources, definitions of the agreed figures and the owner of each dataset.
- Ingest, storage and transformation code with infrastructure as code, deployed in your accounts.
- Data quality checks on schema, volume and key figures, with failed loads held back and reported.
- Lineage from each derived table back to its sources and load versions.
- A handover covering reruns, backfills, schema changes and the responsibilities of the team running the platform.
What we need from you
- The reports, figures or downstream uses the platform must support, and who relies on them.
- Access to the source systems or exports, with documentation of formats and delivery schedules.
- Decisions on definitions, acceptable freshness, retention and data residency.
- An account owner and a team that will operate the platform after delivery.
Constraints to settle before rollout
- Source systems determine what history, granularity and change information is available.
- Conflicting definitions between teams must be settled by the business; a pipeline can expose the conflict but not resolve it.
- Volume, freshness requirements and query patterns shape storage choices and running cost, and are agreed before the scope widens.
Ownership and handover
The system runs in your accounts, with the code and infrastructure as code handed over to your team. We build the service; your team owns daily operation. Read our delivery methodology and approach to accounts, access and data residency for the wider delivery boundaries.
Related work
- System integration — Connect existing platforms with reliable handoffs, validation, retries and audit trails. Landvex builds integration services in your accounts.
- Cloud foundation — Accounts, IAM, networking and infrastructure as code in your cloud, under your billing, with EU or US data residency set at the account boundary.
Bring one concrete workflow
Describe the figures people rely on, where their data comes from and where the numbers disagree or arrive late. Include who uses the result and how often it is produced.
Discuss your workflow