Technical note
From BI to Analytics Engineering: Why the Role Emerged
Analytics engineering did not appear because teams needed another job title.
It appeared because the old reporting operating model broke under the weight of modern business data.
For a while, business intelligence was enough. A company had an operational database, a reporting tool, a few extracts, and a small number of people who knew where the numbers came from.
Then the business changed.
Systems multiplied. Cloud warehouses became cheap enough to centralise more data. SaaS tools produced their own metrics. Dashboards spread. Analysts wrote more SQL. Executives expected numbers to be current, reconcilable, and available across every meeting.
The result was not automatic trust.
It was often more reporting surface area with less shared certainty.
Analytics engineering emerged to close that gap.
The symptom: the dashboard exists, but the logic is scattered
At Brightside Bikes, the early reporting setup might look simple.
Shopify holds orders. Stripe holds payments. HubSpot holds customers and leads. GA4 holds website events. Inventory snapshots arrive from a warehouse file. Finance keeps a board spreadsheet with month-end adjustments.
A BI dashboard can connect to some of these sources.
That does not mean the reporting system is trustworthy.
The dashboard may calculate revenue from Shopify. Finance may adjust revenue after refunds and timing differences. Growth may use order date while Finance uses recognition date. Operations may care about SKU margin after fulfilment cost. The board pack may include a manual bridge that never enters the warehouse.
Nothing here requires bad intent or poor engineering.
The problem is that business logic has outgrown the place where it lives.
Transactional systems are not reporting systems
Most source systems are designed for operations.
They help the business take orders, collect payment, fulfil products, manage campaigns, handle support, or record accounting events.
Those systems are usually transactional. They care about capturing a specific operational event correctly at the time it happens.
For Brightside Bikes:
- Shopify needs to process an order.
- Stripe needs to record a payment or refund.
- HubSpot needs to track a lead or customer interaction.
- The warehouse system needs to know what stock moved.
Those are operational concerns.
Reporting has a different job.
It asks: what does this event mean in the context of a business question?
That is why an operational table and a reporting model are not the same thing.
An operational order table may be excellent for checkout processing and still be awkward for weekly operating reporting. It may contain refunds, discounts, taxes, shipping, payment states, fulfilment states, cancelled orders, test orders, and partially updated records. The table is useful, but it is not yet a trusted metric.
Analytics engineering translates operational records into decision-ready assets.
BI made data visible
Traditional BI helped companies see data that used to be trapped inside systems.
That mattered.
Dashboards gave teams shared surfaces. Reports replaced some manual spreadsheet work. Leaders could ask better questions because more information was visible.
But visibility is not the same as trust.
If a BI tool contains the main revenue calculation, that logic may be hard to review, test, version, document, or reuse. If two dashboards recreate the calculation differently, both may look official. If a field changes upstream, users may only discover the issue when the meeting starts.
BI made reporting easier to consume.
It did not always make reporting logic easier to govern.
Warehouses centralised data, but not meaning
The modern data warehouse changed the economics of analytics.
Teams could load more source data into Snowflake, BigQuery, Redshift, Databricks, or another analytical platform. Storage and compute became more flexible. Analysts could write SQL over broader datasets instead of relying only on extracts.
That was a major step forward.
But centralising data does not automatically centralise meaning.
Brightside Bikes can load Shopify orders, Stripe payments, ad spend, inventory, and finance adjustments into one warehouse and still disagree on revenue.
The warehouse answers where the data is.
It does not automatically answer:
- Which revenue definition applies?
- What grain should the model use?
- Which orders should be excluded?
- How should refunds be treated?
- Which source wins when systems disagree?
- Who approves the logic?
- Which tests prove the number is safe?
Those are analytics engineering questions.
The role emerged when SQL became production logic
In many companies, SQL started as analysis.
An analyst would write a query to answer a question. The query would be copied into a dashboard, adapted for another team, pasted into a spreadsheet extract, or turned into a scheduled report.
Over time, that analytical SQL became production business logic.
But the operating habits often did not change.
Important SQL still lived in personal folders, BI calculated fields, unnamed worksheets, and one-off models. The business depended on it, but the engineering discipline around it was inconsistent.
Analytics engineering treats transformation logic as a product surface.
That means the logic should be:
- Version controlled.
- Reviewed before release.
- Tested against known risks.
- Documented in business language.
- Connected to lineage.
- Owned by a team or person.
- Deployed deliberately rather than accidentally.
The goal is not to make analysts pretend to be software engineers.
The goal is to make business logic reliable enough for business use.
Why dbt, Dataform, and similar tools mattered
Tools such as dbt and Dataform became important because they gave SQL transformation work an operating model.
They helped teams turn scattered SQL into structured projects.
Models could depend on other models. Sources could be declared. Tests could run automatically. Documentation could sit near the logic. Git could track changes. CI/CD could check code before production. Lineage could show how a number moved from raw source to dashboard.
That did not invent analytics engineering by itself.
It made the discipline easier to practise.
The tool is useful because the underlying need is real: business definitions need a controlled path from source systems to decision workflows.
A Brightside Bikes example
Suppose Brightside Bikes has three revenue surfaces:
- Shopify Net Revenue for trading performance.
- Finance Revenue for board reporting.
- Campaign Revenue for marketing attribution.
All three may be legitimate.
But they are not interchangeable.
Without analytics engineering, those definitions may live in three dashboards, two spreadsheets, and several Slack explanations.
With analytics engineering, the team can make the distinction explicit:
select
reporting_week,
shopify_net_revenue,
finance_revenue,
finance_revenue - shopify_net_revenue as revenue_bridge_difference,
bridge_reason
from mart_weekly_revenue_bridge
where reporting_week = date '2026-06-08';The important part is not the SQL itself.
The important part is that the business can see that the definitions are different, that the difference is expected, and that the bridge is owned.
That is the shift from reporting as output to reporting as an operating system.
What changed over time
The evolution can be summarised like this:
| Stage | Main contribution | Common trust failure |
|---|---|---|
| Operational databases | Captured business events | Not designed for analytical questions |
| BI reports | Made data visible to users | Logic became hidden in dashboards |
| Data warehouses | Centralised analytical data | Meaning and ownership still drifted |
| Modern data stack | Made loading and transforming data easier | More assets, more definitions, more sprawl |
| Analytics engineering | Made business logic versioned, tested, and documented | Requires operating discipline, not just tools |
Each stage solved a real problem.
Each stage also created a new Reporting Trust requirement.
What to inspect first
For an important dashboard, inspect:
- Is the key business logic in the BI layer, warehouse models, spreadsheets, or source systems?
- Can the team show the source-to-report path?
- Is the metric definition version controlled?
- Are transformations reviewed before release?
- Are tests mapped to business risks?
- Can Finance, Sales, Operations, and Growth explain why their numbers differ?
- Is there a named owner for the model?
If the answer is unclear, the company may have BI outputs without analytics engineering controls.
What good looks like
Good analytics engineering does not replace BI.
It makes BI safer.
The dashboard remains the surface where users consume the number, but the warehouse and transformation layer carry the approved logic. Tests protect known failure modes. Documentation explains meaning and caveats. Git records change history. Orchestration makes timing visible.
For Brightside Bikes, that means a weekly operating dashboard is not just a set of charts.
It is the visible end of a controlled path from source systems to business decision.
That is why the role emerged.
Not because dashboards went away.
Because dashboards became too important to leave their logic unmanaged.