Data virtualization

Data virtualization#

Data virtualization is the technology that lets applications access, combine and govern data from many different systems through a single logical layer — in real time, and without first copying or moving the data. It hides where each dataset lives, what platform stores it and which interface reaches it, presenting everything as if it were one database.

In Querona, data virtualization is the core technology that powers the Data Fabric — the unified, SQL Server–compatible layer through which every connected source is accessed, integrated and governed. This page explains the technology itself; for how it is assembled into the platform, see General architecture.

What it is#

Data virtualization integrates data from disparate sources, locations and formats — without replicating or moving it — to create a single, virtual data layer that delivers unified data services to multiple applications and users. (after Gartner)

An abstraction layer sits between the people and tools that consume data and the systems that store it. Consumers work against business-friendly views and never need to know:

where the data is stored,
what platform or technology stores it,
how it is processed, or
which interface or driver is needed to reach it.

Because the layer is decoupled from the sources beneath it, you can change, move or replace a source without rewriting the views — or the reports and applications that depend on them.

Note

Data virtualization does not replace your existing systems. It coexists with databases, warehouses and lakes, and can be adopted incrementally to augment and modernise what you already run.

The problem it solves#

In most organisations, data is spread across many systems — operational databases, data warehouses and lakes, SaaS and cloud applications, files, and REST APIs — each with its own technology, location and access method.

The traditional way to bring it together is to copy and consolidate it: build ETL pipelines that extract data from every source and load it into a central warehouse or lake before anyone can query it. That approach is powerful, but it has costs:

Latency — consumers see data as of the last load, not as it is now.
Duplication — every integrated dataset becomes a second copy to store, secure and keep in sync.
Brittleness — each new source or change means re-engineering a pipeline.
Time-to-value — useful data can take weeks or months to reach the people who need it.

Data virtualization flips the model: instead of moving data to a central place and querying the copy, it leaves data where it lives and integrates it logically, on demand.

How it works#

A data-virtualization solution turns scattered sources into one model in four steps:

Connect — establish access to each source through its native driver or protocol.
Abstract — expose each source object as a virtual table: a metadata-only relational table that maps to the source object, with no data copied.
Combine — build views that join, clean and reshape virtual tables into a canonical, business-friendly model, computed at query time.
Consume — publish that model through a single interface so any tool can read it as if it were one ordinary database.

At query time the engine plans the request, pushes down as much work as possible to each source (so filtering and aggregation happen close to the data), federates the partial results, and returns them in real time. When a view is queried often or is expensive to compute, it can be materialized into a fast store — combining the freshness of virtualization with the speed of a warehouse. (This blend of virtualization and materialization is what the industry calls a logical data warehouse.)

For a picture of these layers in Querona, see General architecture.

Core capabilities#

A complete data-virtualization platform brings together:

Logical abstraction — one model decoupled from the technology, location and format of each source.
Real-time federation — a single query can join data from several systems, computed as it runs, with no data movement.
Query optimization and push-down — work is delegated to the sources that can do it best, and only the results travel across the network.
Materialization — selectively persist hot or expensive views for warehouse-class performance, without warehousing everything. See Materialization.
A semantic layer and catalog — business-friendly names, reusable views and searchable metadata for self-service.
Universal delivery — expose the same model over many interfaces (for Querona: SQL Server (TDS), REST, GraphQL and MCP).
Unified security and governance — central access control, row-level security, masking and auditing applied once, across every source. See Data Security and Governance.

Data virtualization vs. ETL, warehouses and lakes#

Data virtualization and physical consolidation solve the same problem in opposite ways:

Aspect	Copy-first (ETL → warehouse)	Data virtualization
Where data lives	Copied into a central store	Stays in the source
Freshness	As of the last load	Real time, on every query
Duplication	A second copy to store and sync	None — unless you choose to materialize
Time to change	Re-engineer the pipeline	Edit a view
Up-front effort	Model and build before any use	Connect and query
Best fit	Heavy historical and aggregate	Real-time, agile, many sources

These are complementary, not mutually exclusive. Warehouses and lakes remain excellent for large historical and heavily pre-aggregated workloads; data virtualization gives you real-time, agile access across everything — and can sit on top of those warehouses and lakes as well, presenting them alongside every other source in one model.

When to use it — and when not#

Data virtualization is the right tool when you need current data, when it spans many heterogeneous sources, when copying it is undesirable (cost, governance, data residency), or when requirements change often and you want to model in views rather than rebuild pipelines.

Because it reads from live systems, two honest trade-offs apply — and both have well-understood answers:

A virtual query depends on the availability and performance of its sources. Push-down keeps most of the work in the sources themselves, and materialization insulates consumers from slow or busy systems.
Very large historical or repeatedly aggregated workloads can put load on operational sources. For those, materialize the relevant views so they are served from a fast store instead.

In other words, the cases where pure virtualization struggles are exactly the cases materialization is designed to cover.

Benefits#

Real-time insight — decisions run on current data, not last night’s extract.
Agility — integrate a new source or change a model by editing views, not rebuilding pipelines.
One model to maintain — your logic lives as SQL views in a single model, so Querona can refresh, refactor and re-engine it for you; there is no web of pipelines and connectors to operate separately.
Lower cost and footprint — far less data copied, stored, secured and synchronised.
Self-service and democratization — business-friendly views let consumers serve themselves, with less dependence on IT.
Consistent governance — security, masking and lineage are applied once, centrally, across every source.

The value lands differently for different roles:

Role	What data virtualization gives them
Business leaders	Faster, fact-based decisions from a single, trusted view of the business.
Information consumers	Instant, self-service access to all the data they need, the way they want it.
CIOs and IT leaders	An agile integration approach that responds to changing needs for less cost.
Architects	Freedom to evolve sources and platforms without breaking consumers.
Integration developers	More delivered value, sooner, with models that are easy to maintain.

Data virtualization in Querona#

Querona is a Data Fabric platform powered by data virtualization. It provides a common abstraction over any source — shielding users from its complexity and back-end technology — and exposes it through an emulation of one of the most widely used database servers in the world, so your existing tools, drivers and skills work unchanged.

You model with virtual databases, virtual tables and views (see Virtual tables and views), and integrate across systems in real time with Federation. Where on-the-fly performance is not enough, views can be materialized into any mix of the supported processing engines, used simultaneously and chosen per workload:

Apache Spark — built-in, managed and distributed with Querona; no Scala or SparkSQL to learn.
Any external Apache Spark instance, with or without Hadoop, on-premise or in the cloud (Azure Databricks, Azure HDInsight, Amazon EMR, and others).
StarRocks
Microsoft SQL Server (on-premise or Azure)
Azure Synapse Analytics
MySQL
Amazon Redshift
SAP HANA (on-premise or cloud)
Oracle
Vertica

Crucially, Querona has no vendor lock-in: it is not tied to a single materialization engine. Some virtualization and federation tools can only persist materialized data in one fixed store — for example PolyBase, which lands it in Microsoft SQL Server, or federation layers built around a single engine such as SAP HANA. Querona instead lets you choose, per virtual database, which engine its data is materialized into — each virtual database can be backed by a different engine — so you mix engines within one solution to use the best tool for each job. Heavy transformations can run on Apache Spark while fast ad-hoc queries are served from a columnar store such as StarRocks or Vertica, at the same time. For control at the level of an individual view, group views into separate virtual databases.

And, unlike a cloud service, Querona runs entirely on infrastructure you provide and control — on-premise, cloud or hybrid — so data never has to leave your environment.

Because Querona keeps no internal data store of its own, its responsibilities map onto a classic database server differently:

Feature or responsibility	Database server	Querona
data dictionary management	Yes	Yes
data storage management	Yes	external, e.g. Apache Spark
data transformation and presentation	Yes	Yes
security management	Yes	Yes
multi-user access control	Yes	Yes
backup and recovery management	Yes	external
data integrity management	Yes	Yes, not enforced
database access languages and API	Yes	Yes
database communication interfaces	Yes	Yes
transaction management	Yes	per processing engine involved

Querona and the data fabric#

A data fabric is an architectural approach, not a single product — an integrated, governed layer that connects data across your systems and delivers it to the people and applications that need it. Its foundation is data integration and data preparation and delivery: reaching data wherever it lives, shaping it, and serving it in a consistent, governed way. That foundation is what Querona provides — through data virtualization, a SQL-accessible model over your sources, selective materialization, and SQL Server–compatible, REST, GraphQL and MCP endpoints — on infrastructure you control.

Querona concentrates on those foundational capabilities rather than the more autonomous, metadata-driven automation that some data-fabric designs layer on top. It also pairs naturally with a data mesh: because you model each domain as virtual databases and views and publish them over multiple protocols, Querona is a practical way to build and serve domain data products — without imposing any particular ownership model. Fabric and mesh are complementary, and Querona supports either, or both.