Understanding your Databricks usage in the NHS England SDE

When you use the Secure Data Environment (SDE), some activities in Databricks consume Databricks Units (DBUs). 

A DBU is a unit Databricks uses to measure how much of its services you use.

Your DBU consumption can come from 3 types of activity:

1. activity you carry out directly
2. automated workloads that you or your team have set up
3. platform and supporting activity

The SDE service will only include DBU consumption from activity you carry out directly or automated workloads setup by your team in the customer-chargeable DBU calculation. Platform and supporting activity will be excluded from DBU totals after 1 October 2026.

You will use DBUs when you carry out analysis or other work that needs Databricks compute.

This activity will appear on your DBU usage report under the following 2 origin products: All-purpose and Interactive.

All purpose

This represents classic all-purpose compute.

You consume DBUs while this compute is running. This can include using notebooks to develop code, explore data or carry out analysis.

You may continue to consume DBUs while the compute is running, even when you are not actively running code.

Interactive

Interactive represents serverless interactive compute.

You consume DBUs when you use serverless compute to carry out interactive work, such as running commands in a notebook.

Databricks manages the underlying compute for you.

Some DBU consumption can happen when you are not actively using Databricks.

This is usually because you or your team have set up a workload that runs automatically.

This activity will appear on your DBU usage report under the following 2 origin products: Jobs and SQL.

Jobs

Jobs represents compute used to run Databricks jobs and workflows.

For example, a job may run a notebook or data processing task at a scheduled time.

You consume DBUs each time the job runs. This can happen when you are not logged in to the SDE.

SQL

SQL represents Databricks SQL activity, including workloads that run on SQL warehouses.

You consume DBUs when SQL workloads use compute.

Some SQL activity can also run automatically. For example, Databricks can use serverless compute to refresh a materialised view.

Some DBU consumption is not caused by you directly running an analysis.

It can happen because Databricks is maintaining or supporting the platform and your workloads.

This activity will appear on your DBU usage report under the following 2 origin products: Predictive optimisation and Networking.

Predictive optimization

Predictive optimization is an automated Databricks service that optimizes managed tables.

Databricks carries out this work in the background to maintain and improve tables. This means you can see DBU consumption even when you are not actively using Databricks.

Networking

Networking represents some costs associated with connecting serverless compute to resources and transferring data back to clients.

This activity is usually generated as a result of another workload rather than something you run separately.