Senior Data Engineer

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £90,000

We're working with an innovative, data-driven organisation that is investing heavily in its Databricks platform and looking to add a Senior Data Engineer to its growing Data Engineering team.

This role is ideal for someone who enjoys being hands-on with Databricks and wants to play a key role in building and evolving a modern Lakehouse platform. You'll be working with large-scale datasets, designing robust data pipelines, and helping shape best practices across a cloud-first Azure environment.

You'll join a collaborative team where Data Engineering is viewed as a core business function, with the opportunity to influence architecture, tooling decisions and the future direction of the platform.

The Opportunity

As a Senior Data Engineer, you'll spend most of your time designing, building and optimising solutions within Databricks.

Key responsibilities include

Building and maintaining scalable data pipelines using Databricks and PySpark.

Developing high-performance batch and streaming data solutions.

Designing and implementing modern Lakehouse architectures using Delta Lake and medallion design principles.

Optimising Databricks workloads for performance, reliability and scalability.

Implementing data quality, monitoring, and observability frameworks.

Supporting governance, security, and access management within the data platform.

Collaborating with Data Scientists, Analysts and Product teams to deliver business-critical data products.

Driving engineering excellence through code reviews, knowledge sharing and technical best practices.

Identifying opportunities to improve platform performance and reduce cloud costs.

Contributing to the ongoing enhancement of the Databricks ecosystem and data engineering standards.

What You'll Need

We're particularly interested in engineers with strong hands-on Databricks experience.

Essential Skills

Commercial experience building solutions within Databricks.

Strong hands-on expertise with PySpark, Spark SQL, and Delta Lake.

Experience designing and supporting production-grade ETL/ELT pipelines.

Strong Python and SQL development skills.

Experience working with large-scale distributed data processing environments.

Understanding of Lakehouse architecture and modern data engineering best practices.

Cloud experience within Azure, AWS or GCP.

Excellent stakeholder management and communication skills.

What's On Offer?

Salary up to £90,000.

Hybrid working model.

Significant investment in Databricks and modern data technologies.

Opportunity to work on a large-scale Lakehouse platform.

Exposure to complex data engineering challenges.

Strong engineering culture with a focus on technical excellence.

Ongoing training, certification support, and career development.

Clear progression opportunities within a growing organisation.