Senior Data Engineer

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000

Databricks - Apache Spark - Delta Lake - Python - SQL - Azure

We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team where you'll play a key role in shaping a modern Databricks environment, influencing technical decisions, and delivering scalable data solutions that directly impact the business.

Working within a cloud-first environment, you'll collaborate closely with Data Scientists, Analysts, Product teams, and senior stakeholders to build robust, high-performing data products whilst helping drive engineering best practice across the organisation.

The Role

As a Senior Data Engineer, you'll be responsible for

Building and maintaining scalable data pipelines within Databricks.

Developing and optimising batch and streaming data workflows.

Designing and implementing Lakehouse architectures and medallion data models.

Implementing data quality, monitoring, and observability frameworks.

Maintaining security, governance, and access control standards.

Collaborating with Data Scientists and Analysts to deliver business-critical data products.

Driving engineering best practice through code reviews and technical standards.

Identifying platform optimisation and cloud cost reduction opportunities.

Translating business requirements into scalable technical solutions.

Contributing to tooling decisions, documentation, and continuous improvement initiatives.

What's On Offer?

Salary up to £85,000.

Hybrid working model.

Modern Databricks and Azure technology stack.

Exposure to large-scale, enterprise-level data platforms.

Opportunity to influence architecture and engineering standards.

Collaborative and supportive engineering culture.

Ongoing learning and professional development opportunities.

Strong career progression within a growing technology business.

Key Experience Required

Proven experience as a Data Engineer within production environments.

Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL.

Experience building and supporting scalable ETL/ELT pipelines.

Strong understanding of distributed data processing and data modelling.

Cloud platform experience across Azure, AWS, or GCP.

Experience implementing data quality frameworks and pipeline monitoring.

Excellent communication and stakeholder engagement skills.

Desirable Experience

Databricks certification.

Unity Catalog and Databricks Asset Bundles.

Terraform or Bicep.

dbt or similar transformation frameworks.

Kafka, Event Hubs, or other streaming technologies.

Retail, supply chain, or consumer goods industry experience.