Lead Data Engineer (Azure)

Overview

We are partnering with a fast-growing SaaS business to hire a Lead Data Engineer to help shape and scale their modern data and analytics platform. This is a hands-on leadership role where you will guide a small team, own core data pipelines, and influence platform architecture while remaining close to the technology.

Key Responsibilities

• Lead, mentor, and develop a small team of data engineers and analysts, setting technical direction and establishing best practices.
• Own the end-to-end data lifecycle, from ingestion to analytics delivery, building and operating robust data pipelines using Databricks, Spark, and Delta Lake.
• Partner with product, engineering, and other cross-functional teams to deliver on the data roadmap, creating trusted datasets, dashboards, and APIs for both internal and external customers.
• Drive improvements in data quality, reliability, governance, and observability across the platform.
• Support and enhance CI/CD processes, automated testing, and cloud operations to ensure a scalable and resilient data infrastructure.

Required Skills & Experience

• Proven experience leading or mentoring data engineers, preferably within a SaaS or product-focused environment.
• Strong hands-on proficiency with the Databricks ecosystem, including Spark and Delta Lake.
• Expert-level skills in Python and SQL for data manipulation, pipeline development, and analysis.
• Solid experience with a major cloud platform (Azure is highly preferred).
• Excellent communication and collaboration skills, with a proven ability to work effectively with cross-functional teams.

Nice-to-Have

• Experience with data orchestration tools (e.g., Airflow, Dagster).
• Familiarity with Infrastructure as Code (IaC) principles and tools like Terraform.
• Knowledge of BI and data visualization tools (e.g., Power BI, Tableau).