Data Engineer

Do you want your data engineering skills to make a real difference to the people an organisation serves?

SF Partners is recruiting a Data Engineer on behalf of a client, a well-established, purpose-driven UK organisation. This is a permanent role on a hybrid basis, working from the Birmingham office 2 to 3 days per week. Visa sponsorship is not available for this role.

About the data team

The organisation recognises the power of data, but siloed, disconnected systems make it hard to get a complete picture of activity and performance across a complex organisation. To change that, it is building a small central team to own data architecture and standards and create a data ecosystem that supports everything from strategic to transactional decisions. You will work closely with data leadership and other data teams across the organisation.

Key responsibilities

• Collaborate with cross-functional teams to understand data requirements and design pipelines that meet them.
• Develop and maintain ETL processes that move data from various sources into the data lake and data warehouse.
• Create and manage pipelines using SQL Server, Azure Data Lake, Data Factory, Databricks and Synapse.
• Build and maintain data models and schemas for efficient storage and retrieval.
• Write and optimise complex SQL queries to extract insight.
• Develop custom applications and scripts in .NET (including .NET Core), Python, ReactJS, C# or other relevant technologies to automate data tasks and provide user interfaces for data access.
• Implement data security and access control measures to protect sensitive information.
• Monitor pipelines for performance, reliability and scalability, and troubleshoot issues as they arise.
• Keep up to date with data engineering trends and recommend new technologies where they add value.

Required skills and experience

• Substantial real-world experience as a Data Engineer or in a similar role.
• Strong SQL proficiency for data extraction and manipulation.
• Experience with Microsoft Azure DevOps for CI/CD and version control.
• Proficiency in Microsoft Azure services, including Databricks, Data Factory, Synapse and Data Lake.
• Solid programming skills in .NET (including .NET Core), Python or similar languages.
• Strong problem-solving skills and attention to detail.
• Excellent communication and teamwork skills.
• Ability to travel occasionally to other UK offices as required.

Nice to have

• Familiarity with ReactJS and C#.

Why join

This is an exciting time for data at the organisation. You will help shape a new central data function from the ground up, turning disconnected systems into a joined-up ecosystem that improves understanding, insight and ultimately the services the organisation delivers.