Data Engineer

Department: IT & Change

Employment Type: Permanent - Full Time

Location: Manchester, United Kingdom, Manchester, UK, Hybrid

Key Responsibilities

• Build and maintain ingestion and transformation pipelines on Microsoft Fabric (Data Pipelines, PySpark notebooks, Dataflows Gen2) across the Landing, Curate and Enrich layers of the medallion architecture.
• Onboard new source systems by setting up the team’s reusable ingestion patterns (Fabric mirroring from Snowflake and SQL Server, change data capture, file and API loads). Do this through configuration rather than one-off code.
• Write and tune PySpark and T-SQL for Lakehouse and Warehouse workloads, including fact and dimension models in the enterprise data warehouse.
• Implement data quality rules and validation pipelines, and help investigate and fix data issues found by DQ checks and reconciliations.
• Work through the team’s engineering workflow: Git-based development in Azure DevOps, pull requests and peer review, and CI/CD promotion from Dev to UAT to Production.
• Contribute to the infrastructure-as-code (Terraform) and the orchestration config database (SQL projects / DACPAC) as needed.
• Work with Data Stewards, Data Analysts and the Data Architect to understand source data and turn mapping specs and business rules into working transformations.
• Keep documentation current, including source-to-target mappings, transformation rules and runbooks, so the work stays auditable and supportable.
• Support the platform day to day: monitor pipeline runs, triage failures and escalate where needed.
• Help prepare trusted datasets and Power BI semantic models for portfolio analytics, asset strategy, finance and risk.
• 
  Our tech stack
  • Platform: Microsoft Fabric (Lakehouse, Warehouse, Notebooks, Data Pipelines, Dataflows Gen2, Mirroring, OneLake)
  • Languages: Python / PySpark, Spark SQL, T-SQL
  • Sources: Snowflake, SQL Server, files and third-party feeds
  • Azure: Azure SQL Database, ADLS Gen2, Key Vault, Data Factory, Entra ID (service principals and managed identities)
  • Engineering: Azure DevOps (Repos, Pipelines, Boards), Git, fabric-cicd, Terraform / HCP Terraform, SQL Database Projects
  • Governance and reporting: Microsoft Purview (data quality), Power BI (semantic models)

About you

• Experience building data pipelines as a Data Engineer or in a similar role
• Strong Python/PySpark and SQL skills
• Hands-on experience with Microsoft Fabric, or with Databricks / Azure Synapse / Azure Data Factory and a willingness to move to Fabric
• Comfortable with Git, pull requests and CI/CD. Terraform or other infrastructure-as-code experience is a plus
• Understanding of data modelling (dimensional / medallion) and data quality practices
• Works well in a team, takes feedback in code review, and is keen to learn and develop
• Strong analytical, verbal and written communication skills
• Degree in Computer Science, Information Systems, Data Science or a related field preferred
• Experience in financial data domains (loan servicing, transactions, reference data, IBOR/ABOR) or in Real Estate, Credit, Banking or NPL Asset Management is desirable
• Microsoft certification (e.g. DP-700 Fabric Data Engineer) a plus