RQ00241 - Software Developer - ETL - Senior

Must Haves:

• Significant hands-on experience designing and developing scalable, production-grade data pipelines for AI/ML and Generative AI workloads, including data ingestion, transformation, feature engineering, training and inference datasets, RAG/grounding data, and AI/ML pipeline integration.
• Advanced hands-on expertise in Python, SQL, ETL/ELT, APIs, data integration and transformation, and cloud-based data platforms, applying strong software engineering practices to build reliable, scalable, maintainable, and production-ready data solutions.
• Strong working knowledge of the Azure data and AI ecosystem, Azure Machine Learning, and Microsoft Foundry/Azure AI Foundry, with an understanding of ML pipelines, feature engineering, model training/testing, inference, monitoring, and the data requirements needed to support reliable enterprise AI/ML solutions.

Location: Hybrid – 3 days a week onsite

Public Sector Experience: preferred but not required

# of submissions/supplier: 1

Description

The Senior Software Developer - ETL will design, build, and maintain scalable data pipelines, integrations, and reusable data services that enable enterprise AI, analytics, and data initiatives.

The role will help improve access to trusted data across enterprise systems, operationalize data architecture and integration patterns, and reduce the time and effort required to onboard data for new AI and analytics use cases.

Key Responsibilities

• Design, develop, test, and maintain ETL/ELT pipelines and data integration solutions across enterprise data sources.
• Build reusable data pipelines, services, and integration components that can support multiple AI and analytics use cases.
• Ingest, transform, and prepare structured and unstructured data for analytics, AI, and machine learning workloads.
• Implement data architecture, modelling, and integration patterns established by enterprise architecture.
• Integrate data across cloud platforms, databases, applications, APIs, and enterprise systems.
• Design solutions that support scalability, reliability, performance, monitoring, and maintainability.
• Implement data quality and validation controls to improve the reliability of data provided to downstream consumers.
• Work with Data Architects, AI/ML Engineers, Data Scientists, product teams, and business stakeholders to translate data requirements into production-ready solutions.
• Optimize existing pipelines and integration processes to reduce duplication and accelerate onboarding of new data sources.
• Support deployment, troubleshooting, documentation, and ongoing improvement of enterprise data solutions.

Qualifications & Experience

• Degree in Computer Science, Information Systems, Engineering, Data/Analytics, or a related discipline, or equivalent experience.
• Significant experience in data engineering, ETL/ELT development, software development, or enterprise data integration.
• Demonstrated experience designing and developing scalable data pipelines and integration solutions.
• Strong programming and SQL skills, with experience working with modern data engineering frameworks and cloud-based data platforms.
• Experience integrating data through databases, APIs, file-based sources, and enterprise applications.
• Knowledge of data modelling, data quality, metadata, lineage, security, and data governance concepts.
• Experience supporting data platforms used for AI, machine learning, advanced analytics, and business intelligence.
• Strong analytical, troubleshooting, communication, and collaboration skills.