Azure Databricks Architect

Role Overview

Habilelabs is seeking an experienced Lead Architect with strong expertise in Microsoft Azure, Databricks, and modern Data & AI platforms. This role will design and deliver scalable, secure, high-performing enterprise data solutions while working closely with business stakeholders, engineering teams, and clients.

Key Responsibilities

• Architect end-to-end data and analytics solutions on Azure and Databricks.
• Design scalable platforms supporting batch, streaming, and real-time analytics.
• Lead solution design discussions, architecture reviews, and technical governance.
• Define data lake, data warehouse, lakehouse, and data mesh architectures.
• Translate business requirements into practical technical solutions.
• Establish platform security, governance, and compliance standards.
• Design data integration frameworks using Azure-native services and Databricks.
• Drive performance optimization, cloud cost management, and modernization initiatives.
• Provide technical leadership and mentor engineering teams.
• Support proposal development, effort estimation, and client presentations.
• Ensure alignment with enterprise architecture standards and best practices.

Required Technical Skills

• Azure: Azure Data Factory, Azure Databricks, ADLS Gen2, Azure Synapse Analytics, Azure SQL Database, Azure Event Hub, Azure Service Bus, Azure Functions, Azure Key Vault, Azure Monitor, and Microsoft Entra ID.
• Databricks: Lakehouse Architecture, Delta Lake, Unity Catalog, Databricks Workflows, Structured Streaming, Medallion Architecture, cluster optimization, and performance tuning.
• Data Engineering: Python, PySpark, SQL, data modeling, ETL/ELT frameworks, data warehousing, and real-time data processing.
• DevOps and Automation: Azure DevOps, CI/CD pipelines, Git, and Infrastructure as Code. Terraform is preferred.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10+ years of experience in Data Engineering, Cloud Architecture, or Solution Architecture.
• 5+ years of hands-on experience with Azure cloud services.
• 4+ years of hands-on experience with Databricks and Spark-based solutions.
• Demonstrated experience designing enterprise-scale data platforms.
• Experience working with large-scale structured and unstructured datasets.
• Strong stakeholder management and client-facing communication skills.