Key Responsibilities

Data Pipeline Development

• Design, build, and maintain scalable ETL/ELT pipelines for batch and real-time data.
• Develop parameterized and reusable pipelines using Microsoft Fabric, Azure Data Factory, or Synapse Pipelines.
• Optimize large-scale data ingestion and transformation processes using Databricks, PySpark, SQL, and Dataflows.

Data Integration & Migration

• Work on data migration projects, moving workloads from on-premises to Azure Cloud or Microsoft Fabric.
• Integrate data from multiple sources (databases, APIs, Event Hubs, third-party systems, etc.).
• Implement data partitioning, chunking, and metadata-driven approaches for performance optimization.

Data Modeling & Storage

• Design and maintain Lakehouse, Data Warehouse, and Delta Lake structures.
• Build and manage semantic models for reporting with Power BI.
• Collaborate with analysts and business stakeholders to create efficient data models supporting analytics use cases.

Governance, Security & Compliance

• Ensure proper data governance, lineage, and cataloging using Microsoft Purview.
• Manage access and security via Microsoft Entra ID (Azure AD), roles, and service principals.
• Implement best practices for data quality, integrity, and compliance with organizational and industry standards.

Collaboration & Documentation

• Work closely with data analysts, BI developers, and business stakeholders to understand requirements and deliver solutions.
• Document technical designs, processes, and best practices for maintainability.
• Contribute to knowledge sharing and mentoring within the data team.

Key Skills & QualificationsTechnical Skills

• Strong experience with Microsoft Fabric, Azure Data Factory, Synapse Analytics, and Power BI.
• Proficiency in SQL (T-SQL, query optimization, stored procedures).
• Experience with PySpark, Python, or Databricks-like environments for big data processing.
• Knowledge of data modeling (star schema, data vault, dimensional modeling).
• Familiarity with real-time streaming (Event Hubs, Kafka, Service Bus, or similar).
• Experience with data governance tools (e.g., Microsoft Purview).
• Solid understanding of CI/CD pipelines, Git, and DevOps practices.

Professional Skills

• 4+ years of proven experience in a Data Engineering role (or similar).
• Strong problem-solving skills and ability to optimize data processes.
• Excellent communication and ability to translate technical solutions into business language.
• Experience in client-facing or cross-functional roles is an advantage.
• Relevant Microsoft certifications is a plus.