Tags: Lakehouse, Python, SQL
Role Title: Azure Databricks Data Eng
Employer: Leading Professional Services Firm
Required Experience: 2–5 Years
Location: Kolkata
Date published: 25 September 2026
A Leading Professional Services Firm is seeking a detail-oriented Azure Databricks Data Eng to build enterprise data platform solutions.
The Azure Databricks Data Eng must combine deep PySpark programming capabilities with hands-on architecture experience across Azure Databricks, Fabric, and Lakehouse patterns.
Key Responsibilities
- Design, build, test, and optimize production-grade ETL and ELT data pipelines on Azure Databricks and Microsoft Fabric.
- Develop, debug, and accelerate complex analytical workloads using PySpark, SQL, and Python.
- Translate complex enterprise data requirements into scalable Lakehouse architectures and data models.
- Ensure complete data quality, lineage tracking, schema consistency, and audit integrity across data lakes.
- Diagnose query processing bottlenecks, troubleshoot pipeline runtime failures, and tune data performance.
- Maintain technical architecture documentation, pipeline workflow diagrams, and operational production runbooks.
- Collaborate with data architects, business analysts, and cross-functional Agile engineering pods.
Requirements and Qualifications
- Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related discipline.
- 2-5 Years of hands-on data engineering experience building cloud data ingestion and transformation pipelines.
- Demonstrated mastery of Azure Databricks (notebooks, job orchestration) and high-volume PySpark processing.
- Advanced SQL querying and performance optimization skills combined with core Python automation capabilities.
- Beneficial exposure: Microsoft Fabric, Delta Lake, Unity Catalog, ADLS Gen2, and Azure Data Factory.