Databricks ( Lead) at Statsby Solutions in Pune, Maharashtra, IN

We are looking for an experienced Data Engineer (Lead) with strong expertise in Databricks to design, develop, and maintain scalable data pipelines and data processing solutions.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks.
• Develop data processing solutions using PySpark and Python.
• Work with Apache Spark for large-scale data processing.
• Build and optimize ETL/ELT pipelines.
• Work with Delta Lake for data storage and processing.
• Develop and manage workflows using Databricks Workflows.
• Perform data transformation, cleansing, and validation.
• Optimize Spark jobs and improve pipeline performance.
• Collaborate with data engineers, analysts, and other technical teams.
• Troubleshoot data pipeline issues and ensure data quality.

Required Skills

• 3–5 years of experience in Data Engineering.
• Strong hands-on experience with Databricks.
• Good knowledge of PySpark and Python.
• Strong understanding of Apache Spark.
• Experience with Delta Lake and data lake architecture.
• Good knowledge of SQL.
• Experience in building ETL/ELT data pipelines.
• Understanding of cloud platforms such as Azure, AWS, or GCP.
• Experience with Git and CI/CD is an added advantage.

Good to Have

• Experience with Azure Databricks.
• Knowledge of Azure Data Factory, ADLS, or similar cloud data services.
• Experience with DBT, Snowflake, or other modern data platforms.
• Databricks certification is a plus.

What We Offer

• Opportunity to work on real-world Data Engineering and AI projects.
• Exposure to modern data technologies and cloud platforms.
• Collaborative and learning-focused work environment.
• Opportunity to work with a growing technology team.