AWS Data Engineer-7-10Yrs-DELHI/NCR at Crescendo… · Hybrid -…

Job Title: AWS Data Engineer

Experience: 710 Years
Location: Delhi/NCR (Hybrid)
Employment Type: Full-Time

About the Role

We are looking for a highly skilled AWS Data Engineer with 7–10 years of experience in designing, developing, and maintaining scalable cloud-native data solutions on AWS. The ideal candidate should have strong hands-on expertise in Python, SQL, PySpark, AWS Glue, EMR, Redshift, and modern ETL/ELT frameworks. You will be responsible for building robust data pipelines, optimizing data processing, and enabling enterprise-scale analytics.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS services.
• Build and optimize data pipelines using AWS Glue, PySpark, EMR, and Lambda.
• Develop efficient data ingestion frameworks from multiple structured and unstructured data sources.
• Create and maintain data lakes on Amazon S3 and data warehouses on Amazon Redshift.
• Implement batch and streaming data pipelines using Kinesis or Kafka (preferred).
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Develop reusable frameworks and automate workflows using AWS Step Functions and Airflow (MWAA preferred).
• Collaborate with cross-functional teams to understand business requirements and deliver high-quality data solutions.
• Implement data quality checks, monitoring, logging, and error-handling mechanisms.
• Follow CI/CD best practices and support deployment automation.
• Troubleshoot production issues and ensure high availability of data pipelines.

Mandatory Skills

• 7–10 years of experience in Data Engineering.
• Strong programming skills in Python.
• Excellent knowledge of SQL.
• Hands-on experience with PySpark / Apache Spark.
• Strong expertise in AWS Glue.
• Experience with Amazon EMR.
• Hands-on experience with Amazon S3.
• Experience with Amazon Redshift.
• Knowledge of AWS Lambda.
• Experience with Amazon Athena.
• Hands-on experience with AWS Step Functions.
• Strong understanding of ETL/ELT, Data Warehousing, and Data Lake concepts.

Preferred Skills

• Apache Airflow (MWAA)
• Kinesis or Kafka
• Databricks
• Snowflake
• Terraform or CloudFormation
• Docker & Kubernetes
• Git, Bitbucket, or GitLab CI/CD
• AWS Certified Data Engineer – Associate (Preferred)

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Strong analytical and problem-solving skills.
• Excellent communication and stakeholder management abilities.
• Experience working in Agile/Scrum environments.

Good to Have

• Experience with real-time streaming pipelines.
• Exposure to DevOps and Infrastructure as Code.
• Experience in Banking, Financial Services, Insurance, Retail, or Healthcare domains.