Big Data Support at Weekday AI (YC W21) in Hyderabad, Telangana, IN

This role is for one of the Weekday's clients

Salary range: Rs 1500000 - Rs 2200000 (ie INR 15-22 LPA)

Experience: 4+ yrs

Location: Hyderabad, Telangana, India

Job Type: Full-Time

We are looking for a skilled Big Data Support Engineer to support, monitor, and maintain enterprise-scale Big Data platforms and distributed data processing environments. The ideal candidate will have hands-on experience with Hadoop ecosystems, Kafka, and large-scale data infrastructure, along with a strong understanding of production support, troubleshooting, and performance optimization.

In this role, you will be responsible for ensuring the stability, availability, and performance of critical Big Data applications and data pipelines. You will work closely with Data Engineering, Development, Infrastructure, and Operations teams to resolve production issues, monitor platform health, optimize system performance, and implement proactive improvements. This position requires strong analytical skills, a customer-focused mindset, and the ability to manage multiple priorities in a fast-paced production environment.

Key Responsibilities

• Monitor, maintain, and support enterprise Big Data platforms to ensure high availability and optimal performance.
• Provide production support for Hadoop clusters, Kafka environments, and distributed data processing applications.
• Investigate, troubleshoot, and resolve production incidents related to Big Data infrastructure, applications, and data pipelines.
• Monitor system performance, resource utilization, and application health using enterprise monitoring tools.
• Perform root cause analysis for recurring production issues and implement preventive measures to improve platform stability.
• Support Hadoop ecosystem components, including HDFS, YARN, Hive, Spark, HBase, and related technologies where applicable.
• Manage Kafka topics, brokers, consumer groups, and messaging workflows while ensuring reliable data streaming.
• Collaborate with Data Engineers and Development teams to support deployment activities, configuration changes, and production releases.
• Execute maintenance activities, system upgrades, patching, and performance tuning with minimal business disruption.
• Develop automation scripts and operational documentation to streamline support processes and improve operational efficiency.
• Maintain incident reports, knowledge base articles, and standard operating procedures to support continuous improvement.
• Participate in on-call support rotations and ensure timely resolution of critical production issues while meeting SLA commitments.

What Makes You a Great Fit

• 4+ years of experience supporting Big Data platforms in enterprise production environments.
• Strong hands-on experience with Hadoop and distributed Big Data technologies.
• Proven expertise in Apache Kafka, including cluster monitoring, topic management, and troubleshooting.
• Good understanding of distributed computing concepts, data storage, and large-scale data processing.
• Experience supporting Big Data applications built using Spark, Hive, HDFS, YARN, or related Hadoop ecosystem components.
• Strong knowledge of Linux/Unix operating systems and shell scripting for system administration and automation.
• Experience working with SQL and relational or NoSQL databases for data validation and troubleshooting.
• Familiarity with production monitoring, alerting tools, log analysis, and performance tuning methodologies.
• Understanding of batch processing, streaming architectures, and enterprise data integration workflows.
• Exposure to cloud platforms such as AWS, Azure, or Google Cloud is an added advantage.
• Knowledge of scheduling and workflow orchestration tools such as Airflow, Oozie, or Control-M is desirable.
• Excellent analytical, troubleshooting, and problem-solving skills with the ability to work under pressure.
• Strong communication and collaboration skills with experience working across cross-functional technical teams.
• A proactive mindset with a focus on operational excellence, continuous improvement, and delivering reliable support for mission-critical Big Data environments.