Senior Databricks / Machine Learning Engineer at Supreme… · Hyderabad

ROLE OVERVIEW

We are looking for an experienced Senior Databricks / Machine Learning Engineer with 8+ years of experience for a part-time job support assignment.The ideal candidate must have strong hands-on experience in Azure, Azure Databricks, Apache Spark, PySpark and Machine Learning, with experience building data ingestion, processing, ML experimentation and deployment workflows.Candidates must be available for a 3-hour assignment within the 6:00 PM10:00 AM support window.

MUST-HAVE TECHNOLOGIES

Cloud:Microsoft Azure

Data Ingestion:Azure Synapse Pipelines

Storage:Azure Data Lake Storage (ADLS)

Data Processing:Azure DatabricksApache SparkPySpark

Data Formats:Delta Lake / Parquet

Machine Learning:PythonKerasMachine Learning model development

Hyperparameter Tuning:Optuna

Experiment Tracking:MLflow

Model Registry:MLflow Model Registry

Workflow Orchestration:Databricks JobsYAML Configurations

Source Control:GitHub

Artifact Management:JFrog Artifactory

Documentation:Confluence

Operational / AI Analysis:Genie AgentsFuture custom-agent ecosystem

Deployment:Plant Servers

Supporting Systems:DTRBOPH / Equipment MetadataPlant Telemetry Systems

KEY RESPONSIBILITIES

• Develop and support Azure Databricks data and ML solutions
• Build scalable data processing pipelines using Spark/PySpark
• Work with Azure Synapse Pipelines for data ingestion
• Process and manage data in ADLS using Delta/Parquet formats
• Develop ML solutions using Python and Keras
• Implement hyperparameter optimization using Optuna
• Manage ML experiments and model lifecycle using MLflow
• Maintain MLflow Model Registry
• Configure and manage Databricks Jobs and YAML-based workflows
• Use GitHub for source control and JFrog Artifactory for artifact management
• Support deployment of models/solutions to plant servers
• Work with plant telemetry, equipment metadata, DTR and BOPH systems
• Support operational analysis involving Genie agents and future custom-agent ecosystems
• Troubleshoot data, ML and deployment issues
• Maintain technical documentation in Confluence

CANDIDATE MUST HAVE

• 8+ years overall IT experience
• Strong Azure Databricks experience
• Strong Machine Learning experience
• Strong Python + PySpark + Apache Spark
• Azure Data Lake Storage experience
• Azure Synapse Pipelines experienceKeras / ML model development experienceOptuna / hyperparameter tuning experience
• MLflow + MLflow Model Registry experience
• Databricks Jobs + YAML experience
• GitHub experience
• Experience with Delta / Parquet
• Good understanding of ML lifecycle and deployment
• Experience working with production/operational systems
• Available for 3 hours daily
• Available within 6 PM10 AM support window