Lead Cloud Data Engineer (Databricks) - Assistant Vice President

Lead Cloud Data Engineer (Databricks) - Assistant Vice President Location: Hyderabad, India Who we are looking for: A strong Sr. Data Engineer with deep hands-on expertise in Databricks and modern data engineering platforms. Proven ability to design, build, and optimize scalable data pipelines on cloud platforms. Experience working in agile teams and collaborating across business, architecture, and engineering stakeholders. Strong problem-solving mindset with focus on delivering high-quality, reliable data solutions. What you will be responsible for: Design, develop, and maintain scalable data pipelines and workflows using Databricks and Spark. Build and optimize ETL/ELT pipelines for batch and near real-time data processing. Collaborate with architects and business stakeholders to translate requirements into technical implementation. Ensure data quality, governance, and consistency across data platforms. Perform performance tuning and optimize cost efficiency of data workloads. Support production deployments, troubleshoot issues, and ensure reliability of data systems. What we value: Strong engineering and ownership mindset with attention to quality and performance. Collaboration and ability to work effectively in global distributed teams. Continuous learning and adoption of modern data engineering practices. Ability to communicate technical concepts clearly to both technical and non-technical stakeholders. Mandatory skills / experience: 14-18 Hands-on experience with Databricks and Apache Spark (Python/Scala). Strong experience in Data Engineering and Big Data technologies. Experience with cloud platforms such as AWS or Azure. Strong SQL and Python/Scala programming skills. Experience with data pipeline design, ETL/ELT frameworks, and data modeling. Experience in performance optimization and scalability of data solutions. Additional requirements / Good to have skills: Experience with streaming technologies such as Kafka. Knowledge of Delta Lake and data lakehouse architecture. Experience with CI/CD and DevOps practices for data engineering. Familiarity with containerization tools like Docker and Kubernetes. Exposure to data governance and regulatory environments. Additional information (if any): Focus on building scalable and production-grade data platforms. Opportunity to work on large-scale data transformation initiatives using Databricks. Are you the right candidate? Yes! We truly believe in the power that comes from the diverse backgrounds and experiences our employees bring with them. Although each vacancy details what we are looking for, we don’t necessarily need you to fulfil all of them when applying. If you like change and innovation, seek to see the bigger picture, make data driven decisions and are a good team player, you could be a great fit. Why this role is important to us: Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our business to deliver data and insights to our clients. We’re driving the company’s digital transformation and expanding business capabilities using industry best practices and advanced technologies such as cloud, artificial intelligence and robotics process automation. We offer a collaborative environment where technology skills and innovation are valued in a global organization. We’re looking for top technical talent to join our team and deliver creative technology solutions that help us become an end-to-end, next-generation financial services company. Join us if you want to grow your technical skills, solve real problems and make your mark on our industry. About State Street Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success. We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future. As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law. Discover more information on jobs at StateStreet.com/careers Read our CEO Statement