Director, Data Integration & Workflows

Director, Data Integration & Workflows Location: New York, NY About the Role: Grade Level (for internal use): 13 The Team: The Director of Data Integration & Workflows leads the team that serves as the backbone of enablement at S&P Dow Jones Indices (SPDJI), responsible for building, templating, and standardizing the data pipelines and workflow patterns that power our index and analytical solutions. Reporting to the Director of Data AI & Enablement, this role is critical to the Data Platform enablement strategy, accelerating delivery, reducing production risk, and ensuring that every pipeline and workflow is scalable, observable, and maintainable. Responsibilities and Impact: Strategic Leadership & Technical Vision Define and drive the data automation and transformation strategy for the Data Integration & Workflows group, setting standards and approaches for building production-ready pipelines and workflow automation across value streams Establish reference architectures and engineering standards for ETL/ELT, orchestration, error handling, observability, and performance/cost optimization with clear "definition of done" criteria for production readiness Collaborate with the PPD Group to build sustainable, transformational enhancements to the data platform and associated tools Foster a culture of technical excellence, craftsmanship, reusable component development, and continuous improvement in automation maturity Contribute to the broader Data AI & Enablement strategy, ensuring Data Integration & Workflows capabilities align with organizational strategic goals Delivery Through Enablement Lead delivery through enablement by assessing SME technical capability and selecting the right engagement model Partner with value stream SMEs to co-develop and review pipelines, adapting support based on SME technical capability and fostering their growth Oversee the design and implementation of robust, reusable data integration and workflow patterns for both batch and streaming use cases Partner with PPD on feasibility and planning, providing realistic estimates, identifying dependencies, and shaping technical scope to ensure delivery commitments are achievable and measurable Develop and implement training programs to enhance the technical proficiency of value stream SMEs in data engineering practices Quality Assurance & Production Readiness Run the code review and quality gate process for SME-built pipelines, ensuring consistency in maintainability, testing, logging, data validation, and documentation prior to IT handover Ensure all solutions are production-ready with comprehensive documentation, testing, error handling, and operational monitoring Coordinate seamless IT handover and production gateway readiness, ensuring complete deployment packages (runbooks, architecture notes, testing evidence, monitoring/alerting expectations) Partner with IT during QA to resolve issues quickly and ensure solutions meet enterprise standards for quality, security, and supportability Implement and maintain governance frameworks specific to data integration, ensuring compliance with organizational policies and industry standards Operational Excellence Provide L3 support for production business-logic issues (in collaboration with value stream SMEs), driving root-cause analysis and permanent fixes for recurring pipeline failures or data breaks Ensure implementation of strong data reliability controls including validation rules, reconciliation checks, anomaly detection, and completeness/timeliness controls that protect downstream index processes Drive performance and cost optimization through appropriate partitioning, caching, incremental processing patterns, and compute usage tuning—balancing speed, stability, and platform spend Establish and monitor operational metrics to track solution delivery timeliness, pipeline reliability, and platform performance Team Development & Collaboration Lead, mentor, and develop a high-performing team of Data Integration Leads and Experts Build data engineering capability across the organization through structured mentorship, knowledge sharing, and hands-on coaching Collaborate effectively with the other pillars within Data AI & Enablement (AI Solutions and Data Governance) to ensure cohesive platform enablement Foster strong partnerships with PPD teams, IT, Data Value Streams, and Data Services & Strategy to align technical enablement efforts with business priorities Shared Accountabilities With PPD: Collaborate on prioritization and alignment of data integration efforts with business requirements and strategic goals; provide realistic technical assessments to inform planning. With IT: Partner to ensure infrastructure readiness, smooth deployment of production-ready solutions, and operational excellence; coordinate handover processes and support production gateway requirements. With Data Value Streams: Engage with value stream SMEs to co-develop solutions, ensuring alignment with business logic and domain expertise; assess and develop SME technical capabilities. With Data Services & Strategy: Work together with Vendor Governance and Catalog teams to establish scalable data integration patterns and ensure proper metadata and lineage tracking. With AI Solutions & Data Governance Teams: Coordinate on cross-cutting concerns including data quality standards, AI data pipeline requirements, and governance compliance. Ownership Data Integration & Workflow Strategy: Own the technical strategy, standards, and execution approach for all data pipeline and workflow automation initiatives Production Readiness Framework: Responsible for defining and enforcing the "definition of done" for production-ready data solutions SME Enablement in Data Engineering: Own the capability development strategy for value stream SMEs in data integration and automation Quality Gates & Code Review Process: Maintain the quality assurance framework for all data pipelines prior to production deployment What Success Looks Like Accelerate Solution Delivery: Streamline data pipeline development and deployment processes to deliver production-ready solutions efficiently and on schedule Enhance SME Technical Capabilities: Develop and implement comprehensive training and enablement programs to increase technical proficiency of value stream SMEs in data engineering Ensure Production Quality: Establish and maintain robust quality standards ensuring all pipelines are reliable, maintainable, observable, and cost-effective Drive Reusability & Standardization: Build a library of reusable components, templates, and patterns that accelerate consistent delivery across domains Key Performance Indicators (KPIs) Solution Delivery Timeliness: Percentage of production-ready data pipelines delivered on schedule SME Capability Improvement: Measurable improvement in technical skills of value stream SMEs through evaluations, certifications, and feedback Reusability Adoption: Adoption rate of standardized components and templates across value streams Platform Cost Optimization: Demonstrated improvements in pipeline performance and cost efficiency Manage and maintain key relationships with other groups within the department and across other departments, particularly Product Management, Client Coverage, Index Services and Technology Teams. Compensation/Benefits Information: (This section is only applicable to US candidates) S&P Global states that the anticipated base salary range for this position is $149,031 to $228,996. Final base salary for this role will be based on the individual’s geographic location, as well as experience level, skill set, training, licenses and certifications. In addition to base compensation, this role is eligible for an annual incentive plan. This role is eligible to receive additional S&P Global benefits. What We’re Looking For: Basic Required Qualifications: Education & Experience Bachelor's degree in Computer Science, Engineering, Information Systems, or related field; Master's degree preferred 10+ years of experience in data engineering, data integration, or related technical roles 5+ years of leadership experience managing technical teams and delivering complex data solutions Proven track record of building and scaling production data platforms in complex entertrise environments Experience in financial services or index management preferred Technical Expertise Deep expertise in ETL/ELT design patterns, data pipeline architecture, and workflow orchestration framework Strong knowledge of both batch and streaming data processing technologies Experience with modern data platforms, cloud infrastructure (AWS, Azure, or GCP), and data engineering tools Proficiency in programming languages commonly used in data engineering (Python, SQL, Scala, etc.) Understanding of data quality frameworks, observability, monitoring, and alerting systems Knowledge of DevOps practices, CI/CD pipelines, and infrastructure-as-code Familiarity with data governance principles and compliance frameworks Leadership & Soft Skills Strong leadership and team management skills with ability to inspire, mentor, and develop technical talent Excellent collaboration skills with ability to partner effectively across technical and business teams Outstanding communication abilities, able to articulate complex technical concepts to non-technical audiences and translate business requirements into technical solutions Strategic thinking with ability to align team activities with business objectives and drive innovation Problem-solving mindset with focus on root-cause analysis and sustainable solutions Experience managing stakeholder expectations and delivering on commitments in a matrixed organization Preferred Qualifications: Experience across the breadth of index concepts and asset classes Familiarity with SPDJI products, methodologies, or index calculation processes Expertise in Python or SQL Development Experience with Agile/Scrum methodologies and product-oriented delivery models Knowledge of data lineage, metadata management, and data cataloging Right to Work Requirements: This role is limited to persons with indefinite right to work in the United States. We require all external candidates who reach the final stage of our interview process to attend at least one in-person interview, which is ordinarily at your nearest S&P Global office. This must be completed before we can proceed to an offer. About S&P Global Dow Jones Indices At S&P Dow Jones Indices, we provide iconic and innovative index solutions backed by unparalleled expertise across the asset-class spectrum. By bringing transparency to the global capital markets, we empower investors everywhere to make decisions with conviction. We’re the largest global resource for index-based concepts, data and research, and home to iconic financial market indicators, such as the S&P 500® and the Dow Jones Industrial Average®. More assets are invested in products based upon our indices than any other index provider in the world. With over USD 7.4 trillion in passively managed assets linked to our indices and over USD 11.3 trillion benchmarked to our indices, our solutions are widely considered indispensable in tracking market performance, evaluating portfolios and developing investment strategies. S&P Dow Jones Indices is a division of S&P Global (NYSE: SPGI). S&P Global is the world’s foremost provider of credit ratings, benchmarks, analytics and workflow solutions in the global capital, commodity and automotive markets. With every one of our offerings, we help many of the world’s leading organizations navigate the economic landscape so they can plan for tomorrow, today. For more information, visit www.spglobal.com/spdji. What’s In It For You? Our Mission: Advancing Essential Intelligence. Our People: We're more than 35,000 strong worldwide—so we're able to understand nuances while having a broad perspective. Our team is driven by curiosity and a shared belief that Essential Intelligence can help build a more prosperous future for us all.From finding new ways to measure sustainability to analyzing energy transition across the supply chain to building workflow solutions that make it easy to tap into insight and apply it. We are changing the way people see things and empowering them to make an impact on the world we live in. We’re committed to a more equitable future and to helping our customers find new, sustainable ways of doing business. Join us and help create the critical insights that truly make a difference. Our Values: Integrity, Discovery, Partnership Throughout our history, the world's leading organizations have relied on us for the Essential Intelligence they need to make confident decisions about the road ahead. We start with a foundation of integrity in all we do, bring a spirit of discovery to our work, and collaborate in close partnership with each other and our customers to achieve shared goals. Benefits: We take care of you, so you can take care of business. We care about our people. That’s why we provide everything you—and your career—need to thrive at S&P Global. Our benefits include: Health & Wellness: Health care coverage designed for the mind and body. Flexible Downtime: Generous time off helps keep you energized for your time on. Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills. Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs. Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families. Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference. For more information on benefits by country visit: https://spgbenefits.com/benefit-summaries Global Hiring and Opportunity at S&P Global: At S&P Global, we are committed to fostering a connected and engaged workplace where all individuals have access to opportunities based on their skills, experience, and contributions. Our hiring practices emphasize fairness, transparency, and merit, ensuring that we attract and retain top talent. By valuing different perspectives and promoting a culture of respect and collaboration, we drive innovation and power global markets. Recruitment Fraud Alert: If you receive an email from a spglobalind.com domain or any other regionally based domains, it is a scam and should be reported to reportfraud@spglobal.com. S&P Global never requires any candidate to pay money for job applications, interviews, offer letters, “pre-employment training” or for equipment/delivery of equipment. Stay informed and protect yourself from recruitment fraud by reviewing our guidelines, fraudulent domains, and how to report suspicious activity here. ----------------------------------------------------------- Equal Opportunity Employer S&P Global is an equal opportunity employer and all qualified candidates will receive consideration for employment without regard to race/ethnicity, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, marital status, military veteran status, unemployment status, or any other status protected by law.  Only electronic job submissions will be considered for employment. If you need an accommodation during the application process due to a disability, please send an email to: EEO.Compliance@spglobal.com and your request will be forwarded to the appropriate person. US Candidates Only: The EEO is the Law Poster http://www.dol.gov/ofccp/regs/compliance/posters/pdf/eeopost.pdfdescribes discrimination protections under federal law. Pay Transparency Nondiscrimination Provision - https://www.dol.gov/sites/dolgov/files/ofccp/pdf/pay-transp_%20English_formattedESQA508c.pdf ----------------------------------------------------------- 20 - Professional (EEO-2 Job Categories-United States of America), ANLYTC202.2 - Middle Professional Tier II (EEO Job Group), SWP Priority – Ratings - (Strategic Workforce Planning)