Senior Data Engineer

Senior Data Engineer Location: IND-Pune-Smartworks About the Role: Take your software engineering career to the next level as a Senior Product Software Data Engineer. You will play a key role in designing, developing, and troubleshooting software programs. Your technical skills and innovative mindset will contribute to the continuous improvement of our products, making a significant impact on user experience and satisfaction. Responsibilities: Collaborate with cross-functional teams to review use cases and translate business requirements into scalable data solutions. Design and develop highly optimized, reliable, and secure enterprise-level data pipelines for real-time and batch processing. Analyze existing business processes, identify gaps, and recommend data-driven improvements. Build modern real-time data solutions usingScala,Apache Spark, andAzure Databricks. Partner with data scientists and business stakeholders to operationalize machine learning models. Champion data governance, metadata management, and operational excellence across data platforms. Ensure data quality and integrity by developing validation frameworks and monitoring mechanisms. Adhere to coding standards, participate in peer code reviews, and contribute to technical documentation. Mentor junior engineers and contribute to knowledge sharing across the team. Skills: • Software Engineering: The ability to design, develop, and maintain software systems and applications by applying principles and techniques of computer science, engineering, and mathematical analysis. This includes the capacity to understand user requirements, create and test the software, and resolve any software-related issues. • Software Development: The ability to design, write, test, and implement software programs, applications, and systems. This includes understanding various programming languages, software architecture, and software testing methods. It also involves problem-solving capabilities to fix software issues and improve functionality. • Problem Solving: The ability to understand a complex situation or issue and devise a solution by defining the problem, identifying potential strategies, and ultimately choosing and implementing the most effective course of action. • Analysis: The ability to examine complex situations or problems, break them down into smaller parts, and understand how these parts work together. • Testing: The skill of evaluating a system or process, often in software or product development. It involves the ability to identify problems, measure effectiveness, and ensure quality or functionality. • Agile: The ability to swiftly and effectively respond to changes, with an emphasis on continuous improvement and flexibility. In the context of project management, it denotes a methodology that promotes adaptive planning and encourages rapid and flexible responses to changes. • Source Code Repository: The ability to effectively use a source code repository, a file archive and web hosting facility where a large amount of source code, for software or for web pages, is kept, either publicly or privately. This skill involves the ability to manage and track changes to code, identify and fix issues, merge code from different branches, and collaborate with other developers. • Relational Database: The ability to design, implement, and manipulate a relational database, a type of database that stores and organizes data in a structured way and where data is logically interrelated. This skill often requires proficiency in SQL, database management systems, and understanding of database design principles. • Documentation: The ability to create, manage, organize, and maintain important information and documents in various physical and digital formats. This skill may include preparing reports, managing files, storing data, and keeping records organized and updated for easy retrieval and understanding. 7+ years of overall experience in data engineering or related fields. 5+ years of experience building data pipelines for structured and unstructured data. 2+ years of experience with big data technologies includingPySpark,Hadoop,Kafka,EventHub, andStream Analytics. 3+ years of experience in relational and dimensional data modeling usingSQL Server and data warehousing concepts. 3+ years of experience withT-SQL, stored procedures, and performance tuning. 3+ years of experience withAzure Data Factory,Azure Data Lake,Azure SQL DB, andAzure Synapse Analytics. Good to have 2+ years of experience withAzure ML,Python, andScala. Good to have2+ years of experience withPower BI andAzure Analysis Services. Experience working with business stakeholders for requirements gathering and use case analysis. Strong communication, collaboration, and creative problem-solving skills. Preferred Qualifications Bachelor’s degree in computer science, Engineering, or equivalent practical experience. Experience withAgile/Scrum methodologies and DevOps practices. Familiarity with CI/CD pipelines and version control systems (e.g., Git). Azure Data Engineer Certification or equivalent is a strong advantage. Our Interview Practices To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process. Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.