HCLTech→
Apache Spark Technical Specialist - Scala, Python at HCLTech · Chennai
Skills
Job Description
Job Summary
Scala Data Engineer
Lloyds Banking Group is the UK’s largest Retail, Digital and Mobile bank, supporting over 30 million customers. Our products and services touch the lives of millions of people every day, giving us a unique opportunity to Help Britain Prosper. Technology, data and engineering are central to our strategy, with continued investment in cloud, data platforms, artificial intelligence and automation.
The Finance Platform has an opportunity within its Data Application Solutions Lab function for an individual to support the delivery of modern data solutions and reporting capabilities across Insurance Finance. Working within a cloud-first environment, you will help build, enhance and support strategic data products that enable critical Finance processes and decision-making.
Our engineering teams develop data solutions primarily using Scala and Apache Spark on Google Cloud Platform (GCP). We leverage modern cloud-native technologies including BigQuery, DataProc, GCS and automated CI/CD pipelines to deliver scalable and resilient data platforms. Whilst experience in Scala is beneficial, we welcome applications from engineers with strong software engineering fundamentals gained in other languages or technologies who are keen to develop their skills in modern data engineering. Data Engineering experience with problem solving in industry is particularly important.
You will begin by contributing to enhancements and support activities across our existing platforms. We work using Agile delivery practices, including Scrum, Kanban, sprint planning, backlog refinement and continuous improvement.
This is a hands-on role where you will work closely with Finance, Accounting, Risk and Engineering colleagues to solve complex business problems through practical and scalable data engineering solutions. The role requires strong analytical thinking, problem-solving skills and a passion for delivering high-quality technology outcomes.
We are committed to creating an inclusive environment where colleagues can thrive and be themselves. We actively support diversity across all backgrounds and experiences and are proud to offer reasonable adjustments throughout our recruitment process.
Technical Proficiencies
- Computer Science, Software Engineering or related degree, or equivalent industry experience.
- Experience developing and supporting data solutions using Apache Spark and distributed processing frameworks.
- Experience developing applications in Scala or another object-oriented/JVM-based programming language.
- Strong understanding of software engineering principles including testing, code quality and maintainability.
- Experience with unit testing frameworks such as Xray or equivalent.
- Experience delivering solutions on Google Cloud Platform (GCP).
- Strong SQL \ Python skills for data analysis, troubleshooting, optimisation and data profiling.
- Experience working with BigQuery and large-scale analytical datasets.
- Familiarity with CI/CD and DevOps tooling including Maven, Git, Nexus, SonarQube, Harness and automated deployment pipelines.
- Experience using JIRA and Confluence to manage Agile delivery and technical documentation.
- Experience working within Agile teams using Scrum, sprint planning and iterative delivery practices.
- Strong communication and stakeholder management skills.
- Demonstrable ability to apply practical problem-solving techniques to real-world business and data challenges.
Nice to Have
- Experience with DataProc, Cloud Composer, GCS or other GCP data services.
- Experience within Financial Services, particularly Finance, Insurance, Regulatory Reporting or Risk data domains.
- Knowledge of modern data platform architectures and cloud migration programmes.
- Experience supporting production platforms and investigating data or operational issues.
- Exposure to DevOps, platform engineering or infrastructure-as-code practices.
- Previous experience with SAS, particularly in reporting, analytics or legacy platform support.
Key Responsibilities
Scala Data Engineer
Lloyds Banking Group is the UK’s largest Retail, Digital and Mobile bank, supporting over 30 million customers. Our products and services touch the lives of millions of people every day, giving us a unique opportunity to Help Britain Prosper. Technology, data and engineering are central to our strategy, with continued investment in cloud, data platforms, artificial intelligence and automation.
The Finance Platform has an opportunity within its Data Application Solutions Lab function for an individual to support the delivery of modern data solutions and reporting capabilities across Insurance Finance. Working within a cloud-first environment, you will help build, enhance and support strategic data products that enable critical Finance processes and decision-making.
Our engineering teams develop data solutions primarily using Scala and Apache Spark on Google Cloud Platform (GCP). We leverage modern cloud-native technologies including BigQuery, DataProc, GCS and automated CI/CD pipelines to deliver scalable and resilient data platforms. Whilst experience in Scala is beneficial, we welcome applications from engineers with strong software engineering fundamentals gained in other languages or technologies who are keen to develop their skills in modern data engineering. Data Engineering experience with problem solving in industry is particularly important.
You will begin by contributing to enhancements and support activities across our existing platforms. We work using Agile delivery practices, including Scrum, Kanban, sprint planning, backlog refinement and continuous improvement.
This is a hands-on role where you will work closely with Finance, Accounting, Risk and Engineering colleagues to solve complex business problems through practical and scalable data engineering solutions. The role requires strong analytical thinking, problem-solving skills and a passion for delivering high-quality technology outcomes.
We are committed to creating an inclusive environment where colleagues can thrive and be themselves. We actively support diversity across all backgrounds and experiences and are proud to offer reasonable adjustments throughout our recruitment process.
Technical Proficiencies
- Computer Science, Software Engineering or related degree, or equivalent industry experience.
- Experience developing and supporting data solutions using Apache Spark and distributed processing frameworks.
- Experience developing applications in Scala or another object-oriented/JVM-based programming language.
- Strong understanding of software engineering principles including testing, code quality and maintainability.
- Experience with unit testing frameworks such as Xray or equivalent.
- Experience delivering solutions on Google Cloud Platform (GCP).
- Strong SQL \ Python skills for data analysis, troubleshooting, optimisation and data profiling.
- Experience working with BigQuery and large-scale analytical datasets.
- Familiarity with CI/CD and DevOps tooling including Maven, Git, Nexus, SonarQube, Harness and automated deployment pipelines.
- Experience using JIRA and Confluence to manage Agile delivery and technical documentation.
- Experience working within Agile teams using Scrum, sprint planning and iterative delivery practices.
- Strong communication and stakeholder management skills.
- Demonstrable ability to apply practical problem-solving techniques to real-world business and data challenges.
Nice to Have
- Experience with DataProc, Cloud Composer, GCS or other GCP data services.
- Experience within Financial Services, particularly Finance, Insurance, Regulatory Reporting or Risk data domains.
- Knowledge of modern data platform architectures and cloud migration programmes.
- Experience supporting production platforms and investigating data or operational issues.
- Exposure to DevOps, platform engineering or infrastructure-as-code practices.
- Previous experience with SAS, particularly in reporting, analytics or legacy platform support.