We are seeking a highly skilled Senior Software Test Engineer with deep functional knowledge of financial systems to lead the automation of data quality validation across large-scale data pipelines processing petabytes of data.
As part of the Finance Data Platform, you will design, build, and maintain scalable, automated testing frameworks that ensure the integrity, accuracy, and reliability of critical financial data. Your expertise in PySpark, Python, SQL, and Hive will be central to driving data quality standards across our enterprise data ecosystem.
This is a hybrid role. In-office expectations will be confirmed by the hiring manager.
Key Responsibilities
- Design, develop, and maintain automated data quality checks for complex, petabyte-scale data pipelines.
- Build scalable data validation frameworks using PySpark, PyTest, Python, SQL, and Hive, ensuring comprehensive test coverage.
- Partner with Data Engineering and DevOps teams to integrate automated testing into CI/CD pipelines.
- Analyse test results to identify data anomalies, root causes, and remediation strategies.
- Develop and maintain production monitoring and alerting solutions for data quality.
- Document testing standards, frameworks, and best practices, and mentor junior engineers in data quality automation.
- Continuously enhance test frameworks to improve performance, scalability, and reliability.
