Total 7 job vacancies by Shell Infotech Pte. Ltd.

• Experience Requirements:
• Must have QA/Testing Experience.
• Data Testing Experience specifically in Big Data, Hadoop, or Cloud Data Warehouse environments.
• Databricks Experience of testing pipelines within a Databricks environment.
• Automation Focus: Proven track record of moving from manual SQL checks to automated Python-based testing frameworks.
• Mandatory: Databricks Certified Data Engineer Associate (at minimum).
• Preferred: ISTQB Foundation or Advanced Level (Test Automation Engineer).
• Core Technical Skills:
• Data Validation & Frameworks
• Great Expectations / Pandera: Proficiency in using Python-based libraries to define data "contracts" and automated validation suites.
• DLT Expectations: Deep understanding of Delta Live Tables (DLT) expectations (Fail, Drop, Quarantining bad records).
• Advanced SQL: Expert-level SQL for complex data reconciliation, identifying duplicates, and null-value analysis across billions of records.
• Python for QA (PySpark):
• Pytest-Spark: Experience using pytest to write unit tests for PySpark transformations and logic.
• Notebook Testing: Ability to write automated test notebooks that validate Medallion Architecture transitions (Bronze to Silver, Silver to Gold).
• Data Reconciliation: Building Python scripts to perform "source-to-target" counts and checksums across distributed file systems.
• Performance & Integration Testing:
• Scalability Testing: Ability to validate that data pipelines meet performance SLAs when data volume spikes.
• End-to-End Orchestration Testing: Testing the reliability of Databricks Workflows and handling of job failures/retries.
• Schema Evolution: Testing how pipelines handle upstream schema changes without breaking downstream Gold tables.
• Governance & Security Testing:
• Unity Catalog Validation: Testing Row-Level Security (RLS) and Column-Level Masking to ensure unauthorized users cannot see sensitive data.
• Data Lineage: Validating that data lineage in Unity Catalog correctly reflects the movement of data across the Lakehouse.

• Job Description
• Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns.
• PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization.
• Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration.
• Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration.
• Data Modelling: Experience implementing data models including dimensional modelling, data vault, or Lakehouse architectures.
• Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques.
• Python: Strong Python programming skills for data processing and automation.
• SQL proficiency for data querying and transformation
• Experience with cloud platforms (Azure, AWS, or GCP)
• Knowledge of streaming data processing (Structured Streaming)
• Familiarity with DevOps practices and CI/CD pipelines
• Experience with version control systems (Git)
• Requirements:
• Must have experience in data engineering or related roles
• Hands-on experience with Databricks platform
• Proven track record of refactoring legacy code to modern frameworks
• Experience building and maintaining production data pipelines at scale
• Background working across multiple data sources and formats
• Experience in agile development environments
• Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional
• Additional Certifications (Preferred)
• Databricks Certified Associate Developer for Apache Spark
• Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
• Relevant data engineering or big data certifications.

• The ideal candidate will have a strong background in maintaining and optimizing large-scale monitoring and logging systems.
• Must be with specific expertise in Elastic Stack (Elasticsearch, Kibana, Logstash, Beats), Apache Kafka, and Grafana.
• As an SRE, they will play a crucial role in ensuring the reliability, scalability, and performance of our production systems by designing, implementing, and managing robust observability solutions.

• Understand the product roadmap and ensure that testing is conducted as per the QA process.
• Work closely with the test lead to devise test strategies for the applications/projects assigned. Ensure that the test strategy is implemented as per plan.
• Adhere to the QA process, frameworks, principles, and guidelines always.
• Have good knowledge and hands on experience of agile testing methodology.
• Ensure that to Sprint/Project timelines are met and any deviations are resolved with the scrum master or reported to the test lead for resolution.
• Ensure that all deliverables including but not limited to feature files/test cases, test results, automated test scripts, automated regression test results etc., are submitted timely and reviewed by the test lead.
• Ensure that all domain, application, functional and technical issues are duly to the test lead for necessary advice, resolution, and support.
• Regularly attend all scrum ceremonies - daily stand up, sprint planning, retrospectives for assigned application/project.
• Ensure that realistic estimates are provided during Sprint Planning.
• Ensure that all test artifacts are uploaded to Zephyr/Test Management Tool and Teams in adherence to the QA guidelines.
• Share EOD status and sign off requests in the standard formats for test lead approval.
• Ensure that all QA KPIs are met, and regular evaluation is conducted with the test lead to ensure that the squad is on track.
• Constantly ideate and come up with innovative solutions to improve the effectiveness & efficiency of QA capabilities.
• Work closely with the test lead to implement new solutions and recommend improvements to the existing processes and frameworks based on regular feedback gathered from the team and from own experience.

• Understand the product roadmap and ensure that testing is conducted as per the QA process.
• Work closely with the test lead to devise test strategies for the applications/projects assigned. Ensure that the test strategy is implemented as per plan.
• Adhere to the QA process, frameworks, principles, and guidelines always
• Have good knowledge and hands on experience of agile testing methodology.
• Ensure that to Sprint/Project timelines are met and any deviations are resolved with the scrum master or reported to the test lead for resolution.
• Ensure that all deliverables including but not limited to feature files/test cases, test results, automated test scripts, automated regression test results etc., are submitted timely and reviewed by the test lead.
• Ensure that all domain, application, functional and technical issues are duly to the test lead for necessary advice, resolution, and support.
• Regularly attend all scrum ceremonies - daily stand up, sprint planning, retrospectives for assigned application/project.
• Ensure that realistic estimates are provided during Sprint Planning.
• Ensure that all test artifacts are uploaded to Zephyr/Test Management Tool and Teams in adherence to the QA guidelines.
• Share EOD status and sign off requests in the standard formats for test lead approval.
• Ensure that all QA KPIs are met, and regular evaluation is conducted with the test lead to ensure that the squad is on track.
• Constantly ideate and come up with innovative solutions to improve the effectiveness & efficiency of QA capabilities.
• Work closely with the test lead to implement new solutions and recommend improvements to the existing processes and frameworks based on regular feedback gathered from the team and from own experience.

• The ideal candidate will have a strong background in maintaining and optimizing large-scale monitoring and logging systems.
• Must be with specific expertise in Elastic Stack (Elasticsearch, Kibana, Logstash, Beats), Apache Kafka, and Grafana.
• As an SRE, they will play a crucial role in ensuring the reliability, scalability, and performance of our production systems by designing, implementing, and managing robust observability solutions.

Job Description
• Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns
• PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization
• Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration
• Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration
• Data Modelling: Experience implementing data models including dimensional modelling, data vault, or Lakehouse architectures
• Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques
• Python: Strong Python programming skills for data processing and automation
• SQL proficiency for data querying and transformation
• Experience with cloud platforms (Azure, AWS, or GCP)
• Knowledge of streaming data processing (Structured Streaming)
• Familiarity with DevOps practices and CI/CD pipelines
• Experience with version control systems (Git)

Requirements
• Must have experience in data engineering or related roles
• Hands-on experience with Databricks platform
• Proven track record of refactoring legacy code to modern frameworks
• Experience building and maintaining production data pipelines at scale
• Background working across multiple data sources and formats
• Experience in agile development environments
• Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional

Additional Certifications (Preferred)
• Databricks Certified Associate Developer for Apache Spark
• Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
• Relevant data engineering or big data certifications