We are seeking an experienced Senior Data Engineer to lead the design, development, and optimization of large-scale data platforms across Azure, Google Cloud, and hybrid environments.
You will architect data pipelines that power AI-driven analytics, knowledge graphs, and agentic automation across enterprise and national-level projects.
The ideal candidate combines deep expertise in data architecture, orchestration, and cloud infrastructure with hands-on experience using AI-assisted development tools to accelerate delivery and ensure scalability, reliability, and intelligence in every layer of the data stack.
Key Responsibilities
Architect and lead the design of end-to-end data pipelines using Apache Airflow, Azure Data Factory, and Google Cloud Composer.
Design and manage data lake, data warehouse, and OLAP architectures using MSSQL, SSIS, and SSAS.
Build and optimize ETL/ELT frameworks integrating structured, semi-structured, and unstructured data across APIs, databases, and streaming platforms.
Implement and maintain graph and NoSQL databases (e.g., Neo4j, Cosmos DB, MongoDB, Elasticsearch) to support semantic analytics, search, and recommendation systems.
Establish standards for data quality, lineage, governance, and observability.
Collaborate with AI engineers, data scientists, and business analysts to enable retrieval-augmented generation (RAG), knowledge graph, and agentic orchestration use cases.
Drive multi-cloud infrastructure automation through Terraform, Pulumi, or Bicep and CI/CD pipelines.
Champion AI-assisted coding practices (e.g., GitHub Copilot, ChatGPT, Cursor) to enhance productivity, testing, and documentation.
Mentor junior engineers, review designs, and promote best practices in data architecture, DevOps, and AI-native engineering.
Contribute to strategic roadmap planning, including capacity scaling, cost optimization, and data-driven innovation initiatives.
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field.
- 5–10 years of progressive experience in data engineering, data architecture, or analytics infrastructure roles.
- Deep expertise in SQL/T-SQL, performance optimization, and data modeling.
- Proven hands-on experience with SSIS and SSAS, including cube design and deployment.
- Advanced proficiency with Airflow (custom DAGs, operators, sensors, and monitoring).
- Solid understanding of Azure (Data Factory, Synapse, Blob Storage, Functions) and Google Cloud (BigQuery, Cloud Storage, Pub/Sub) ecosystems.
- Strong Python development skills for ETL, automation, and API integration.
- Experience with Docker, Kubernetes, and CI/CD workflows for data services.
- Hands-on experience with graph databases (Neo4j, Neptune, TigerGraph) and NoSQL stores (MongoDB, Cosmos DB, Elasticsearch, etc.).
- Demonstrated use of AI-powered development tools (GitHub Copilot, ChatGPT, Replit Ghostwriter, Code Interpreter) in professional workflows.
- Knowledge of data security, governance, and compliance frameworks (GDPR, ISO 27001, NIST)
Our Commitment to Equal Opportunities: At Whiteshield, we are committed to providing equal opportunities in employment regardless of individual characteristics. We recognize that our employees feel appreciated when their thoughts and values are respected and considered. We’re committed to maintaining and driving an inclusive culture and workplace where all talents are nurtured and feel empowered to contribute.
Celebrating Achievements: At Whiteshield, we seek to reward those who demonstrate exceptional individual achievements above and beyond expectations in the pursuit of continuously seeking excellence for our clients. When individuals excel, we show our appreciation and celebrate achievements by generously recognizing and supporting the career progression of those who make unique contributions to the firm.
Empowering Our People: At Whiteshield, we recognize that our employees are the core to our success and believe that when our exceptional talent comes together, and wellbeing is a priority – innovation thrives. We are dedicated to promoting and encouraging a healthy work-life balance and adopting a genuinely flexible working model to empower our employees to pursue a balance that suits their personal needs.
