The Lead Data Engineer will drive the design, development, and optimization of scalable data solutions that power analytics, reporting, and strategic decision making across the organization. This role requires deep technical expertise, strong problem solving skills, and the ability to guide teams in building robust data pipelines, analytical datasets, and data driven applications. The ideal candidate combines hands on engineering capability with leadership in architecture, quality, and cross functional collaboration.Key ResponsibilitiesData Engineering & Pipeline DevelopmentAnalyze and process large structured datasets using SQL and Python (Pandas, PySpark) to uncover insights and support strategic decisions.Build and optimize data transformations and analytical datasets using Pandas DataFrames and PySpark to ensure high quality, analysis ready data.Perform data cleansing and validation using regular expressions and other techniques to improve data accuracy and reliability.Develop and execute BigQuery queries for analytical exploration and production reporting.Integrate external data sources through APIs and web scraping to expand analytical capabilities and automate ingestion.Support and enhance scheduled pipelines using Apache Airflow to ensure timely, reliable data delivery.Apply Big Data, data modeling, and optimization techniques to design efficient, scalable ETL/ELT pipelines.Analytics & VisualizationDesign and maintain dashboards in Tableau or Power BI to visualize insights and monitor key performance metrics.Manage end to end analytical deliverables including requirement gathering, data validation, iterative development, and final reporting.Cloud & Architecture LeadershipWork with cloud data platforms—preferably GCP—to architect scalable, secure, and high performance data solutions.Lead best practices in data engineering including version control, CI/CD, data quality, and performance tuning.Cross Functional CollaborationPartner with business, product, and analytics teams to translate requirements into technical solutions and ensure alignment with organizational goals.Mentor junior engineers and contribute to establishing engineering standards, documentation, and reusable frameworks.Web UI & Application SupportUtilize foundational web UI concepts (HTML, CSS, JavaScript, React, Angular) to support data driven applications; hands on experience is a plus.Collaborate with frontend and platform teams to integrate data workflows into user facing tools and internal applications.Preferred Qualifications10+ years of experience in data engineeringStrong expertise in SQL, Python, Pandas, PySpark and distributed data processing.Experience with GCP (BigQuery, Cloud Storage, Composer, Dataflow) or equivalent cloud platforms.Experience in Tableau or Power BI for building dashboard and reports.Proven ability to lead complex data initiatives and drive architectural decisions.Experience working in Agile environments with cross functional teams.
