Senior Data Engineer

About the Role

Experience: 6+yrs

Work Location: Bangalore/Hyderabad

Mode of Work: Hybrid

Responsibilities

Design, develop, and maintain data pipelines using Apache Airflow (including Managed Workflows for Apache Airflow - MWAA).
Build and orchestrate data workflows and jobs in Databricks (Delta Lake, Jobs, Workflows).
Develop robust Python scripts for data extraction, transformation, validation, and loading.
Write and optimize advanced SQL queries (PostgreSQL, Azure SQL) for large-scale datasets.
Implement and manage production-grade data pipelines for batch, incremental, and near-real-time processing.
Design and optimize AWS Redshift schemas and manage data solutions using AWS services such as RDS, Redshift, Glue, and IAM.
Ensure efficient data.

Qualifications

Bachelor’s degree in computer science or equivalent experience in leading development teams.

Required Skills

6+ years of Apache Airflow (including Managed Workflows for Apache Airflow - MWAA)
4+ Databricks (Delta Lake, Jobs, Workflows)
4+ Python (data pipelines, transformation, validation)
4 years of Advanced SQL (PostgreSQL, Azure SQL, query optimization)
4 years of Data pipeline orchestration and production deployment
Experience with large-scale data processing (batch, incremental, near-real-time)

Preferred Skills

AWS RDS, Redshift, Glue, IAM, AWS Redshift – schema design
AWS Glue, Athena, Databricks, S3