Greenlight Financial Technology8 дней назад
Data Engineer
Зарплата не указана
Full TimeBengaluru
Обязанности
- 01Building and maintaining scalable data pipelines using Airflow to orchestrate data workflows that ingest, transform, and deliver data from various sources into Snowflake and Databricks
- 02Designing and implementing data models in Snowflake that support analytics, reporting, and ML use cases with a focus on performance, reliability, and scalability
- 03Developing infrastructure as code using Terraform to automate and manage cloud resources in AWS, ensuring consistent and reproducible deployments
- 04Monitoring data pipeline health and implementing data quality checks to ensure accuracy, completeness, and timeliness of data as business needs evolve
- 05Optimizing data processing workflows to improve performance, reduce costs, and handle growing data volumes efficiently
- 06Troubleshooting and resolving data pipeline issues, working through ambiguity to get to the root cause and implementing long-term fixes
- 07Bridging gaps between data and the business by working with cross-functional teams across the US and India office to understand requirements and translate them into robust technical solutions
- 08Creating comprehensive documentation on data pipelines, data models, and infrastructure, keeping documentation up to date and facilitating knowledge transfer across the team
Требования
- 012+ years of data engineering experience with strong technical skills and the ability to architect scalable data solutions
- 02Hands-on experience with Python for data processing, automation, and building data pipelines
- 03Proficiency with workflow orchestration tools, preferably Airflow, including DAG development, task dependencies, and monitoring
- 04Strong SQL skills and experience with cloud data warehouses like Snowflake, including performance optimization and data modeling
- 05Experience with cloud platforms, preferably AWS (S3, Lambda, EC2, IAM, etc.), and understanding of cloud-based data architectures
- 06Experience working cross-functionally with data analysts, analytics engineers, data scientists, and business stakeholders to understand requirements and deliver solutions
- 07An ownership mentality – this engineer will be responsible for the reliability and performance of their data pipelines and expected to fully understand data flows, dependencies, and their implications on downstream users
- 08A proactive mindset. While work is assigned, engineers are expected to independently drive their work forward, ask thoughtful questions, and bring structure to ambiguous technical problems