Bengaluru, Karnataka, India, India
October 16, 2023
Apply NowJob description
Responsibilities
- Architect highly metadata-driven data pipelines with algorithms for data duplication, data harmonisation, fuzzy matching, identity resolution
- Design and architect relational, time series and graph databases to run OLAP queries
- Design and develop SDKs and APIs to enable configurable data consumption paradigms
- Build tools to monitor the health of the data pipelines and data infrastructure
- Develop and lead the Data Engineering team
- Have 5+ years of experience with developing data applications using big data technologies such as Hadoop, Spark, Flink, Dataflow etc.
- Experience with workflow orchestration tools such as Airflow/Luigi/Azkaban etc.
- Experience with coding languages like Python/Java/Scala
- Experience with at least one cloud platform AWS/GCP/Azure
- Hands-on experience and highly advanced knowledge of SQL, Data Modeling, ETL Development, and Data Warehousing
- Experienced in scalable, configurable, parameterized, modular programming practices for data engineering.
- Knowledge and experience with Data Management and Data Storage best practices.
- Exposure to large databases, BI applications, data quality and performance tuning
- Good to have understanding of job management, resiliency
- Good to have prior experience with Graph, Time-series databases
