Responsible for the monitoring and uptime of all production and Non-Production Big Data Stack.
Strive to improve the stability, security, efficiency, scalability, and availability of production systems by applying software engineering practices and by implementing monitors and alerts.
Estimate Bigdata stack capacities; develop methods for monitoring capacity and usage.
Lead efforts to develop and improve procedures for automated monitoring and proactive intervention, reducing any need downtime.
Provide and drive architecture and the design of Big Data solutions
Design and implement high performance systems supporting large volume real time and batched data processes
Propose and design data architecture solutions for scalability, high availability, fault tolerance, and elasticity for various application needs.
Develop ETL pipelines with robust monitoring and alarming
Develop data models that are optimized for business usability and understanding Develop and optimize data tables using best practices for partitioning, compression, parallelization, etc.
Optimize SQL and ETL solutions to solve various reporting requirements.