The Data Engineer plays an integral role in supporting and testing a cohesive information and data management strategy across AWM for traditional and cloud solutions. Responsible for data management, data profiling and testing related to the AWM data lake. Design appropriate processes to oversee data lake expansion efforts to a wide range of users and outline appropriate data management and data use guidelines. Close partnership with Data Enablement technology team to develop cloud-enabled campaigns, list execution and Business Intelligence capabilities. Provide input on tool and technology selection and shares evolving best practices. Partner with business users/advanced analytics teams to provide technical insight/input on how to best acquire, process, store and manage data. Ensure adherence to enterprise data governance standards.Responsibilities
o Analyze and organize raw data
o Build data systems and pipelines
- Evaluate business needs and objectives
o Understanding business requirements and complex strategies to implement data management for internal business and technical partners. Ensuring Six Sigma levels of quality with automated privacy and audit controls.
- Design and improve processes related to data acquisition and data curation for different use cases. Review the data, validation, reporting, list production etc. to support implementation and delivery of various types of analytic solutions.
- Develop KPIs for monitoring lake use & storage, etc. to inform uptake
- Work closely with technology partners for data architecture, access controls, governance and system
- Define user profiles to determine data access and tool access needs. Develop user agreement policies and guidelines with internal audit and risk management and GCO to ensure appropriate data use
- Work with Domain Owners and Quality Assurance lead to develop and implement a consistent testing strategy and process for curated data elements
o Interpret trends and patterns
o Conduct complex data analysis and report on results
o Prepare data for prescriptive and predictive modeling
o Combine raw information from different sources
o Explore ways to enhance data quality and reliability
- Identify data quality gaps and devise remediation plan including prioritization and funding in concert with data owners and business stakeholders
- Coordinate with Analytic Tools workstream to create plan for bringing new tools to the data lake and educate lake users including creating training materials
- Engineering (B.E./ B.Tech.) graduate from a well-recognized institute would be preferred
- Strong knowledge of Data Management and Data Warehousing Concepts
- 2-4 years of experience in with expertise in Python, Pyspark and SQL Languages
- Experience on working with AWS Cloud framework and associated tools.
- Excellent technical, analytical and quality control skills
- Excellent communication skills – will be required to liaise independently with senior US counterparts on a variety of business deliverables / projects
- Exposure to Sharepoint,Powerapps, Linux, VBA preferred
- Competent in utilization of MS Excel, PowerPoint and Word
- Knowledge on financial industry, industry trends
- Highly proficient independent communicator
- Knowledge about DataIku, DataRobot, PowerBI and other BI and analytical tools is preferred.
- Exposure to Visualization tools such as Power BI or Tableau is advantage