A successful resource would be someone with 7+ years' experience who can work in a fast-paced business focused environment, an expert Data Modeler with extensive experience in building end to end Data Pipelines on Azure from any given data source which would include and not limited to Structured/Unstructured, AVRO, Parquet, csv, other file systems, Streaming, Messaging, and application data sources. Should have extensive experience in overall architecture and utilization of Azure Data Factory, Azure Synapse Analytics, Azure SQL/DW, Azure Analysis services. Ability to ensure on-time project delivery to the client, worked in an Agile Application Lifecycle Management framework, strong communicator, and ability to question the status quo.
As part of our Miracle Soft Data Practice and Analytics group, the Data Engineer will participate in the design, development and maintenance of end-to-end Business Intelligence solutions with Microsoft’s BI Stack (Power BI, SSIS, SSRS, SSAS,ADF, Azure Analysis Services, Azure Synapse Analytics, etc.). The role will involve working with both technical and business teams to understand KPI's and Metrics towards building the semantic layer and ensuring that the data is available within the DW. The Azure Data Engineer will prioritize work in alignment with business objectives and strategies as well as think like an entrepreneur when addressing various client challenges.
- 7+ years' experience working in a data function or equivalent position where primary responsibility is working with data, database design, data development and ETL is the primary responsibility.
- Help developers understand the business requirement and develop detail technical solution specification while working with Business, Solution, Technical and Data Architectures teams.
- Participate in the analysis, design and development of systems, including logical and physical data modeling, database design, database creation, analysis and evaluation of data and information requirements for new applications and reports.
- Conduct full technical discovery, identifying pain points, business and technical requirements, “as is” and “to be” scenarios
- 2-3 years of experience working on Data bricks, Azure Data Factory, Azure Synapse Analytics and other Azure data solutions ecosystems (Mandatory)
- 2 years strong development experience in one of Snowflake, Azure SQL DW, Big Query
- 2-3 years of experience working on Spark SQL, Hive SQL, USQL.
- 2-3 years of experience working on Spark, Scala and Python.
- 2-3 Experience on creating the frameworks towards building the data pipelines. (Mandatory) · Experience on configure the data streams between Event Hub and Azure Service Bus with other integration systems such as Data Bricks etc.·
- Well versed in DevSecOps and CI/CD deployments
- Cloud migration methodologies and processes including tools like Azure Data Factory, Data Migration Service, SSIS, Attunity (Qlik), Event Hub, Kafka, etc.
- Experience in data mining techniques and methodologies (data prep/modeling, classification, regression, clustering, causal modeling, AI, machine learning, ensemble approaches)
- Advanced experience in data visualization tools with a strong grasp of effective data modeling and visualization practices
- Design and Develop Business intelligence solutions with combining knowledge of SSIS/SSAS/SSRS/Azure Analysis Services/Azure Data Factory/Power BI technologies with a deep understanding of data structure / data models to design, develop, and tune BI solutions and reports
- Perform Advanced data analytics by designing and building solutions using technologies such as Azure Data Factory, Azure Data Lake, HD Insights, Azure Synapse Analytics, Azure Data Bricks, CosmosDB, Azure Stream Analytics, Azure Machine Learning Service, R server
- Work with Business Engagement and Business stakeholders to understand requirement, translate them into technical requirements, and partner.