Microsoft

Senior Data Engineer

Job description

Microsoft is on a mission to empower every person and every organization on the planet to achieve more. Our culture is centered on embracing a growth mindset, a theme of inspiring excellence, and encouraging teams and leaders to bring their best each day. In doing so, we create life-changing innovations that impact billions of lives around the world. You can help us achieve our mission.
Cloud Operations + Innovation (CO+I) is the engine that powers Microsoft’s core cloud platforms and services that millions of people use every day. With more than 95% of Fortune 500 business on Azure, 180 million using Office 365, and millions using other services – all running on Microsoft's cloud infrastructure – CO+I builds and operates the foundation upon which Microsoft’s mission to empower every person and organization comes to life.

Are you passionate about cloud computing? Do you get excited about taking a hands-on approach to transforming Microsoft’s most critical business through investigation, data analysis, and automation? If so, come and help us build the most reliable & efficient datacenter infrastructure on the planet. The CO+I Critical Environment Hardware and Solutions Engineering (CEHSE) team is responsible for designing and delivering solutions to support global datacenter operations and to improve availability. CEHSE is helping to drive CO+I’s transition to a Customer centric, data driven, live service culture. As a Software Engineer, you will a key player in this transition.

In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day.

Responsibilities


As a CEHSE Data Engineer, you will:
  • Be self-driven to research and identify data related problems or potential problems and drive to resolution
  • Be self-driven to drive improvements into our and our dependent data pipelines and sources
  • Design, build, test, and maintain a robust, reliable, and scalable data pipeline infrastructure
  • Assemble, validate, and document large, complex data sets that meet functional / non-functional business logic requirements using a variety of programming languages and development tools
  • Work closely with data and site reliability engineers, analysts, and other team members to ensure data is delivered to meet our end users’ needs
  • Monitor the performance and reliability of the data storage and data analytics systems and make recommendations for improving performance and reliability

Qualifications

Required Qualifications:
  • 5+ years of end-to-end experience with Data Engineering across SQL, Azure, or other big data solutions
  • 5+ years' experience with Azure Big Data stack (Data Lake, Spark, Hadoop) and distributed processing like Apache Spark, Azure Data Factory, Azure Automation or equivalent
  • 5+ years of experience with ETL/ELT environments with Azure Data Factory, Azure Synapse Integrate, SSIS, or similar technologies
Preferred Qualifications:
  • Expertise with Python/PySpark, complex SQL development, and Kusto query language in production systems
  • Hands-on experience with Azure Synapse Analytics, Azure Log Analytics, and/or any analytical and visualization tool (PowerBI, Tableau, etc)
  • Modern DevOps practices (ex: Git, CI/CD, Azure DevOps/VSTS), including build and deployment pipelines, yaml development, and unit/integration testing
  • Familiarity with one or more general purpose programming languages including but not limited to: Python, PowerShell, C#, JavaScript
  • Experience working in a fast-paced/competitive environment, and have a deep desire to work collaboratively, solve problems with groups, find win/win solutions and celebrate successes.
  • Ability to effectively deal with ambiguity and work independently to derive actionable solutions.
  • Ability to collaborate across the globe with diverse teams and be able to travel as necessary to learn and support operations in US and internationally.
  • Strong communicator with both words and data - you understand what it takes to go from raw data to something a human understands.
  • BA/BS in Computer Science, Computer Engineering, or equivalent work experience

Background Check Requirements:
Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings:
  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
  • Citizenship Verification: This position requires verification of US Citizenship to meet federal government security requirements.
#COICareers

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable laws, regulations and ordinances. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request via the Accommodation request form.

Benefits/perks listed below may vary depending on the nature of your employment with Microsoft and the country where you work.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.