Data Engineer

Company:
Location: Fremont, CA

*** Mention DataYoshi when applying ***

The Data Engineer is responsible for processing structured and unstructured data, validating data quality, and developing and supporting data products. We are looking for someone with strong hands on experience in all layers of data Integration and analytics ! We especially need experience in using Python as an ETL tool. The Data Engineer plays a significant role in Agile planning, providing advice and guidance, and monitoring emerging technologies.

Technology We Use:

  • Python
  • Informatica
  • SQL Server and MySQL
  • Vertica
  • Kafka


Your Role:

  • Design, code, test, correct and document programs and scripts using agreed standards and tools to achieve a well-engineered result
  • Derive an overall strategy of data management, within an established information architecture that supports the development and secure operation of existing and new information and digital services
  • Plan effective data storage, security, sharing and publishing within the organization
  • Gathers and processes raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks
  • Ensures data quality and implements tools and frameworks for automating the identification of data quality issues
  • Collaborate with internal and external data providers on data validation providing feedback and making customized changes to data feeds and data mappings
  • Mentor and lead data engineers providing technical guidance and oversight
  • Provides ongoing support, monitoring, and maintenance of deployed products

Qualifications


Experience:

  • 5 – 8 years of development experience at an Enterprise level in the following tools and languages: Informatica , Python
  • Strong experience with relational databases like SQL Server, MySQL and Vertica. NoSQL databases experience is a plus
  • Strong background with data modeling, data access, and data storage techniques
  • Experience with design, development, and implementation of highly scalable, high-volume software systems and components, source of truth systems for different business areas, developing and maintaining web services in an agile environment
  • Working experience with Kafka Streaming layer
  • Experience in Spark Framework on both batch and real-time data processing is a plus
  • Experience in Big Data Integration & Analytics is a plus
  • Experience in Supply Chain and Logistics data is a plus
  • Bachelor’s degree in Computer Science or related field or equivalent combination of industry related professional experience and education

*** Mention DataYoshi when applying ***

Offers you may like...

  • inovex GmbH

    Data Engineer / Machine Learning Engineer*
    Home Office
  • XOi Technologies

    Data Engineer (Analytics)
    Nashville, TN
  • Cepheid

    Sr Data Engineer
    New York, NY 10001
  • Pacific Biosciences

    Engineer II, Software Data Engineer
    Remote
  • Health Catalyst

    Data Engineer (Life Sciences)
    Remote