Data Engineer, Web Scraping (Remote)

Job description

Lightcast is looking for a talented Data Engineer to work on the Web Scraping team. Their primary responsibility is to design, implement, and maintain data pipelines that transform data, adding value to it in the process. In carrying out the objective of this position, a successful candidate will be committed to our values by working with gratitude, operating as a team, and partnering with our customers.

Core Responsibilities

  • Create and improve processes, tools, workflows, and resilient data architecture to scrape web content.
  • Manage data accuracy and quality.
  • Identify and rectify any issues with breaks as well as scale scrapers as needed
Organizational Relationships

  • This position reports to the Senior Director of Data Innovation
Knowledge, Skills, and Abilities

  • Web scraping experience (Scrapy or similar technologies)
  • Solid understanding of web technologies (HTML, JavaScript, CSS, XPath, JSON, etc)
  • Experience with ETL and creating data pipelines
  • Basic Linux and Git experience
  • Python
  • Scrapy Python library
  • AWS/AWS Batch
  • Familiarity with data processing tools (pandas, regex, SQL)

  • Attention to detail
  • Self-starter
  • Curious
  • Customer Oriented
  • Organized
  • Problem solving

Credentials and Experience

  • 2+ years Python experience, preferred
  • Professional experience with web scraping preferred
  • Bachelor’s degree, preferred
Lightcast is proud to be an equal opportunity workplace and is committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. Lightcast has always been, and always will be, committed to diversity, equity, and inclusion. We seek dynamic professionals from all backgrounds to join our team, and we encourage our employees to bring their authentic, original, and best selves to work.

Please let the company know that you found this position on this Job Board as a way to support us, so we can keep posting cool jobs.

Similar jobs

Browse All Jobs
MedPro Group
November 26, 2022
Kamo Placements
November 26, 2022

Product Data Engineer - 7 Years Contract

Phanda Personnel
November 26, 2022

Product Data Engineer