Data Engineer

Company:
Location: Remote

*** Mention DataYoshi when applying ***

Two Six is looking to bring on a Data Engineer to join our PULSE team. The Pulse Platform is used to engage with populations around the world to provide insight into their thinking and engage them in two-way conversations. Pulse also has the added capability of scraping open data across the surface, deep, and dark web to provide a robust view of the Information Environment in which one is operating.

Responsibilities Include:

  • Create, document, and implement web scraping processes to gather data on various public platforms
  • Integrate data collection capabilities into the Pulse Platform
  • Decompose, parse, and extract relevant content from public pages into structured data
  • Author tests scripts to verify and validate data
  • Manage data accuracy and quality
  • Maintain data collection capabilities as target platforms change and evolve

Minimum Qualifications:

  • Bachelor’s Degree in Computer Science, Engineering, or similar. Additional 4 years of work experience can be considered a substitute
  • Minimum of 2 years web scraping experience with Scrapy or similar tool (BeautifulSoup, Requests, Fiddler, Postman)
  • Minimum of 2 years of experience utilizing Git/Github or a similar version control tool
  • Minimum of 1 year with experience in headless browser automation (PhantomJS, Selenium, Robot Framework, Puppeteer)

Nice If You Have:

  • Experience with large-scale and distributed web crawling frameworks (Scrapy Cluster, Frontera, Scrapyd)
  • Familiarity with Docker and containerization
  • Experience with Redis, Apache Kafka, or Apache Zookeeper
  • Working knowledge of HTML, Javascript, and other commonly used web technologies

Clearance/Citizenship:

  • US Citizenship is Required; applicants selected may be subjected to a security investigation

Two Six Technologies is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment at Two Six Technologies without regard to race, color, religion, national origin, sex, age, physical and mental disability, sexual orientation, gender identity or expression, genetic information, veteran, marital, pregnancy or citizenship status; or any other status prohibited by applicable national, federal, state or local law.

If you are interested in applying for employment with Two Six Technologies and require an accommodation, please contact Human Resources at Two Six Technologies by calling 703-543-9662 or sending an email to hr@twosixtech.com. Information provided will be kept confidential and used only to the extent required to provide needed reasonable accommodations.

*** Mention DataYoshi when applying ***

Offers you may like...

  • True Rx Management Services

    Data Engineer
    Washington, IN 47501
  • Candid.

    Associate Data Engineer
    New York, NY
  • Empower Retirement

    Sr. Data Engineer
    Redwood City, CA
  • Bayer

    Data Engineer
    Chesterfield, MO 63005
  • Federal Reserve Bank of San Francisco

    Sr Data Engineer
    San Francisco, CA 94105