Data Engineer - Remote

Job Description

Job Description

Requirements: US Citizenship and Ability to Clear a Drug and Background Check

Location: Remote

Full Time Annual Salary - 120K-140K

Project Details:

This position is on an initiative is dedicated to the Observation, Collection, and Analysis of Social Media data, underpinning the importance of accurate and reliable information for societal well-being. In today's digital age, misinformation spread via social media can negatively impact various sectors, including national security, crisis management, and public health.

The project aims to develop web applications that leverage Artificial Intelligence (AI) and Machine Learning (ML) models. These tools will enable researchers and stakeholders in the U.S. and Europe to analyze social media data meticulously, facilitating data-driven decisions for the public good.

Job Description:

We are seeking a skilled Data Systems Engineer with a strong background in data systems design and engineering. The ideal candidate will have a solid foundation in Python programming, extensive experience with distributed systems, and a deep understanding of data storage technologies. Practical experience in developing solutions using data lakes and data mesh architectures, as well as proficiency with relational databases and Elasticsearch, is essential. This role requires expertise in tuning data systems for enhanced performance and reliability.

Key Responsibilities:

• Design and Implement Data Storage Solutions:
• Develop and optimize data storage solutions, utilizing data lakes and data mesh architectures to facilitate scalable data accessibility and analysis.
• Manage relational databases and Elasticsearch for structured data storage and efficient querying.
• Distributed Systems Management:
• Work with distributed systems to manage and process large datasets efficiently.
• Ensure data systems are optimized for improved performance and reliability, ensuring smooth operation and quick access to critical information.
• Data Integration and Engineering:
• Extract and preprocess data from various sources, including APIs and web scraping, to feed into data systems.
• Collaborate with cross-functional teams to integrate data engineering solutions into broader projects and applications.
• System Optimization and Performance Tuning:
• Continuously monitor and tune data systems to ensure high performance and reliability.
• Implement best practices for data storage, retrieval, and processing to maximize efficiency.
• Machine Learning Frameworks:
• Utilize PyTorch and TensorFlow for developing and deploying machine learning models on both CPU and GPU targets, focusing on data systems integration rather than direct model development.
• Mentorship and Team Development:
• Mentor younger team members and interns, providing guidance and support to help them develop their technical skills and understanding of data systems engineering.
• Foster a collaborative and inclusive team environment, encouraging continuous learning and professional growth.
  

Qualifications:

• Programming and Development:
• Proficient in Python, with a strong background in software development and data systems engineering.
• Distributed Systems:
• Experienced with distributed systems and their application in data processing and analysis.
• Data Storage Technologies:
• Solid understanding of data storage technologies, with hands-on experience in data lakes and data mesh architectures.
• Familiarity with relational databases and Elasticsearch.
• Performance Tuning:
• Proven track record of tuning data systems for performance and reliability.
• Machine Learning Integration:
• Development experience with PyTorch and TensorFlow, particularly for integrating machine learning frameworks with data systems on both CPU and GPU platforms.
• Data Extraction and Preprocessing:
• Skilled in extracting data from APIs and web scraping.
• Analytical Skills:
• Excellent problem-solving abilities and strong analytical skills.
• Mentorship:
• Demonstrated willingness and capability to mentor younger team members and interns.
  

Attendee panel closed