Data Engineer (Kafka Admin)

Company:
Location: Mumbai, Maharashtra

*** Mention DataYoshi when applying ***

  • Setting up and administering Kafka cluster on premise and on cloud platforms
  • Handle all Kafka environment builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring
  • Manage Kafka brokers, zookeepers, Kafka connect, schema registry, KSQL, Rest proxy and Kafka Control center
  • Ensure optimum performance, high availability and stability of cluster
  • Create topics, setup redundancy cluster, deploy monitoring tools, alerts and has good knowledge of best practices
  • Provide administration and operations of the Kafka platform like provisioning, access lists Kerberos and SSL configurations

Qualifications

You have a M.Sc. or Bachelors in a quantitative field (Information Systems, Computer Science, Physics, Mathematics, etc.) and demonstrated industry experience in data engineering – 4 to 5 years


Core Technical Requirements:

  • Experience as Unix/Linux Systems Administrator
  • Atleast 3 years of professional experience in Kafka Admin
  • Hands-on experience with seting up Kafka clusters on premise and on cloud platforms
  • Experience in Kafka build pipelines using Ansible, Cloud formation templates, shells etc
  • Experience in Jenkins, GitHub
  • Experience in implementing security & authorization (permission based) on Kafka cluster
  • Experience in open source Kafka, zookeepers, Kafka connect, schema registry Avro schemas


Supportive Requirements:

  • Version Control Technologies – Experience working with version control (e.g., Git, SVN etc.)
  • SDLC Model – Knowledge of SDLC models (e.g., Waterfall, Agile, CI/CD)
  • Containerization - Experience with container-based infrastructures (Docker/Kubernetes)
  • Experience in the development of API services (REST, e.g. with Flask)
  • Basic knowledge of common libraries and frameworks (NumPy, pandas, scikit-learn etc.).
  • Demonstrable knowledge of applying Data Engineering best practices (coding practices, testing (unit, system, acceptance, black box etc.), code review)

Additional Information

Are you a game changer and want to shape the future with us?


  • An exciting international environment of projects, customers and consultants
  • Extensive training opportunities in data analytics and data engineering
  • Competitive salary

*** Mention DataYoshi when applying ***

Offers you may like...

  • EAB

    Data Engineer, Commercial Analytics
    Washington, DC 20036
  • NN Tech, LLC

    Data Engineer
    Remote
  • The University of Pittsburgh

    Data Engineer (remote)
    Pittsburgh, PA
  • Exact Sciences Corporation

    Data Engineer II
    Madison, WI 53711
  • Spotify

    Data Engineer, Insights Platform
    New York, NY