About Scispot

Scispot is building the world's first scientific ETL for unstructured, semi-structured and structured data. Scispot staging lakehouse connects with all the popular R&D sources such as Benchling, Quartzy, Lab Instruments, and others & converts them into machine-readable data to support downstream pipelines.

The Opportunity:

We're searching for a seasoned DevOps Engineer with experience in Data Engineering. The right candidate will have an opportunity to become a senior architect as we are moving really fast and scaling our Infrastructure. In this role, you'll be instrumental in ensuring the scalability and stability of our infrastructure as we continue to grow. This position involves a balance of maintaining robust AWS infrastructure, implementing DevOps practices, and handling ETL tasks.

What You'll Do:

• Design, manage, and optimize a scalable, secure AWS infrastructure focusing on EC2, ECS, S3, and Lambda services.
• Implement strategies and solutions for scaling and stabilizing infrastructure.
• Integrate with various lab instruments using protocols such as ASTM, HL7.
• Build, configure, and maintain ETL pipelines for various Lab Instruments and Scispot cloud.
• Maintain high-availability systems, targeting 99.99% uptime.
• Drive the adoption of Infrastructure as Code (IAC) practices.
• Work with pub-sub architecture and Rabbit MQ.
• Develop and maintain ETL processes.
• Contribute to AI infrastructure using RAG and embeddings.
• Collaborate with cross-functional teams to translate their needs into technical requirements.

What You'll Need:

• Proven experience as a DevOps Engineer focusing on infrastructure management and scalability.
• Experience with Bioinformatics pipelines
• Extensive experience with AWS infrastructure including services like EC2, ECS, API Gateway, S3, and Lambda.
• Strong familiarity with maintaining high-availability systems.
• Proficiency in Infrastructure as Code (IAC) practices.
• Experience in a Data Engineering or similar role involving ETL processes.
• Experience with Rabbit MQ and pub-sub architectures.
• Familiarity with ETL tools.
• Experience with Elastic Search
• Familiarity with Java and Spring Boot is a plus.