Staff Software Engineer, Data Incrementalization

TRM is on a mission to build a safer financial system for billions of people. We deliver a blockchain intelligence data platform to financial institutions, crypto companies, and governments to fight cryptocurrency fraud and financial crime. We consider our business — and our profit — as a way to move towards our mission sustainably and at scale.

The Data Platform team collaborates with an experienced group of data scientists, engineers, and product managers to build highly available and scalable data infrastructure for TRM's products and services. As a Senior Engineer on the Core Data team, you will help to develop internal tools to empower the Data Science and Machine Learning Engineers to refine Blockchain Data into Blockchain Intelligence at petabyte scale.

The impact you’ll have here:

• Design and build our Cloud Data Warehouse with a focus on incremental updates to improve cost efficiency and scalability.
• Research innovative methods to incrementally optimize data processing, storage, and retrieval to support efficient data analytics and insights.
• Develop and maintain ETL pipelines that transform and incrementally process petabytes of structured and unstructured data to enable data-driven decision-making.
• Collaborate with cross-functional teams to design and implement new data models and tools focused on accelerating innovation through incremental updates.
• Continuously monitor and optimize the Data Platform's performance, focusing on enhancing cost efficiency, scalability, and reliability.

What we’re looking for:

• Bachelor's degree (or equivalent) in Computer Science or a related field.
• 5+ years of experience in building distributed system architecture, with a particular focus on incremental updates from inception to production.
• Strong programming skills in Python and SQL.
• Deep technical expertise in advanced data structures and algorithms for incremental updating of data stores (e.g., Graphs, Trees, Hash Maps).
• Comprehensive knowledge across all facets of data engineering, including:
  • Implementing and managing incremental updates in data stores like BigQuery, Snowflake, RedShift, Athena, Hive, and Postgres.
  • Orchestrating data pipelines and workflows focused on incremental processing using tools such as Airflow, DBT, Luigi, Azkaban, and Storm.
  • Developing and optimizing data processing technologies and streaming workflows for incremental updates (e.g., Spark, Kafka, Flink).
  • Deploying and monitoring scalable, incremental update systems in public cloud environments (e.g., Docker, Terraform, Kubernetes, Datadog).
  • Expertise in loading, querying, and transforming large datasets with a focus on efficiency and incremental growth.

About TRM's Engineering Levels:

Engineer: Responsible for helping to define project milestones and executing small decision decisions independently with the appropriate tradeoffs between simplicity, readability, and performance. Provides mentorship to junior engineers, and enhances operational excellence through tech debt reduction and knowledge sharing.

Senior Engineer: Successfully designs and documents system improvements and features for an OKR/project from the ground up. Consistently delivers efficient and reusable systems, optimizes team throughput with appropriate tradeoffs, mentors team members, and enhances cross-team collaboration through documentation and knowledge sharing.

Staff Engineer: Drives scoping and execution of one or more OKRs/projects that impact multiple teams. Partners with stakeholders to set the team vision and technical roadmaps for one or more products. Is a role model and mentor to the entire engineering organization. Ensures system health and quality with operational reviews, testing strategies, and monitoring rigor.