Senior Software Engineer, Data Platform
About the Role
You will build and maintain highly reliable data services that integrate with many blockchains, develop complex ETL pipelines that process petabyte-scale structured and unstructured data in real time, and design data models to support sub-second querying. You will deploy, monitor, and optimize large database clusters for performance and availability, automate routine scaling and maintenance tasks, and implement observability and monitoring solutions. You will collaborate with data scientists, backend engineers, and product managers to translate requirements into production-ready data systems and ship fast solutions when playbooks do not exist.
Requirements
- Bachelor's degree or equivalent in Computer Science or related field
- 8+ years of experience architecting distributed system architecture and delivering production deployments
- Exceptional programming skills in Python
- Proficiency with SQL or SparkSQL
- Experience with ClickHouse, Elasticsearch, Postgres, Redis, and Neo4j
- Familiarity with Airflow, DBT, Luigi, Azkaban, or Storm
- Experience with Spark, Kafka, or Flink for data processing and streaming
- Competence deploying and monitoring infrastructure using Docker, Terraform, Kubernetes, and Datadog
- Proven ability to load, query, and transform very large datasets
Responsibilities
- Build highly reliable data services to integrate with multiple blockchains
- Develop complex ETL pipelines to transform and process petabyte-scale data in real time
- Design and architect data models for optimal storage and sub-second retrieval
- Oversee deployment and monitoring of large database clusters with focus on performance and high availability
- Collaborate with data scientists, backend engineers, and product managers to implement data models and services
- Automate scaling and maintenance tasks to enable self-serve operations
- Identify and deliver fast MVP solutions to compress timelines
- Implement observability and monitoring for data infrastructure
Benefits
- Remote-first work
- Flexible hours and async-friendly communication