Senior Data Engineer
About the Role
You will design, develop, and optimize scalable data pipelines and backend ETL infrastructure that ingest blockchain and social data. You will lead end-to-end ETL design, manage database ingestion and production deployments, and build real-time analytics features on a large data lake. You will work with Python, PySpark, Iceberg, Trino, StarRocks, Postgres, MongoDB and cloud services to deliver well-tested, production-ready solutions.
Requirements
- At least 4 years of experience in data infrastructure, big data, or data engineering.
- Proficiency with Apache Iceberg.
- Expertise in PySpark.
- Strong Python skills.
- Experience with SQL and NoSQL databases such as Postgres and MongoDB.
- Familiarity with real-time analytics databases (ClickHouse, Apache Druid, Firebolt, SingleStore, StarRocks or equivalent).
- Experience working in Linux environments.
- Proven experience building and deploying code in production.
- Cloud computing experience (AWS, GCP, Azure or similar).
Responsibilities
- Architect and implement data pipelines.
- Design and lead end-to-end ETLs, data structures, and database ingestion and management.
- Develop and enhance real-time analytics on a blockchain and social data lake.
- Build and maintain backend ETL infrastructure and production deployments.
- Collaborate with team members to deliver tested features and oversee processes from design to production.