Senior Data Engineer
We are looking for a hands-on Senior Data Platform Engineer to own, scale, and modernize the company's data platform, ensuring it is scalable, reliable, secure, cost-efficient, and capable of supporting both operational and analytical workloads.. You will maintain our current platform while actively helping us transition toward a lakehouse architecture: keeping raw data in S3, utilizing Redshift external schemas, and leveraging dbt for optimized transformations.
Key Responsibilities
- Platform & Infrastructure Management: Own the deployment, scaling, and orchestration of our core data stack using AWS Managed Workflows for Apache Airflow (MWAA). Design and evolve the overall data platform architecture, making thoughtful tradeoffs around scalability, reliability, cost, and maintainability.
- Lakehouse Engineering: Configure and optimize raw data storage in S3, managing Redshift external schemas (Spectrum) and AWS Glue Data Catalog. Define efficient partitioning strategies, leverage columnar file formats (Parquet), and optimize storage layouts for query performance, scalability, and cost.
- Data Architecture & Modeling (lakehouse, dimensional models, Redshift). Design scalable analytical data models using dimensional modeling and lakehouse best practices to support reporting, analytics, and downstream consumers.
- Data Transformation: Manage and optimize dbt to ensure all raw data from external schemas is efficiently transformed, tested, and materialized strictly as processed data within Amazon Redshift.
- Data Reliability & Operations (monitoring, testing, incident response). Build observability into the data platform through monitoring, logging, alerting, and operational dashboards.
- Performance & Cost Optimization (Redshift tuning, S3 layout, workloads)
- Governance & Collaboration (security, documentation, working with stakeholders)
- Modernization Pipeline: Help design and implement the transition toward event-based ingestion into S3.
- CI/CD & DevOps: Implement and maintain robust CI/CD deployment pipelines for our dbt projects, Airflow DAGs, and infrastructure.
- BI Support: Ensure high performance, access control, and uptime for Metabase connecting to Redshift.
Technical Skills & Requirements
- Core AWS Stack: Extensive hands-on experience deploying and managing AWS data services, specifically MWAA (Airflow), Redshift / Redshift Spectrum, S3, IAM, and Glue.
- Data Transformation: Advanced proficiency with dbt (structuring dbt projects, configuring sources, writing custom macros, and optimizing incremental models).
- Infrastructure as Code (IaC): Solid hands-on experience deploying AWS data platform components using Terraform or AWS CloudFormation.
- SQL & Performance Tuning: Expert-level SQL skills, with a deep understanding of Redshift distribution/sort keys and optimizing queries across external schemas.
- Programming: Strong Python experience for developing Airflow DAGs, automation, integrations, and data engineering tooling
- Data Storage & Lakehouse: Experience designing efficient data lakes using Parquet, partitioning strategies, metadata catalogs, and external table technologies such as Redshift Spectrum..
- Modern Lakehouse Technologies: Experience with open table formats such as Apache Iceberg, Delta Lake, or Apache Hudi, including an understanding of ACID transactions, schema evolution, time travel, and metadata management.
Bonus / Nice-to-Have
- Experience with Apache Kafka or AWS MSK for event-driven data streaming and real-time ingestion into S3.
- Streaming & Event-Driven Architecture: Experience designing resilient streaming pipelines with appropriate delivery guarantees, reconciliation, backfill strategies, and schema contract management.
- Familiarity with modern analytical query engines such as DuckDB or MotherDuck is a plus.
Skills Required
- Hands-on experience with AWS Managed Workflows for Apache Airflow (MWAA)
- Extensive experience with Amazon Redshift and Redshift Spectrum (external schemas)
- Deep experience with Amazon S3, IAM, and AWS Glue Data Catalog
- Advanced proficiency with dbt (project structure, macros, incremental models, testing)
- Infrastructure as Code using Terraform or AWS CloudFormation
- Expert-level SQL and Redshift performance tuning (distribution/sort keys, query optimization)
- Strong Python skills for Airflow DAGs, automation, and tooling
- Designing data lakes with Parquet, partitioning strategies, metadata catalogs, and external table tech
- Familiarity with modern open table formats: Apache Iceberg, Delta Lake, Apache Hudi
- Implementing CI/CD pipelines for dbt, Airflow DAGs, and infrastructure
- Experience with event-driven ingestion and streaming architectures (bonus: Kafka or AWS MSK)
- Experience designing resilient streaming pipelines, delivery guarantees, reconciliation, and backfill strategies
- Familiarity with analytical query engines such as DuckDB or MotherDuck
What We Do
Redefine the way money works for you, with Fasset. Fasset lets you seamlessly and securely manage all of your digital and real-world financial assets - from wherever you are, in your currency of choice. We believe in democratizing access to global assets, and that everyone, everywhere should be able to use their hard-earned money to buy high-quality, globally-traded assets to build and preserve their wealth. Through Fasset, you can: ⚫ Invest in both digital as well as real-world assets like stocks, bonds, commodities and precious metals. ⚫ Tap into our trading platform and take advantage of a wide range of tokenized assets, including real-time market data. ⚫ Connect to a digital wallet that makes shopping and transferring funds user-friendly, with integrated payment solutions for easy conversion. ⚫ Transfer or receive funds securely and conveniently from wherever you are worldwide with the Fasset Connect Network. The security of our customers’ funds, data and earnings are our top priority. We are regulated by the Virtual Assets Regulatory Authority (VARA). No aspect of this platform is intended to provide, or should be interpreted as providing, any investment, tax, legal, financial, or other advice of any kind. The materials presented on this platform should not be considered a substitute for professional advice. If you decide to engage in transactions based on any material found on this platform, the decisions, transactions, and any consequences that arise are solely your responsibility. While other users may share investment advice or opinions, please be aware that such advice or opinions are merely conversational exchanges between individuals who may be anonymous or unidentifiable. Fasset explicitly does not provide investment advice directly, indirectly, implicitly, or in any manner whatsoever. It is recommended that you use any information obtained from this platform as a starting point for your own independent research.
