CONVO is seeking a Senior Data Engineer to build reliable data pipelines and connectors powering its CPG-focused agentic platform. You’ll develop scalable batch and streaming pipelines using Python, Spark, Databricks, Kafka, dbt, and Delta Lake, transforming source data into governed RAW and CUBE layers while ensuring data quality, performance, observability, and reliability. You’ll also build data-product and feature-serving interfaces that enable ML models, agents, and downstream applications to consume trusted data efficiently.
Technical mission- Build reliable connectors and RAW-to-CUBE pipelines across batch and streaming paths, and expose governed data and ML features to downstream services.
- Develop Python/Spark ingestion and transformation pipelines from source systems into RAW and Enriched CUBE layers.
- Implement streaming paths alongside scheduled workloads.
- Create dbt/SQL transformations, data-quality controls and contract-validation checks.
- Optimize Delta Lake layout, SQL performance, partitioning and incremental processing.
- Build feature-serving and data-product interfaces for models, agents and applications.
- Instrument pipelines for lineage, freshness, throughput, failure recovery and cost.
- Minimum experience: 5+ years in data engineering, including 3+ years delivering production Spark or python pipelines.
- Advanced Python and production Apache Spark/Databricks development.
- Kafka or comparable event-streaming technology.
- Strong SQL performance tuning and dimensional/data-product implementation.
- dbt and Delta Lake, including incremental patterns and table optimization.
- Batch and streaming reliability patterns: idempotency, checkpointing, replay and late-arriving data.
- Automated data quality, observability and schema-contract testing.
- ML feature stores or online/offline feature consistency.
- CPG, retail, ERP, POS or syndicated-data pipelines.
- Kubernetes-based data workloads and cloud cost optimization.
- Production connectors and RAW-to-CUBE pipelines with automated tests.
- Batch/streaming operational dashboards and recovery procedures.
- Documented data products and feature-serving interfaces.
- Performance and cost baselines for the implemented workloads.
- Works under the Data Architect with source-system owners, ML/optimization teams, platform infrastructure and downstream application teams.
Skills Required
- 5+ years of experience in data engineering
- 3+ years delivering production Apache Spark or Python pipelines
- Advanced Python development
- Production Apache Spark or Databricks development
- Kafka or comparable event-streaming technology experience
- Strong SQL performance tuning and dimensional or data-product implementation experience
- dbt and Delta Lake experience, including incremental patterns and table optimization
- Experience with batch and streaming reliability patterns, including idempotency, checkpointing, replay, and late-arriving data
- Experience with automated data quality, observability, and schema-contract testing
- Experience with ML feature stores or online/offline feature consistency
- CPG, retail, ERP, POS, or syndicated-data pipeline experience
- Kubernetes-based data workloads and cloud cost optimization experience
What We Do
Convo provides an enterprise social collaboration platform that enables easy, secure conversations between desk / non-desk workers to accelerate company productivity and engagement. Unlike existing email-focused or chat-centric collaboration platforms, only Convo combines the ease of social networks with rich collaboration capabilities to simplify and optimize work interactions for all employees -- no email required.
Gallery
