The Role
Design and implement BigQuery-based medallion (lakehouse) architectures and ingestion pipelines on GCP; define datasets, partitioning, clustering, and cost controls; support batch/incremental/on-demand processing; implement data quality, monitoring, and search/AI integration; produce architecture docs and work with engineering, data, DevOps, and product teams for handover.
Summary Generated by Built In
This is a remote position.
Key Responsibilities
* Assess the existing data pipeline, GCP architecture, data sources, and current limitations.
* Design a BigQuery-based medallion architecture: Bronze, Silver, Gold, and Serving layers.
* Design and/or implement data ingestion pipelines from Cloud Storage and Cloud SQL / MySQL into BigQuery.
* Define BigQuery datasets, tables, partitioning, clustering, and cost-control strategies.
* Support batch, incremental, and on-demand processing patterns.
* Design curated and BI-ready datasets for analytics and reporting.
* Support data quality checks, validation, logging, monitoring, and reprocessing patterns.
* Define integration patterns for Elasticsearch or other search / serving layers.
* Support architecture for AI enrichment, embeddings, semantic search, and media metadata where required.
* Prepare technical documentation, architecture diagrams, implementation roadmap, and handover materials.
* Work closely with client engineering, data, DevOps, and product teams.
Requirements
Required Skills
* Strong hands-on experience with Google Cloud data services.
* Strong BigQuery experience, including modeling, optimization, partitioning, clustering, and cost control.
* Experience with Cloud Storage ingestion patterns.
* Experience with Cloud SQL / MySQL to BigQuery integration.
* Experience building batch and incremental data pipelines.
* Experience with Dataform, dbt, Cloud Composer / Airflow, Dataflow, Cloud Run, or similar tools.
* Strong SQL and data modeling skills.
* Good understanding of lakehouse / medallion architecture.
* Experience with data quality, metadata, lineage, logging, and monitoring.
* Ability to work in an ambiguous consulting environment and translate high-level requirements into practical implementation plans.
* Strong communication skills and client-facing experience.
Preferred Skills
* Experience with Elasticsearch or search index publishing.
* Experience with Power BI-ready datasets or analytical serving layers.
* Experience with Vertex AI, Gemini, embeddings, vector search, or semantic search.
* Experience with media, social media, marketing, campaign, influencer, or audience data.
* Experience with Dataplex, Data Catalog, IAM, policy tags, row-level security, or column-level security.
* Experience with CI/CD for data pipelines.
Benefits
WFH or Hybrid and competitive salary
Skills Required
- Hands-on experience with Google Cloud data services (BigQuery, Dataflow, Cloud Storage, Cloud SQL)
- Strong BigQuery modeling, optimization, partitioning, clustering, and cost-control experience
- Experience with Cloud Storage ingestion patterns and Cloud SQL/MySQL to BigQuery integration
- Experience building batch and incremental data pipelines
- Experience with Dataform, dbt, Cloud Composer/Airflow, Dataflow, Cloud Run or similar tools
- Strong SQL and data modeling skills
- Good understanding of lakehouse / medallion architecture (Bronze/Silver/Gold/Serving)
- Experience with data quality, metadata, lineage, logging, monitoring, and reprocessing patterns
- Ability to translate ambiguous consulting requirements into practical implementation plans
- Strong communication skills and client-facing experience
- Experience with Elasticsearch or search index publishing
- Experience with Power BI-ready datasets or analytical serving layers
- Experience with Vertex AI, Gemini, embeddings, vector/semantic search
- Experience with Dataplex, Data Catalog, IAM, policy tags, row-level or column-level security
- Experience with CI/CD for data pipelines
- Experience with media, social media, marketing, campaign, influencer, or audience data
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
iScale Solutions is a full-service outsourcing company providing staff augmentation, IT services, business process outsourcing, digital marketing, creative solutions, and managed IT support.
