Responsibilities
- Data Architecture & Modeling: Design, implement, and maintain scalable data pipelines and architectures for batch and real-time processing.
- ETL & Data Integration: Develop and optimize ETL/ELT processes to ingest structured and unstructured data from multiple sources.
- Data Warehousing: Build and maintain data warehouses and data lakes; ensure efficient storage, retrieval, and partitioning strategies.
- Data Quality & Governance: Implement data validation, monitoring, lineage, and access controls to ensure data accuracy, security, and compliance.
- Performance Optimization: Tune queries, optimize pipelines, and improve reliability and scalability of data workflows.
- Collaboration: Work closely with Data Scientists, Analysts, and Engineers to support business needs with reliable datasets.
- Documentation & Knowledge Sharing: Maintain clear documentation, support knowledge transfer, and contribute to best practices across teams.
Tech Stack
- Data Processing: Apache Spark, Databricks, Apache Beam, or Flink
- ETL/Orchestration: Airflow, dbt, Luigi, Prefect
- Databases/Warehousing: Snowflake, BigQuery, Redshift, PostgreSQL
- Data Lakes & Storage: S3, Azure Data Lake, GCS
- Programming: Python, SQL, Scala
- IaC & Automation: Terraform, Kubernetes (for data services)
- Observability: Prometheus, Grafana, or similar for pipeline monitoring
- Security & IAM: Experience with role-based access, encryption, GDPR/PII compliance
Requirements
- Proven experience as a Data Engineer, with hands-on expertise in building and maintaining data pipelines.
- Strong SQL and Python skills; ability to work with large-scale datasets.
- Experience with data warehouses, data lakes, and modern ETL/ELT frameworks.
- Familiarity with cloud platforms (Azure, and IaC tools.
- Knowledge of data governance, quality frameworks, and compliance requirements.
- Strong problem-solving and debugging skills in distributed data environments.
- Ability to collaborate effectively across teams and communicate technical topics clearly.
Top Skills
What We Do
P3 enables our clients to succeed in their business by delivering tangible value.
Founded in 1996 as a spin-off from a globally-recognized, leading technical university, P3 began with the goal of implementing an innovative new process and has remained focused on serving clients from innovation to implementation. P3 has around 1000 employees across 20 locations globally.
Who we work with
We work within organizations and teams to develop and implement innovative solutions to complex technology challenges. Our clients are open-minded, secure leaders who seek solutions. They have the courage to bring in outside experts who can provide long-term value to their organization.
What we do
Our fresh, innovative application of technical and managerial best practices puts tailored solutions in place – not just on paper – with continuous guidance until a project or process is implemented, and tangible value has been achieved.
.jpg)
