Lead Data Engineer (Azure)

Reposted 4 Days Ago
Be an Early Applicant
Hiring Remotely in Costa Rica
Remote
Senior level
Software
The Role
Lead design and delivery of Azure-based data lakehouse platforms and ETL/ELT pipelines. Architect Microsoft Fabric reporting, ensure PII-safe multi-tenant analytics, migrate legacy systems, build AI-ready pipelines, mentor engineers, and drive governance, performance optimization, and adoption of GenAI-assisted tooling.
Summary Generated by Built In
3Pillar is an AI transformation partner on a mission to help enterprises build the AI-native products and intelligent agents that will define the next era of business. With teams across North America, Europe, Latin America, and Asia, we work with the most ambitious companies in financial services, healthcare, media, and technology — helping them move faster, modernize boldly, and compete on their own terms. Our HelixAI platform and Helix Pods delivery model put our engineers at the center of real agentic transformation — doing work that is open, portable, and built to last. We are building the future of enterprise AI. 
 

Lead, Data Engineer

Embark on an exciting journey into the realm of software product development with 3Pillar! We extend an invitation for you to join our team and gear up for a thrilling adventure. At 3Pillar, our focus is on leveraging cutting-edge technologies that revolutionize industries by enabling data-driven decision-making. As a Lead, Data Engineering, you will hold a crucial position within our dynamic team, actively contributing to thrilling projects that reshape data analytics for our clients, providing them with a competitive advantage in their respective industries.

If your passion for data engineering solutions that make a real-world impact, consider this your pass to the captivating world of Data Science and Engineering! 🌍🔥

Key Responsibilities

  • Understand client requirements and lead the creation of robust data platforms leveraging Data Lakehouse architectures, serving as the primary subject matter expert for the Azure data ecosystem (Microsoft Fabric, Azure SQL, and Power BI).

  • Design and implement scalable data models, ETL/ELT pipelines, and analytics layers using modern cloud ecosystems.

  • Architect Microsoft Fabric reporting solutions, establishing medallion layering, managing capacity planning, ensuring freshness governance, and configuring CDC/mirroring from Azure SQL into Fabric.

  • Enforce strict data-layer segregation ensuring PII is excluded by construction, maintaining multi-tenant isolation within the analytics layer, and strictly adhering to 10+ year regulated data retention policies.

  • Lead complex migration efforts off legacy systems (e.g., AirTable, Smartsheet) to Microsoft Fabric, effectively managing mid-program form and rule drift, and establishing reliable, per-client export channels.

  • Lead end-to-end analytics projects — from requirement gathering to deployment.

  • Mentor junior team members, perform code reviews, and ensure adherence to quality standards.
    Work closely with cross-functional teams (engineering, product, business) to align data strategy with business goals.

  • Develop and optimize complex SQL queries, Power BI semantic models, dashboards, and reports (transitioning from legacy LookML/BigQuery paradigms).

  • Troubleshoot performance bottlenecks, data inconsistencies, and integration challenges.

  • Stay up to date with advancements in the analytics and cloud data ecosystem.

  • Drive innovation in reporting and analytics capabilities.
    Recommend and implement new tools, frameworks, and practices to enhance the data platform.

  • Define best practices, coding standards, and governance for analytics projects.

  • Monitor data systems performance and implement optimization strategies.

  • Ensure data accuracy, integrity, privacy, security, and compliance through quality control procedures.

  • Build and oversee AI-ready data pipelines, focused on feeding scrubbed, segmented data to assistive AI (embeddings, vector stores, and feature stores) and integrating GenAI into analytics and data products.

  • Drive adoption of AI-assisted data engineering across the team: copilots for SQL/transformation/pipeline code, plus automated testing and documentation.

  • Define governance and quality standards for data consumed by LLMs and agents.

Minimum Qualifications

  • Demonstrated expertise with a minimum of 7 - 10  years of relevant experience in data engineering and business intelligence domain.

  • Strong primary expertise in the Azure Data Stack, specifically Azure SQL, Microsoft Fabric, and Power BI (semantic layer design, robust dashboard development), with a minimum of two years of hands-on experience in these specific tools.

  • Advanced SQL skills and a solid understanding of modern data architecture, including Data Lakehouse concepts, medallion architecture, star/snowflake schemas, and dimensional modeling.

  • Proficiency in Python for data engineering, ETL/ELT workflows, and automation tasks.

  • Experience designing and maintaining enterprise-grade data pipelines — specifically in Azure environments.

  • Proven track record of leading analytics projects and delivering scalable data solutions.

  • Ability to function as a player-coach — leading while also contributing hands-on.

  • Solid understanding of data governance, security (especially around PII and multi-tenant architectures), and performance optimization practices.

  • Strong communication skills and ability to translate complex technical concepts for business stakeholders.

Additional Experience Desired

  • Experience working with DBT 

  • Experience working with other BI Tools

  • Any experience of working with Gen AI tools or technologies

Benefits

  • Medical Insurance benefits as per company policy. 

  • Life Insurance as per company policy

  • 12 days of paid vacation, sick leave and paid holidays as per local law

  • Paternity and maternity leave as per as per local law

  • Marriage, bereavement and graduation leaves as per company policy

  • Sick leave and paid holidays as per local law

  • Christmas bonus as per local law

  • Saving Fund “Asociacion Solidarista”

  • Beach house (Managed by the Asociacion Solidarista)

  • Discounts in local shops

  • Direct deposit of payroll.

  • Paid professional certifications

What is it like working for 3Pillar Global?

    At 3Pillar, we offer a world of opportunity:

  • Imagine a flexible work environment – whether it's the office, your home, or a blend of both. From interviews to onboarding, we embody a remote-first approach. 

  • You will be part of a global team, learning from top talent around the world and across cultures, speaking English everyday. Our global workforce enables our team to leverage global resources to accomplish our work in efficient and effective teams. 

  • We’re big on your well-being – as a company, we spend a whole trimester in our annual cycle focused on wellbeing. Whether it is taking advantage of fitness offerings, mental health plans (country-dependent), or simply leveraging generous time off, we want all of our team members operating at their best.

  • Our professional services model enables us to accelerate career growth and development opportunities - across projects, offerings, and industries.

  • We are an equal opportunity employer. It goes without saying that we live by values like Intrinsic Dignity and Open Collaboration to create cutting-edge technology AND reinforce our commitment to diversity - globally and locally. 

  • Join us and be a part of a global tech community! 🌍💼  Check out our Linkedin site and Careers page to learn more about what it’s like to be part of our #oneteam! 

Skills Required

  • 7-10 years of relevant experience in data engineering and business intelligence
  • Expertise with the Azure Data Stack (Azure SQL, Microsoft Fabric) and Power BI with at least two years hands-on experience
  • Advanced SQL skills and strong understanding of Data Lakehouse concepts, medallion architecture, dimensional modeling, star/snowflake schemas
  • Proficiency in Python for data engineering, ETL/ELT workflows, and automation
  • Experience designing and maintaining enterprise-grade data pipelines in Azure environments
  • Proven track record leading analytics projects and delivering scalable data solutions
  • Ability to function as a player-coach, mentor junior team members, and perform code reviews
  • Solid understanding of data governance, security (PII exclusion, multi-tenant isolation), and performance optimization
  • Strong communication skills and ability to translate technical concepts for business stakeholders
  • Experience with DBT
  • Experience with other BI tools
  • Experience with GenAI tools or integrating embeddings, vector stores, or feature stores
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Fairfax, VA
1,784 Employees
Year Founded: 2006

What We Do

3Pillar Global builds breakthrough software products that power digital businesses. 3Pillar is an innovative product development partner whose solutions drive rapid revenue, market share, and customer growth for industry leaders in Software and SaaS, Media and Publishing, Information Services, and Retail. Leveraging a lean and agile approach, 3Pillar delivers value-generating, digital solutions with specialized product strategy and management, user experience design, as well as software and data engineering expertise across mobile, cloud, and disruptive technologies.

Similar Jobs

TransUnion Logo TransUnion

Java Lead Developer

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Remote or Hybrid
Heredia, Ulloa, Lagunilla, CRI
13000 Employees

TransUnion Logo TransUnion

Devops Engineer

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Remote or Hybrid
Heredia, Ulloa, Lagunilla, CRI
13000 Employees

TransUnion Logo TransUnion

Full-stack Engineer

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Remote or Hybrid
Heredia, Ulloa, Lagunilla, CRI
13000 Employees

Mastercard Logo Mastercard

Consultant

Blockchain • Fintech • Payments • Consulting • Cryptocurrency • Cybersecurity • Quantum Computing
Remote or Hybrid
San Jose, CRI
38800 Employees

Similar Companies Hiring

Hanover Park Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
42 Employees
Kepler  Thumbnail
Fintech • Software
New York, New York
6 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account