Senior Data Engineer

Posted 12 Days Ago
Be an Early Applicant
2 Locations
In-Office
55K-65K Annually
Senior level
Fintech • Cybersecurity
The Role
The Senior Data Engineer will design and maintain data architectures, develop scalable data pipelines, and ensure data quality and governance across systems.
Summary Generated by Built In

Liminal is the actionable intelligence company. We've built a proprietary Living Graph — a verified knowledge architecture that maps the real-time structure of Identity, Fraud, and Cybersecurity — and the agentic AI systems and human verification layer to make it trustworthy. Visa, Mastercard, Google, and JPMC use it to make strategic and revenue decisions. Series A, 80 people, offices in NYC, Salt Lake City, Porto, Lisbon, and Manila. The architecture works, the customers are real, and we're scaling the team.

The Role

Liminal's intelligence platform runs on the Living Graph — a proprietary knowledge architecture that maps 2.5M+ entities across Identity, Fraud, and Cybersecurity and the relationships between them. That graph is only as good as the data infrastructure underneath it. We're looking for a Data Engineer to build and maintain the pipelines, architectures, and systems that keep the graph fed, fast, and reliable.

You'll design and optimize data pipelines that ingest, process, and transform large-scale datasets for AI models, product teams, and internal operations. You'll work across cloud infrastructure, ETL development, and data quality — ensuring the systems that power verified intelligence for Visa, Mastercard, Google, and JPMC are scalable, performant, and built to compound.

This is infrastructure work that matters. Every pipeline you build, every system you optimize, directly improves the intelligence that enterprise customers use to make high-stakes decisions.


What Success Looks Like

In your first 30 days, you've mapped the current data architecture, understand the pipeline landscape, and have shipped your first improvement to an existing workflow.

In your first 90 days, you've owned the design and deployment of a new pipeline end-to-end, improved reliability or performance on a critical data path, and become the go-to partner for at least one cross-functional team.

In your first year, you've meaningfully improved the scalability and resilience of Liminal's data infrastructure, built systems that other teams depend on daily, and shaped the engineering standards for how data moves through the platform.


What You'll Do
  • Design, build, and maintain data pipelines that ingest, process, and transform large-scale datasets for AI/ML models, product features, and internal analytics
  • Build and optimize ETL processes that move data reliably between systems, databases, and analytics platforms — handling both batch and real-time processing
  • Develop and maintain cloud-based data infrastructure (GCP, AWS, or similar), ensuring systems are robust, cost-effective, and built to scale with growing data volumes
  • Implement data storage solutions and distributed databases that integrate cleanly with internal systems and product workflows
  • Establish data validation, quality checks, and governance processes to ensure consistency, integrity, and compliance across all pipelines
  • Set up monitoring, alerting, and automation for pipeline health, performance, and data anomalies — reducing manual intervention and catching issues before they reach production
  • Collaborate with data scientists, product engineers, AI teams, and business stakeholders to ensure data is structured, accessible, and optimized for downstream use
  • Maintain clear documentation of architectures, pipeline designs, and system configurations so other teams can build on your work


What You Bring

You've built and optimized large-scale data systems in production and know how to design pipelines that are reliable, performant, and maintainable. You're proficient in Python (including OOP and software development best practices) and SQL, with hands-on experience across cloud platforms and services for data storage, processing, and orchestration. You've worked with ETL frameworks, workflow orchestration tools like Airflow, and containerization technologies (Docker, Kubernetes, or equivalent). You understand data security, privacy, and governance — and you build with those constraints from the start, not as an afterthought. You communicate clearly across technical and non-technical teams and care about documentation as much as deployment.


Bonus Points

Experience with GCP services (BigQuery, Cloud Run, Cloud Run Functions). Familiarity with Apache Spark or distributed processing frameworks. Understanding of REST APIs and their role in data integration. Exposure to data modeling for AI and machine learning pipelines. Experience working in a high-growth or startup environment where you've had to build systems that scale ahead of demand.


Why Liminal

You'll build infrastructure that powers real decisions. This isn't a data warehouse no one looks at. The pipelines you build feed a Living Graph that Visa, Mastercard, and Google rely on for competitive intelligence, regulatory monitoring, and market strategy.

The engineering problems are real. 2.5M+ entities. Continuous ingestion across patents, filings, code repositories, news, hiring data, and regulatory documents. Batch and real-time processing. Neuro-symbolic AI models that depend on verified, structured data to reason — not just retrieve. You'll build the plumbing for a compound AI system, not a CRUD app.

Your work compounds. Every pipeline you build, every system you harden, makes the Living Graph smarter and the platform more valuable. You're not shipping features that stagnate — you're building infrastructure that gets better with every signal.


Our Process

We respect your time. Here's what to expect:

Recruiter ScreenHiring Manager InterviewBehavioral InterviewPractical InterviewCEO InterviewOffer

Top Skills

Airflow
AWS
Azure
BigQuery
Cloud Storage
Docker
GCP
Git
Kubernetes
MongoDB
MySQL
Postgres
Python
Redshift
S3
SQL
Synapse
Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: New York, NY
28 Employees
Year Founded: 2016

What We Do

Liminal is a boutique strategy advisory firm serving digital identity, fintech, and cybersecurity clients, and the private equity / venture capital community. Since 2016, we have offered objective, high-impact strategic advice, and analytical services, helping to support clients in crucial business decisions at all stages of the product and business lifecycle. We’ve advised many of the world’s most innovative business leaders, investors, and government officials on building, buying, and investing in the next generation of integrated digital identity platforms and technologies. As a result, our clients trust us to set strategic direction in light of radically evolving ecosystem dynamics, pursue new growth strategies, capitalize on M&A opportunities, and optimize deal flow. We see the solutions to these complex digital challenges not as a ‘what’ but as a ‘how.’ We don’t just tell you about the destination, we show you how to get there.

Similar Jobs

FARFETCH Logo FARFETCH

Senior Data Engineer

eCommerce • Fashion • Retail
Hybrid
Porto, PRT
5294 Employees

Planet (weareplanet.com) Logo Planet (weareplanet.com)

Senior Data Engineer

Information Technology • Consulting
In-Office
Porto, PRT
2648 Employees

Jscrambler Logo Jscrambler

Senior Data Engineer

Information Technology • Software
In-Office
Porto, PRT
88 Employees

Feedzai Logo Feedzai

Senior Data Engineer

Information Technology
Remote or Hybrid
Portugal
650 Employees

Similar Companies Hiring

Rain Thumbnail
Web3 • Payments • Infrastructure as a Service (IaaS) • Fintech • Financial Services • Cryptocurrency • Blockchain
New York, NY
80 Employees
Scotch Thumbnail
Software • Retail • Payments • Fintech • eCommerce • Artificial Intelligence • Analytics
US
25 Employees
Citizens Bank Thumbnail
Fintech
US
17000 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account