Principal Architect - Databricks

Posted 2 Days Ago
Be an Early Applicant
Hiring Remotely in US
Remote
195K-250K Annually
Expert/Leader
Information Technology • Software
The Role
Lead the technical side of Databricks pursuits, including discovery, target architecture, estimation, proposals, migration strategy, and risk assessment. Provide architectural oversight through delivery, shape repeatable Databricks offerings, and partner with Databricks field teams. The role requires deep expertise in lakehouse architecture, enterprise migrations, data engineering, production AI and machine learning, governance, consumption economics, and principal-level technical leadership.
Summary Generated by Built In
About Us

Since 1989, SHI International Corp. has helped organizations change the world through technology. We’ve grown every year since, and today we’re proud to be a $16 billion global provider of IT solutions and services.

 

Over 17,000 organizations worldwide rely on SHI’s concierge approach to help them solve what’s next. But the heartbeat of SHI is our employees – all 7,000 of them. If you join our team, you’ll enjoy:

  • Our commitment to diversity, as the largest minority- and woman-owned enterprise in the U.S.

  • Continuous professional growth and leadership opportunities.

  • Health, wellness, and financial benefits to offer peace of mind to you and your family.

  • World-class facilities and the technology you need to thrive – in our offices or yours. 

Job Summary

We are expanding our Databricks capability within the Data and AI practice and are looking for a Principal Solutions Architect to lead the technical side of it.
You will work directly with customers from the outset, qualifying opportunities, conducting technical discovery with customer architects and data leaders, designing target-state platforms, and owning the technical win. Much of the work ahead involves platform build-out, migration, and consolidation, helping enterprises move to the Databricks Lakehouse from Hadoop and Spark estates, legacy analytics platforms, established data warehouse and appliance environments, and other cloud platforms. The role also supports AI and machine learning initiatives that Databricks is designed to enable.
This is a senior individual contributor position with significant influence over the solutions we build and bring to market. It is ideal for someone who wants to shape a capability rather than operate within one that is already defined.

Role Description

Own the Technical Side of Databricks Pursuits

  • Lead qualification, technical discovery, target architecture design, effort estimation, and risk assessment.

  • Develop technical content for proposals and Statements of Work (SOWs).

  • Build and maintain technical relationships with customer architects and stakeholders.

  • Provide customers with realistic and accurate assessments of migration complexity, implementation effort, AI workload readiness, and associated risks.

Provide Architectural Oversight Through Delivery

  • Review architecture decisions throughout project delivery.

  • Confirm that the delivered solution aligns with what was originally scoped and proposed.

  • Surface technical risks and issues early in the engagement.

  • Remain closely involved throughout delivery to ensure accountability for architecture and outcomes.

Shape Databricks Service Offerings

  • Define the technical content of market-facing Databricks offerings.

  • Establish what is included in scope, how success is measured, and what evidence validates successful completion.

  • Create repeatable delivery standards and frameworks that teams can execute consistently.

Partner with Databricks

  • Engage directly with Databricks field teams and specialist architects on joint customer accounts.

  • Build productive relationships with Databricks technical teams.

  • Effectively collaborate and navigate technical challenges, feedback, and differing viewpoints from vendor architects and specialists.

Behaviors and Competencies

  • Willingness to Learn: Can apply new learning to daily work, encourage and facilitate learning in others, and actively make changes to work based on feedback.

  • Self-Development: Can demonstrate a commitment to continuous learning and adaptability to new ideas and methods.

  • Leadership: Can take ownership of complex team initiatives, collaborate with others in decision-making processes, and drive team performance.

  • Strategic Thinking: Can analyze complex situations, anticipate future trends, and align and integrate strategies across departments or functions.

  • Problem-Solving: Can proactively identify and take ownership of complex problem-solving initiatives, initiate preventative measures, collaborate with others to find solutions, and drive successful outcomes.

  • Analytical Thinking: Can use advanced analytical techniques to solve complex problems, draw insights, and communicate the solutions effectively.

  • Prioritization: Can take ownership of complex task management, collaborate with others to align priorities, and drive team efficiency.

  • Customer-Centric Mindset: Can take ownership of customer-centric initiatives, ensuring products and services align with customer needs. Collaborates with cross-functional teams to integrate customer feedback into product development.

  • Organization: Can oversee complex projects with multiple moving parts, ensure team alignment with organizational systems, and adapt to changing priorities.

  • Communication: Can effectively communicate complex ideas and information to diverse audiences, facilitate effective communication between others, and mentor others in effective communication.

  • Interpersonal Skills: Can communicate effectively, build relationships, resolve conflicts, influence others, and support others in developing their interpersonal skills in major situations.

Skill Level Requirements

Lakehouse Platform and Architecture

  • Account and workspace design

  • Unity Catalog and metastore architecture

  • Compute strategy across interactive, jobs, and serverless workloads

  • Open table format decisions

  • Cost architecture and design choices that affect long-term scalability and affordability

The ability to design platforms that remain technically and financially sustainable at enterprise scale is critical.

Migration and Consolidation

  • Migrating organizations from Hadoop and Spark environments

  • Modernizing legacy analytics and statistical platforms

  • Consolidating established warehouses and appliance-based solutions

  • Migrating workloads from other cloud platforms

Key areas of expertise include:

  • Platform assessments

  • Wave planning and migration strategy

  • Workload and code conversion

  • Reconciliation and parity validation

  • Cutover planning and execution

This is expected to represent a significant portion of the work within the practice and requires genuine expertise rather than general familiarity.

Data Engineering and Pipelines

  • Batch data ingestion

  • Streaming data ingestion

  • Declarative pipeline development

  • Workflow orchestration

  • Testing and validation frameworks

  • Data quality instrumentation

  • Building and operating reliable, production-ready data engineering solutions

Machine Learning and AI Engineering

  • Feature engineering

  • Experiment tracking

  • Model registry and serving

  • Evaluation and validation frameworks

  • Retrieval and grounding architectures

  • Agent development

This represents one of Databricks’ key differentiators and is an area where many customers require assistance moving from proof-of-concept solutions to production-grade, supportable implementations.

Governance at the Perimeter

  • Catalog-based access controls

  • Data classification

  • Data masking

  • Data lineage and governance frameworks

Candidates should understand how Databricks governance integrates with enterprise governance strategies and be able to address challenges that exist across organizational and platform boundaries.

Consumption Economics

  • Compute sizing strategies

  • Workload placement optimization

  • Serverless versus classic compute trade-offs

  • Consumption attribution and chargeback models

  • Cost management and optimization practices

Candidates must be comfortable discussing platform costs with customers and providing practical guidance on controlling and forecasting consumption.

Technical Depth Expectations

Candidates do not need to be equally strong across all six areas; however, they must possess:

  • Deep expertise in lakehouse architecture and platform design

  • Deep expertise in migration and consolidation

  • Deep expertise in data engineering

  • Sufficient proficiency in the remaining areas to recognize when specialist support should be engaged

Other Requirements

Databricks Expertise

  • Substantial hands-on Databricks experience gained through:

    • A Databricks partner organization

    • Databricks directly

    • Managing a significant Databricks Lakehouse environment internally

  • This role requires Databricks-specific expertise rather than general data platform leadership experience.

Enterprise Migration and Consolidation Experience

  • Proven success delivering enterprise-scale migration and consolidation initiatives.

  • Ability to discuss at least one major migration project in detail, including lessons learned and outcomes.

Production Machine Learning and AI Experience

  • Experience operating and supporting production machine learning or AI solutions.

  • Expertise beyond proof-of-concept implementations.

  • Demonstrated understanding of the operational, governance, and maintenance considerations required for production AI workloads.

Principal-Level Technical Leadership

  • Proven success operating at Principal Architect level or equivalent.

  • Recognized as the senior technical authority in customer engagements.

  • Trusted to commit organizations to scopes, architectures, and technical approaches.

Presales Experience

  • Experience in presales environments, or

  • Delivery leadership experience demonstrating the ability to:

    • Lead customer discussions

    • Define scope and solution approaches

    • Develop proposals and Statements of Work

    • Stand behind technical commitments made during the sales process

Vendor Collaboration

  • Comfortable working with and being challenged by vendor architects and technical specialists.

Certifications

  • Databricks certification preferred.

  • Ability to obtain certification quickly if not currently certified.

  • Certification support and funding are provided.

The estimated annual pay range for this position is $195,000 - $250,000 which includes a base salary and bonus. The compensation for this position is dependent on job-related knowledge, skills, experience, and market location and, therefore, will vary from individual to individual. Benefits may include, but are not limited to, medical, vision, dental, 401K, and flexible spending.

Equal Employment Opportunity – M/F/Disability/Protected Veteran Status 

Skills Required

  • Substantial hands-on Databricks experience from a Databricks partner, Databricks, or management of a significant internal Databricks Lakehouse environment
  • Deep expertise in lakehouse architecture and platform design
  • Deep expertise in enterprise migration and consolidation
  • Deep expertise in data engineering and production-ready data pipelines
  • Proven success delivering enterprise-scale migration and consolidation initiatives
  • Experience operating and supporting production machine learning or AI solutions beyond proof-of-concept implementations
  • Proven success operating at Principal Architect level or equivalent
  • Experience in presales environments or delivery leadership covering customer discussions, solution definition, proposals, Statements of Work, and technical commitments
  • Comfort working with vendor architects and technical specialists
  • Sufficient proficiency in machine learning and AI engineering, governance, and consumption economics to recognize when specialist support is needed
  • Databricks certification
  • Ability to obtain Databricks certification quickly if not currently certified

SHI International Corp. Compensation & Benefits Highlights

The following summarizes recurring compensation and benefits themes identified from responses generated by popular LLMs to common candidate questions about SHI International Corp. and has not been reviewed or approved by SHI International Corp..

  • Healthcare Strength Feedback suggests the medical, dental, vision, and pharmacy offerings are comprehensive, with multiple plan options, telemedicine access, and solid ancillary coverage.
  • Wellbeing & Lifestyle Benefits Feedback suggests onsite fitness centers, wellness programs that can reduce premiums, and discounts/purchase programs provide meaningful lifestyle support.
  • Strong & Reliable Incentives Feedback suggests commission structures in sales and premium totals in certain technical presales roles create notable upside potential for strong performers.

SHI International Corp. Insights

Am I A Good Fit?
beta
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.

The Company
HQ: Somerset, NJ
7,309 Employees

What We Do

Think of SHI as your personal technology concierge. We connect your team with the IT solutions and services you need to support your organizational growth and employee experience. Whether you’re building a modern hybrid workplace, defending against an evolving threat landscape, making the cloud work harder for you, or searching for ways to optimize your software portfolio, our friendly 6,000-person team is ready to solve what’s next for your organization. Our in-house data center integration, device configuration, and deployment and license advisory services, plus our top-tier status with vendors and flexible financing make life simpler for IT decision makers. Execute your IT vision with stress-free, scalable solutions you – and your people – will love. SHI is proud to be the largest Minority/Woman Owned Business Enterprise (MWBE) in the United States.

Similar Jobs

CDW Logo CDW

Architect

Information Technology
Remote or Hybrid
US
15100 Employees
156K-241K Annually

Caylent Logo Caylent

Data Architect

Cloud • Software
Remote
United States
206 Employees
165K-185K Annually

Blue Orange Digital Logo Blue Orange Digital

Solutions Architect

Artificial Intelligence • Machine Learning • Database
In-Office or Remote
10 Locations
75 Employees

Bounteous Logo Bounteous

Architect

Artificial Intelligence • Information Technology • Professional Services • Software • Analytics • Generative AI • Big Data Analytics
Remote
United States
5000 Employees
190K-200K Annually

Similar Companies Hiring

Kepler  Thumbnail
Artificial Intelligence • Fintech • Software
New York, New York
9 Employees
Onshore Thumbnail
Artificial Intelligence • Fintech • Software • Financial Services
New York, New York
60 Employees
Revel.io Thumbnail
Aerospace • Hardware • Robotics • Software
US
50 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account