Databricks Architect - US Based Remote

Company Description

CoEnterprise is an award-winning B2B software and professional services company headquartered in New York City. Founded in 2010, CoEnterprise delivers Data & Analytics solutions and services that transform how companies connect and do business. CoEnterprise approaches each relationship and engagement from the perspective of three core values: collaboration, ownership, and excellence. We value collaboration with both our partners and clients in order to present the best possible outcome for our customers. Our vow to accept ownership ensures that our entire staff takes pride in our work and it is our commitment to excellence that ensures that this work is at the highest standard possible.

Job Description

We are seeking a talented and experienced Databricks Architect to join our growing analytics practice as a full-time team member. This position is US Based remote.

In this role, you will design and implement innovative data solutions, leveraging Databricks and the Medallion Architecture to address complex business challenges for our clients. This is a long-term position offering the opportunity to lead cutting-edge data initiatives while contributing to the strategic growth of our practice.

As a key leader in our team, you will work closely with both clients and internal stakeholders to deliver robust, scalable, and automated data architectures that support advanced analytics and reporting needs. This role requires a blend of technical expertise, hands-on development, and collaboration skills to achieve impactful outcomes.

Key Responsibilities include:

• Design and implement scalable, high-performance data architectures using Databricks and the Medallion Architecture.
• Lead requirement-gathering sessions with clients and internal teams to understand business needs and define best practices for solutions including data workflows.
• Collaborate with cloud platform teams to optimize data storage and retrieval in environments like AWS S3, Azure Data Lake, and Delta Lake.
• Translate complex data processes, such as those in Alteryx and Tableau, into optimized Databricks workflows using PySpark and SQL.
• Develop reusable automation scripts to streamline workflow migrations and improve operational efficiency.
• Provide hands-on development and troubleshooting support to ensure smooth implementation and optimal performance.
• Partner with cross-functional teams to establish data governance frameworks, best practices, and standardized reporting processes.
• Deliver training, documentation, and ongoing support to empower users and enhance organizational data literacy.
• Stay ahead of industry trends, identifying opportunities to integrate new tools and methods that enhance the practice's capabilities.

Qualifications

Qualifications

Technical Expertise

• 7+ years’ experience in data engineering or software development, with a strong focus on data architecture
• Extensive experience (3-5+ years) with Databricks and the Medallion Architecture.
• Expertise in Python, including developing production-grade data pipelines, reusable automation scripts, and leveraging libraries such as Pandas, and NumPy for advanced data manipulation and analytics.
• Proficiency in PySpark and SQL with in-depth experience integrating a wide variety of data sources with Databricks.
• Expertise in cloud platform services (AWS, Azure, GCP), including their data storage solutions (e.g., S3, Azure Data Lake Storage, Google Cloud Storage), with hands-on experience in designing, optimizing, and integrating scalable data pipelines and architectures across one or more of these platforms.
• In-depth experience with Delta Lake, including schema enforcement, time travel, and optimization for ACID transactions.
• Strong understanding of data pipelines, transformation workflows, and modeling for analytics and reporting, including scheduling, monitoring, and error handling in distributed data processing environments.
• Experience in building automation tools or reusable scripts to accelerate data engineering processes.
• Knowledge of Alteryx, Tableau, and Tableau Server integration is a plus.

Leadership and Collaboration

• Proven ability to lead technical workshops, gather complex requirements, and translate them into actionable solutions.
• Strong communication skills to collaborate with clients, internal teams, and stakeholders at all levels.
• Experience mentoring team members or supporting the professional growth of peers.

Preferred Qualifications

• Background in supply chain, retail, or related industries.
• Relevant certifications such as Databricks Certified Data Engineer Professional or equivalent.

Additional Information

Come experience our spirited culture and work with a smart, dedicated and high-energy team in a stable and fast-growing company! Here is a small sample of our benefits and perks we offer:

• Comprehensive Health Insurance with generous employer contribution
• Matching 401(k) - $$$$
• Generous PTO Policy
• Virtual Events and Team Meetings
• Wellness Program

At CoEnterprise, we believe diversity drives innovation. We are committed to creating and maintaining a workplace in which all employees have an opportunity to participate and contribute to the success of our business. In recruiting for our team, we welcome the unique contributions that you can bring. We value employees for their differences represented by a variety of dimensions including demographics, behaviors, work style and perspectives.

We are an AA/EOE employer.
Position is US Based remote.