Be an Early Applicant
Cognite makes data accessible and meaningful for our industrial customers.
The Role
As a Principal Software Engineer, you'll lead technical strategy and architecture for industrial data solutions, mentor teams, and address complex engineering challenges, particularly focused on streaming systems and data processing capabilities.
Summary Generated by Built In
About Cognite
Embark on a transformative journey with Cognite, a global SaaS forerunner in leveraging AI and data to unravel complex business challenges through our cutting-edge offerings including Cognite Atlas AI, an industrial agent workbench, and the Cognite Data Fusion (CDF) platform. We were awarded the 2022 Technology Innovation Leader for Global Digital Industrial Platforms & Cognite was recognized as 2024 Microsoft Energy and Resources Partner of the Year. In the realm of industrial digital transformation, we stand at the forefront, reshaping the future of Oil & Gas, Chemicals, Pharma and other Manufacturing and Energy sectors. Join us in this venture where AI and data meet ingenuity, and together, we forge the path to a smarter, more connected industrial future.
Learn more about Cognite here
Cognite Product Tour 2024
Cognite Product Tour 2023
Data Contextualization Masterclass 2023
Our values
Impact: Cogniters strive to make an impact in all that they do. We are result-oriented, always asking ourselves.
Ownership: Cogniters embrace a culture of ownership. We go beyond our comfort zones to contribute to the greater good, fostering inclusivity and sharing responsibilities for challenges and success.
Relentless: Cogniters are relentless in their pursuit of innovation. We are determined and deliverable (never ruthless or reckless), facing challenges head-on and viewing setbacks as opportunities for growth.
About Cognite & This Role
Cognite is revolutionizing industrial data management through our flagship product, Cognite Data Fusion - a state-of-the-art SaaS platform that transforms how industrial companies leverage their data. We're seeking an exceptional
Staff Data Platform Engineer who will drive technical strategy, architectural excellence, and cross-organizational impact.
As a Principal Software Engineer, you'll be a technical leader who shapes platform direction, mentors engineering teams, and tackles the most complex data engineering challenges in industrial technology. This role combines deep technical expertise with strategic thinking and leadership influence.
Strategic Impact & Leadership
- Technical Strategy & Architecture
- Define platform architecture for next-generation industrial data processing capabilities serving 100M+ daily requests
- Drive technology roadmap decisions that impact multiple engineering teams and product areas
- Lead architectural reviews and establish technical standards across the data platform organization
- Champion innovation initiatives that differentiate Cognite's technical capabilities in the industrial data market
- Architect foundational systems that serve as building blocks for multiple product teams and use cases
- Design advanced abstractions and frameworks that accelerate development velocity across the organization
- Own end-to-end platform reliability including SLAs, disaster recovery, and operational excellence
- Drive platform observability strategy with comprehensive metrics, alerting, and distributed tracing
- Lead Apache Spark ecosystem contributions - modify core components, contribute to open-source projects, influence roadmaps
- Architect multi-petabyte streaming systems using Flink, Kafka, and custom processing engines for industrial IoT data
- Design custom query engines and DSLs optimized for time-series analytics and operational data patterns
- Pioneer advanced optimization techniques for columnar storage, query planning, and distributed execution
- Mentor Staff and Senior engineers across multiple teams, developing technical leadership capabilities
- Drive cross-functional initiatives involving ML platform, product engineering, and infrastructure teams
- Establish engineering culture around technical excellence, innovation, and customer obsession
- Represent Cognite externally through conference speaking, open- source leadership, and technical thought leadership
Platform Engineering Excellence
Advanced Technical Leadership
Organizational Impact
Exceptional Candidate Profile
- Technical Leadership Mastery (8+ years)
- Distributed Systems Architecture - Advanced Apache Spark expertise - contributed to Spark core, Catalyst optimizer, or execution engine with merged commits - Large-scale streaming architecture - designed and operated Flink/Kafka systems processing 1M+ events/second with complex statemanagement - Custom execution engine development - built domain-specific query engines or processing frameworks from scratch - Performance engineering excellence - consistently delivered 10x+ performance improvements through algorithmic and architectural innovations
- Platform Engineering Leadership - Multi-tenant platform design - architected platforms serving 50+ engineering teams with sophisticated resource isolation and management - Advanced data lake architecture -designed exabyte-scale storage systems with intelligent tiering, compaction, and query optimization - Service mesh and infrastructure - implemented sophisticated microservices architectures with advanced networking, security, and observability
- Data Engineering Innovation - OLAP system expertise - deep experience with ClickHouse, Pinot, Druid internals including cluster management and query optimization - Advanced stream processing - implemented complex event processing, windowing, and stateful computations for industrial use cases - Data modeling excellence - designed sophisticated schemas for dimensional modeling, event sourcing, and real-time analytics
- Strategic Technical Decision Making - Technology evaluation leadership - led organization-wide technology adoption decisions with comprehensive POCs and risk analysis - Architecture governance - established and maintained architectural principles, standards, and review processes - Technical debt management - strategically balanced feature velocity, system reliability, and long-term maintainability
- Open-source contributions to major Apache projects in the data space (e.g. Apache Spark or Kafka) is a big plus - Significant open source contributions - maintainer or core contributor to major Apache projects (Spark, Flink, Kafka, Airflow) - Technical thought leadership - regular speaker at major conferences (Strata, Spark Summit, QCon) with recognized expertise - Community building -organized engineering communities, contributed to standards bodies, or led technical advisory roles Cross-Functional Collaboration - Product partnership excellence - translated ambiguous business requirements into robust technical architectures - Engineering leadership - mentored and developed multiple Senior and Staff engineers with measurable career progression - Executive communication - effectively communicated complex technical concepts to C-level stakeholders and board members
- High-Impact Delivery - Rapid scaling experience - led technical initiatives during hypergrowth phases (10x team/infrastructure scaling) - Customer- facing technical leadership - directly engaged with enterprise customers on complex technical integrations - Business impact ownership - drove technical initiatives that resulted in measurable business outcomes (cost reduction, revenue enablement)
- Innovation & Ambiguity Navigation - Greenfield platform development architected major platform components from concept to production at scale - Technical risk management - successfully navigated high-risk technical decisions with incomplete information - Competitive differentiation -developed technical capabilities that created sustainable competitive advantages
Technical Leadership Excellence
Open Source & Community Leadership -
Startup Excellence & Scale
Technical Environment & Challenges
- Advanced Technical Stack
- Core Languages: Kotlin, Scala, Java (advanced JVM optimization)
- Big Data Ecosystem: Apache Spark (internals), Apache Flink, Apache Kafka, Apache Airflow
- Advanced Storage: ClickHouse, PostgreSQL, Elasticsearch, S3-compatible systems with custom optimization
- Infrastructure: Kubernetes (operator development), Terraform,Advanced observability stack
- Table Format Innovation: Apache Iceberg, Delta Lake internals andoptimization
- Query Engine Development: Trino/Presto, Apache Pinot, custom engine development
- Advanced Streaming: Complex event processing, exactly-once semantics, advanced windowing
- ML/AI Integration: Feature stores, model serving infrastructure, MLOps platform integration
- Industrial IoT at Scale: Processing sensor data from millions of industrial assets in real-time
- Complex Time-Series Analytics: Advanced temporal queries, anomaly detection, predictive analytics
- Multi-Cloud Optimization: Cost and performance optimization across AWS, Azure, GCP
- Enterprise Integration: Secure, scalable integration with legacy industrial systems
Cutting-Edge Technologies You'll Drive
Unique Technical Challenges
Join the global Cognite community! 🌐
- Join an organization of 70 different nationalities 🌐 with Diversity, Equality and Inclusion (DEI) in focus 🤝
- Office location Rathi Legacy (Rohan Tech Park ) Hoodi (Bengaluru)
- A highly modern and fun working environment with sublime culture across the organization, follow us on Instagram @cognitedata 📷 to know more
- Flat structure with direct access to decision-makers, with minimal amount of bureaucracy
- Opportunity to work with and learn from some of the best people on some of the most ambitious projects found anywhere, across industries
- Join our HUB 🗣️ to be part of the conversation directly with Cogniters and our partners.
- Hybrid work environment globally
Why choose Cognite? 🏆 🚀
Join us in making a real and lasting impact in one of the most exciting and fastest-growing new software companies in the world. We have repeatedly demonstrated that digital transformation, when anchored on strong DataOps, drives business value and sustainabilityfor clients and allows front-line workers, as well as domain experts, to make better decisions every single day. We were recognized as one of CNBC's top global enterprise technology startups powering digital transformation! And just recently, Frost & Sullivan named Cognite a Technology Innovation Leader! 🥇 Most recently Cognite Data Fusion® Achieved Industry First DNV Compliance for Digital Twins 🥇
Apply today!
If you're excited about the opportunity to work at Cognite and make a difference in the tech industry, we encourage you to apply today! We welcome candidates of all backgrounds and identities to join our team.
We encourage you to follow us on Cognite LinkedIn; we post all our openings there.
Top Skills
Apache Airflow
Apache Flink
Apache Iceberg
Apache Kafka
Spark
Clickhouse
Delta Lake
Elasticsearch
Java
Kotlin
Kubernetes
Mlops
Postgres
Presto
Scala
Terraform
Trino
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
Cognite is an AI company that delivers industrial software to improve the production efficiency of Energy, Process Manufacturing, and other industrial companies.
We deliver an Industrial DataOps platform that liberates siloed data and empowers our customers to solve some of their most complex business problems with AI-powered solutions. The typical solutions we enable drive innovative new ways to approach Data Exploration, Digital Operator Rounds, Production Optimization, Turnaround Planning, and Root Cause Analysis.
We do this by automating and scaling industrial data contextualization of various sources (such as time series, engineering diagrams, equipment logs, maintenance records, 3D facility models, images, large point clouds, and more). We use AI and other tools to find and map the meaningful relationships between the data across these various sources. In addition, we provide intuitive tools that enable efficient use of analytics and automated workflows, as well as prebuilt AI capabilities and a low-code industrial agent builder, Cognite Atlas AI, that enables AI to carry out more complex operations with greater accuracy.
Why Work With Us
Employees at Cognite are pushing the envelope with the latest cloud technology, scaling industrial applications across hundreds of assets, revolutionizing industrial data models, and working with robotics. Cogniters are fast, creative, and resilient. We keep the energy high and fun, learning from our mistakes and celebrating our victories together.
Gallery
