The Role
Lead data management across batch and streaming environments; design and maintain ETL pipelines; develop Databricks and Apache Spark solutions; manage real-time processing, data transformation, routing, and enrichment; optimize storage and retrieval performance; support data security and compliance; and document processes. The role requires expert Databricks knowledge and technical leadership while collaborating with security, data science, and AI engineering teams.
Summary Generated by Built In
What we are looking for…
We are looking for a highly skilled Data Engineer with expert-level knowledge to assume a key role in managing data, both at rest and in streaming, and to lead efforts in data management and optimization. Your proficiency in ETL pipeline development, data tools and your deep understanding of data management principles will be instrumental in ensuring our data infrastructure operates efficiently.
What you will do...
What you will bring…
Stand Out With…
Who we are...
August Schell offers 30 years of experience in providing our customers innovative solutions and engineering services to their most challenging needs. We thrive on navigating complex IT difficulties and are driven to find the best solutions for our customers. Our team delivers expert knowledge in cyber security, big data, endpoint security and cloud solutions. Our ability to serve our customers with continued excellence is a direct reflection of our dedicated and exceptional people. At August Schell, we are not only a team, we are family.
All are welcome…
August Schell is a proud Equal Employment Opportunity and Affirmative Action Employer. We believe that diversity, inclusion, and recognizing that our employees bring unique differences, perspectives, and ideals that are the backbone of our success. We welcome all persons and the talent they have to offer regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, or Veteran status.
We are looking for a highly skilled Data Engineer with expert-level knowledge to assume a key role in managing data, both at rest and in streaming, and to lead efforts in data management and optimization. Your proficiency in ETL pipeline development, data tools and your deep understanding of data management principles will be instrumental in ensuring our data infrastructure operates efficiently.
What you will do...
- Data Management Leadership: Provide technical leadership in data management, overseeing data at rest and data streaming, and guiding the team.
- ETL Pipeline Development: Design, develop, and maintain efficient ETL pipelines to process and transform data, ensuring data quality, integrity, and consistency.
- Databricks Expertise: Optimize data processing by using your expertise in Databricks, develop Spark-based solutions, and enhance data integration and analytics capabilities.
- Streaming Data: Manage streaming data sources and implement real-Ime data processing solutions using tools like Apache Kana or similar technologies.
- Data Transformation: Implement and manage data routing, transformation, and enrichment using Cribl or similar data pipeline orchestration tools.
- Data Optimization: Work on data optimization initiatives, including performance tuning, indexing, and data compression, to ensure efficient data storage and retrieval.
- Data Security and Compliance: Collaborate with security teams to ensure data security and compliance with data privacy regulations.
- Documentation: Build comprehensive documentation for data management processes, ETL pipelines, and facilitating knowledge transfer within the team.
What you will bring…
- 5+ years’ experience as a Data Engineer with a deep understanding of data management, ETL pipelines and streaming data.
- 5+ years’ experience writing code using Python, Scala and/or Java.
- Consultant-level, expert experience in Databricks.
- Apache Spark experience (mandatory).
- 2+ years’ expertise in Databricks, Apache Spark, Snowflake and/or similar data processing frameworks.
- BA / BS degree with 4+ years of experience (or) MS degree with 2+ years of experience as a data engineer.
Stand Out With…
- Experience with data streaming technologies (e.g., Apache Kafka, Apache Flink) and real-time data processing.
- Familiarity with AI and machine learning concepts and the ability to work with data scientists and AI engineers.
- Experience working on SQL or NoSQL data stores.
- Proficiency in tools like Cribl, Airflow or similar for data pipeline orchestration and transformation.
- Knowledge of data security and compliance principles.
Who we are...
August Schell offers 30 years of experience in providing our customers innovative solutions and engineering services to their most challenging needs. We thrive on navigating complex IT difficulties and are driven to find the best solutions for our customers. Our team delivers expert knowledge in cyber security, big data, endpoint security and cloud solutions. Our ability to serve our customers with continued excellence is a direct reflection of our dedicated and exceptional people. At August Schell, we are not only a team, we are family.
All are welcome…
August Schell is a proud Equal Employment Opportunity and Affirmative Action Employer. We believe that diversity, inclusion, and recognizing that our employees bring unique differences, perspectives, and ideals that are the backbone of our success. We welcome all persons and the talent they have to offer regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, or Veteran status.
Skills Required
- 5+ years of experience as a Data Engineer
- Deep understanding of data management, ETL pipelines, and streaming data
- 5+ years of experience writing code using Python, Scala, and/or Java
- Consultant-level, expert experience in Databricks
- Experience with Apache Spark
- 2+ years of expertise in Databricks, Apache Spark, Snowflake, and/or similar data processing frameworks
- BA/BS degree with 4+ years of data engineering experience, or MS degree with 2+ years of data engineering experience
- Experience with data streaming technologies such as Apache Kafka or Apache Flink
- Familiarity with AI and machine learning concepts
- Experience with SQL or NoSQL data stores
- Proficiency with Cribl, Airflow, or similar pipeline orchestration and transformation tools
- Knowledge of data security and compliance principles
Am I A Good Fit?
Get Personalized Job Insights.
Our AI-powered fit analysis compares your resume with a job listing so you know if your skills & experience align.
Success! Refresh the page to see how your skills align with this role.
The Company
What We Do
August Schell is dedicated to delivering agile and innovative cybersecurity solutions to proactively defend and protect organization’s data against cyber attacks. Through our proven expertise with cybersecurity services and solutions, we are masters at designing, deploying and implementing highly secure solutions for on-premise and cloud based computing environments. We offer a wide range of next-generation products and services for: - Enterprise-Scale Identity Management, including Derived Credentials for Mobility - Big Data, including Security and Visualization related to Big Data Network Security and Visibility - Operational Intelligence - Data Center Security, including Firewalls, Endpoint Awareness/Management, and Virtualization - Cloud Security and Visibility - Application and Data Security - Software-Defined Data Center (SDDC), including Compute, Storage, Network & Backup We understand how critically important it is to defend critical digital assets and sensitive data from the constant threat of cyber attacks. We do not take this responsibility lightly and it’s with integrity that August Schell has defined our approach to our customers, without boundaries.
