We are seeking a Senior Software Engineer specializing in data pipelining and modeling. You will play a crucial role in designing, developing, and maintaining our data infrastructure. You will work closely with cross-functional teams to build robust, scalable, and efficient data pipelines, ensuring high data quality and reliability. Your expertise in ETL processes, Airflow, Streaming, Warehousing, and database technologies will drive our data initiatives forward.

What you’ll do:

• Design and implement data pipelines for ingesting, processing, and transforming large volumes of data.
• Develop and maintain data models to support analytical and reporting needs.
• Optimize data pipelines for performance, scalability, and reliability.
• Implement real-time data streaming solutions using streaming technologies.
• Create and manage ETL processes to extract, transform, and load data from various sources into data stores.
• Monitor and troubleshoot data pipeline and ETL issues.
• Evaluate, select, and implement columnar and store databases that best-fit project requirements.
• Understand the tradeoffs between different database technologies and make informed decisions.
• Perform database optimization and tuning for efficient data retrieval and storage.
• Collaborate closely with data scientists, analysts, and other engineering teams to understand data requirements and deliver solutions.
• Document data pipelines, models, and ETL processes for knowledge sharing and troubleshooting.
• Promote and enforce best practices in data engineering and data governance.
• Participate in an on-call rotation schedule to provide timely response and support for engineering-related issues outside of regular business hours, ensuring the continuous operation of critical systems and infrastructure.

Who you are:

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Minimum of 5+ years of experience in software engineering with a focus on data engineering.
• Strong proficiency in data pipelining and modeling, including experience with tools like Apache Airflow.
• In-depth knowledge of data streaming technologies, especially Apache Kafka.
• Expertise in designing and implementing ETL processes.
• Proficiency in SQL and NoSQL databases, understanding the tradeoffs between different types.
• Experience with columnar and store databases.
• Strong problem-solving and debugging skills.
• Excellent communication and teamwork abilities.
• Familiarity with cloud platforms like AWS, GCP, or Azure is a plus.

Examples of systems our teams build:

• We deal with massive amounts of data here at Madhive. To use it, we need the ability to move it around, aggregate, maintain, and turn it into usable information that powers our platform.
• External & internal APIs that enable frictionless integration within our platform, ranging from custom reporting for ad-hoc analysis to platform APIs for programmatic activation of programmatic ad campaigns, and much more!
• Our stack lives in the warm embrace of GCP. Our engineers are language agnostic, but core technologies for our backend include Golang, gRPC, BigQuery, Postgres, Airflow, Bigtable, Docker, Spanner, Firebase, Kubernetes, and Terraform.

The approximate compensation range is $200,000 to $250,000 The actual offer, reflecting the total compensation package and benefits, will be determined by a number of factors including the applicant's experience, knowledge, skills, and abilities, as well as internal equity among our team.