Senior Data Engineer

Agency

Department

The Senior Data Engineer is a critical role responsible for automating geospatial data ingestion and transformation, optimizing ETL pipelines, and ensuring FAIR compliance and real-time data updates across our data platform. The ideal candidate is a seasoned professional in data engineering and automation, working closely with Cloud/DevOps Engineer, Geospatial Developer, and GIS Data Specialists to streamline workflows, enhance data quality, and drive efficiency in a fast-paced, cloud-centric environment. The position reports to the data products leader.

The Texas Community Watershed Partners (TCWP) team provides education, outreach, and planning support for communities around the state of Texas, focusing on approaches that foster collaboration and holistic solutions to reduce risks from floods and other hazards. Leveraging data-driven methodologies and innovative tools, such as CHARM and CommunityViz, our award-winning GIS-based scenario planning software, we facilitate collaborative sessions with community staff and elected officials working together to build a resilient and sustainable future.

Operating within the largest university-based extension program nationwide, TCWP is uniquely positioned to deliver highly impactful community planning tools and services. We cultivate strong partnerships with sister state agencies and federal entities, and aim to share our knowledge, expertise, and tools to communities in need of sustainable planning practices.

Optional two days work from home. Employees are eligible for a generous health insurance plan, including dental and vision. PTO starts at 12 personal days a year, 11 paid holidays, and ten sick leave days. Employees join the Texas Teachers Retirement System (TRS) with employer contributions. Additional retirement savings are available. Office culture is casual but earnest about doing good work and helping communities improve their safety and well-being through planning. As university employees, staff will have no-cost access to LinkedIn learning training and professional credits.

• Develops scalable ETL pipelines to ingest, clean, and update large-scale geospatial datasets.

• Optimizes Azure Databricks and Spark workflows for high-performance and cost-efficient automation.

• Implements robust data validation, quality control, and governance policies to ensure data integrity and FAIR compliance.

• Enhances data integration processes by designing and integrating APIs and third-party GIS tools.

• Automates metadata tagging and enforces FAIR data standards to maintain consistent, high-quality datasets.

• Evaluates emerging technologies and proposes process improvements to further optimize data pipeline performance and cost control.

• Works closely with Cloud/DevOps Engineer and GIS teams to troubleshoot issues, streamline workflows, and continuously improve system performance.

• Contributes to comprehensive documentation, training, and knowledge transfer initiatives to empower internal teams.

Required Education and Experience:

• Bachelor’s degree or equivalent combination of education and experience.

• Ten years of related experience.

Required Knowledge, Skills and Abilities:

• Knowledge of advanced statistical analysis, interactive data visualization, programming or similar software applications. Knowledge of use of data mining.

• Ability to research and develop statistical learning models for data analysis.

• Ability to multitask and work cooperatively with others.

• Excellent written communication, analytical, interpersonal, and organizational skills.

Preferred Education and Experience:

• Bachelor’s degree in Computer Science, Data Science, or a related field with at least seven years of relevant experience in data engineering and ETL automation.

• A Master's or PhD degree is preferred.

• Holds relevant Azure certifications such as Microsoft Certified Azure Fundamentals or Azure Data Fundamentals.

• Holds advanced Azure certifications such as Microsoft Certified Azure Data Engineer Associate, Azure Solutions Architect Expert, or Azure DevOps Engineer Expert.

• Strong experience in ETL tools (Databricks, Azure Data Factory, Apache Airflow, dbt).

• Advanced proficiency in Python, SQL, and Spark for geospatial data processing.

• Proven expertise in cloud-based data pipeline automation (Azure, AWS, GCP).

• Good understanding of FAIR data standards, data governance, and compliance frameworks.

• Experience with geospatial technologies such as PostGIS, GeoJSON, and spatial indexing.

• Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes) is a plus.

Other Requirements:

Administrative physical demands:

• May require extended bending, reaching, stooping, kneeling, squatting, and sitting.

• May require lifting workshop and outreach equipment, up to 30 pounds.

• May require extended communication with visitors and employees in person, telephonically or electronically.

• May require infrequent travel in State vehicles with overnight stays.

• May require operating computers with high-definition screens for extended periods of time.

This is a grant funded position. The salary for this position is commensurate based on applicant's qualifications. Applicants should include a resume, cover letter and three references for consideration.

Texas A&M AgriLife is an Equal Opportunity/Veterans/Disability Employer.

All positions are security-sensitive. Applicants are subject to a criminal history investigation, and employment is contingent upon the institution’s verification of credentials and/or other information required by the institution’s procedures, including the completion of the criminal history check.

Equal Opportunity/Veterans/Disability Employer.