Job Title: Senior Technical Specialist
Education: Any Graduate
Experience: 10+years
Location: Bengaluru, Mumbai, Hyderabad.
We are looking for an experienced Senior Data Engineer / AWS Data Engineer with 10–12 years of hands-on experience in designing, developing, and maintaining scalable data engineering solutions. The ideal candidate should have strong expertise in AWS data services, Python, PySpark, SQL, ETL, infrastructure as code, and data orchestration.The candidate will be responsible for building robust data pipelines, integrating cloud and on-premises data sources, optimizing data processing workflows, and supporting enterprise-scale data platforms.
Technology Required Expertise
AWS Lambda, Glue, SNS, orchestration and related data services
Python Strong hands-on experience
PySpark Strong hands-on experience
SQL Advanced SQL, query optimization, data transformation
Redshift Development, optimization, data loading
RDS Database development and management
SSIS ETL development and maintenance
SSMS SQL Server development and troubleshooting
Terraform Infrastructure as Code
YAML Configuration/automation
Git Version control and source code management
Podman Containerization
VS Code Development environment
Good to Have
Experience with Connect
Experience with TOLSE / TOLENT
Experience with CI/CD and DevOps practices.
Experience working in Agile/Scrum environments.
Knowledge of cloud security, IAM, monitoring, and logging.
Qualifications
Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
10 -12 years of professional experience in data engineering, ETL, cloud data platforms, or related technologies.
Strong problem-solving and analytical skills.
Excellent communication and collaboration skills.
Ability to work independently as well as within a cross-functional team.
Target Profile: Senior Data Engineer / AWS Data Engineer / Lead Data Engineer with 10 -12 years of experience
Key Responsibilities
Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
Develop cloud-native data solutions using AWS services, including Lambda, Glue, SNS, and orchestration services.
Build and optimize data processing jobs using Python and PySpark.
Develop complex SQL queries, stored procedures, and data transformations.
Work with Amazon Redshift and Amazon RDS for data storage, processing, and analytics.
Develop and maintain ETL packages and workflows using SSIS.
Use SSMS for database development, troubleshooting, and performance optimization.
Implement infrastructure and configuration using Terraform and YAML.
Work with Git for source code management, branching, merging, and version control.
Containerize and manage development workloads using Podman.
Use VS Code as the primary development environment.
Implement event-driven and automated workflows using AWS services such as Lambda, SNS, and orchestration frameworks.
Monitor, troubleshoot, and optimize data pipelines and AWS workloads.
Ensure data quality, reliability, security, and performance across data platforms.
Collaborate with architects, developers, business analysts, and other stakeholders to deliver data solutions.
Participate in code reviews, technical discussions, and production support activities.
About UsDatavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner. About the Team
Skills Required
- 10–12 years of professional experience in data engineering, ETL, cloud data platforms, or related technologies
- Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field
- Strong hands-on experience with AWS data services, including Lambda, Glue, SNS, and orchestration services
- Strong hands-on experience with Python and PySpark
- Advanced SQL, query optimization, and data transformation experience
- Experience developing and optimizing Amazon Redshift solutions
- Experience with Amazon RDS database development and management
- Experience developing and maintaining ETL packages and workflows using SSIS
- SQL Server development and troubleshooting using SSMS
- Experience with Terraform, YAML, Git, Podman, and VS Code
- Strong problem-solving and analytical skills
- Excellent communication and collaboration skills
- Ability to work independently and within a cross-functional team
- Experience with Connect
- Experience with TOLSE or TOLENT
- Experience with CI/CD and DevOps practices
- Experience working in Agile/Scrum environments
- Knowledge of cloud security, IAM, monitoring, and logging
What We Do
A premiere data services company serving clients in North America, Datavail has 1,000 data professionals, data engineers, developers, project managers, consultants, and business experts, supported by industry-leading automation and intellectual property. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes. At Datavail, we look for more than smarts, experience and proficiency. On top of those requirements, we seek people who mesh with our corporate values. We seek brilliance without bravado and know-how without a know-it-all attitude. We hold low ego in high regard, embrace problem-solving as a passion and welcome every day as a new opportunity to learn. We’re flexible and hard working. We’re committed to our clients and colleagues. We help our people grow so they can help our clients grow. That makes us grow so we can help even more customers leverage organizational data for business value. Our Core Values: 1. We desire to serve. 2. We embody flexibility for availability 3. We exemplify low ego. 4. We work hard. 5. We strive for continuous improvement. 6. We are growth-oriented.
