Data Engineer (AWS, Big Query, ETL (AWS Glue), Amazon S3,Snowflake,GLUE)

Posted 5 Days Ago
Be an Early Applicant
Hyderabad, Telangana
Senior level
Information Technology • Consulting
The Role
The Data Engineer will design and develop scalable data ingestion pipelines, enhance data flows between systems, and support data migration to Snowflake. Responsibilities include working with stakeholders to define data needs, improving data quality, and documenting processes, while leveraging a range of cloud and data technologies, particularly within AWS.
Summary Generated by Built In

Description
  • At least 8+ years of extensive experience in data warehousing using both RDBMS and non-RDBMS databases
  • Minimum of 5 years of recent hands-on professional experience as a data engineer (backend software engineer also considered), actively involved in coding
  • Professional experience in an agile, dynamic, and customer-facing environment is required
  • Strong understanding of distributed systems and cloud technologies, with preference given to experience with AWS
  • Preferably experienced in data streaming and scalable data processing
  • Experience with large-scale datasets, data lake, and data warehouse technologies such as AWS Redshift, Google Big Query, and strongly preferred is Snowflake
  • Minimum of 2+ years of experience in Amazon RDS, Amazon Kinesis, Amazon Lambda, Apache Airflows, Amazon Step Functions
  • Demonstrated expertise in scripting languages like Python, UNIX shell, and Spark is required
  • Understanding of RDBMS, data ingestion, data flows, and data integrations, among others
  • Technical proficiency in data models, data mining, and segmentation techniques
  • Experience with full SDLC lifecycle and Lean or Agile development methodologies
  • Knowledge of CI/CD and GIT deployments
  • Ability to work effectively in a diverse, multiple stakeholder environment
  • Ability to effectively communicate complex technology solutions to various teams, including technical, business, and management teams
Requirements
  • Work with stakeholders to understand needs for data structure, availability, scalability, and accessibility.
  • Develop tools to improve data flows between internal/external systems and the data lake/warehouse.
  • Build robust and reproducible data ingest pipelines to collect, clean, harmonize, merge, and consolidate data sources.
  • Understanding existing data applications and infrastructure architecture
  • Build and support new data feeds for various Data Management layers and Data Lakes
  • Evaluate business needs and requirements
  • Support migration of existing data transformation jobs in Oracle, and MS-SQL to Snowflake.
  • Ability to write oracle sql, PLSQL scripts and should have demonstrated working experience in managing the oracle scripts.
  • Should have a good hands on with linux scriptings.
  • Able to document the processes and steps.
  • Develop and maintain datasets.
  • Improve data quality and efficiency.
  • Lead Business requirements and deliver accordingly.

Top Skills

Python
Spark
The Company
96 Employees
On-site Workplace
Year Founded: 2012

What We Do

Unison Consulting was launched in Singapore on September 2012, the hub of the financial industry, with innovative visions in the technocratic arena. We are a boutique next-generation Technology Company with strong business-interests in Liquidity risk, Market Risk, Credit Risk and Regulatory Compliance.

Unison provides technology consulting and services to implement Risk Management and Risk Analytics System for Financial Institutions. Our services suite comprises of Techno-Functional consulting, systems integration, Business Intelligence, information management, and custom development of IT solutions, plus project management expertise for financial institutions.

We have expertise in latest cutting edge technology to achieve better total cost of ownership. Through our qualified professionals, we assist you drive your unique risk management strategies, whether that means efficient monitoring, improving risk appetite of the financial institutions, complying with regulations, or capturing growth opportunities through innovation, this is what maximizes your decision taking potential. At Unison Consulting, we view clients as partners, and our success is only measured by the success of our partners. So we put it all on the table in order to exceed expectations.

Our staff consists of young, energetic and innovative consultants who are never afraid to challenge the conventions and push the boundaries in an effort to help our clients. For every project, no matter how large or how small, we strive to not only meet your needs, but deliver a showcase in your field

Similar Jobs

TransUnion Logo TransUnion

Sr Analyst

Big Data • Fintech • Information Technology • Business Intelligence • Financial Services • Cybersecurity • Big Data Analytics
Hybrid
Hyderabad, Telangana, IND
13000 Employees

Warner Bros. Discovery Logo Warner Bros. Discovery

Manager - Data

Artificial Intelligence • Digital Media • Gaming • Machine Learning • News + Entertainment • Software
Hybrid
Hyderabad, Telangana, IND
40000 Employees

Warner Bros. Discovery Logo Warner Bros. Discovery

Sr. Manager - Data

Artificial Intelligence • Digital Media • Gaming • Machine Learning • News + Entertainment • Software
Hybrid
Hyderabad, Telangana, IND
40000 Employees

Warner Bros. Discovery Logo Warner Bros. Discovery

Senior Machine Learning Engineer

Artificial Intelligence • Digital Media • Gaming • Machine Learning • News + Entertainment • Software
Hybrid
Hyderabad, Telangana, IND
40000 Employees

Similar Companies Hiring

Silverfort Thumbnail
Security • Sales • Information Technology • Cybersecurity • Automation
GB
357 Employees
Jobba Trade Technologies, Inc. Thumbnail
Software • Professional Services • Productivity • Information Technology • Cloud
Chicago, IL
45 Employees
InCommodities Thumbnail
Renewable Energy • Machine Learning • Information Technology • Energy • Automation • Analytics
Austin, TX
234 Employees

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account