Data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They support functional streams of work and are accountable for timely delivery. They work on the latest big data tools, frameworks and offerings (data mesh, etc.), while also being involved in enabling credible and collaborative problem solving to execute on a strategy.
- You will collaborate with team members to design intricate data processing pipelines, addressing clients' most challenging problems.
- You will collaborate with data scientists to design scalable implementations of their models.
- You will write clean, iterative code using TDD and leverage various continuous delivery practices to deploy, support and operate data pipelines.
- You will apply different standard models for big data and create data models for at least one type of modeling technique.
- You will incorporate data quality into your day-to-day work.
- Must have-Experience with Databricks (SparkSQL, PySpark, Spark DataFrames) and open table format design & optimisations (Delta Lake, Apache Iceberg).
- 2-5 years professional experience in data engineering or related fields.
- Working knowledge of modern data architecture patterns (Medallion, data mesh, data product approach).
- Experience with dbt (core/cloud) and advanced SQL for analytical transformations, streaming and batch data ingestion frameworks.
- Experience with using orchestration/workflow tools.
- Attained a Cloud Data Engineer certification (GCP, AWS, or Azure) or a Databricks certification (Data Engineer Associate).
- Working knowledge of DevOps platforms (GitLab/GitHub CI/CD, Terraform).
- Experience with data quality, testing, and governance (Great Expectations, dbt tests, data contracts).
- Experience with Python engineering skills with software engineering best practices.
- Working knowledge of LLMs and associated technologies such as vector databases and graph databases for semantic search.
- Working knowledge of containerisation.
- Working knowledge of the delivery of data governance frameworks including security and compliance in cloud data environments.
- Open-source contributions to data or AI projects.
- Experience working in large data teams, collaborating with business, analysts, and data scientists.
- English proficiency for international stakeholder engagement (requirements, delivery, issue triage).
There is no one-size-fits-all career path at Thoughtworks: however you want to develop your career is entirely up to you. But we also balance autonomy with the strength of our cultivation culture. This means your career is supported by interactive tools, numerous development programs and teammates who want to help you grow. We see value in helping each other be our best and that extends to empowering our employees in their career journeys.
Thoughtworks is a dynamic and inclusive community of bright and supportive colleagues who are revolutionizing tech. As a leading technology consultancy, we’re pushing boundaries through our purposeful and impactful work. For 30+ years, we’ve delivered extraordinary impact together with our clients by helping them solve complex business problems with technology as the differentiator. Bring your brilliant expertise and commitment for continuous learning to Thoughtworks. Together, let’s be extraordinary.
#LI-Onsite
See here our AI policy.
Top Skills
What We Do
We are a leading global technology consultancy that integrates strategy, design and software engineering to enable enterprises and technology disruptors across the globe to thrive as modern digital businesses.
Why Work With Us
As technologists, we have a unique role to play in how technology should benefit all of society, pursuing a more equitable future. Part of that role is to continuously educate ourselves on the issues that matter to the causes we believe in. We recognize our privilege and strive to see the world from the perspective of the most vulnerable.
Gallery
.jpeg)
