Important Information
Location: Brazil
Job Mode: Full-time
Work Mode: Work from home
Job Summary
As a Data Engineer, you will play a pivotal role in developing and optimizing data pipelines, integrating diverse data sources, and implementing advanced search algorithms to support LLM/RAG systems. Collaborating closely with cross-functional teams, you will design and maintain scalable data architectures, ensuring efficient data ingestion, transformation, and retrieval.
Responsibilities and Duties
- Implement the ingestion and integration of multiple data sources (audio, video, PowerPoint presentations, documents, etc.);
- Design and implementation of data structure within appropriate storage solutions;
- Enhance data quality through version control and updates;
- Implement efficient search algorithms to optimize data retrieval;
- Participate in the design of an efficient RAG system;
- Hands-on programming using Python.
Essential Skills
- Strong proficiency in Python programming with experience in building scalable and maintainable codebases;
- Solid understanding of embedding models, vector databases, and similarity search techniques;
- Hands-on experience with LLM frameworks such as LangChain;
- Knowledge of data preprocessing techniques for handling textual, audio, and video data;
- Practical expertise with tools like FAISS, Pinecone, Weaviate, or similar is highly desirable;
- Data and ML Knowledge;
- ETL/ELT: Familiarity with data processing workflows or pipelines.
- Experience with file ingestion processes for diverse formats (e.g., text, graphics, tables, images, PPT, video);
- Experience with RAG (Retrieval-Augmented Generation) workflows and their integration with LLMs;
- Understanding of vector mathematics, including cosine similarity and Euclidean distance.
About Encora
Encora is the preferred digital engineering and modernization partner of some of the world’s leading enterprises and digital native companies. With over 9,000 experts in 47+ offices and innovation labs worldwide, Encora’s technology practices include Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering.
At Encora, we hire professionals based solely on their skills and qualifications, and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.
What We Do
Headquartered in Santa Clara, California, and backed by renowned private equity firms Advent International and Warburg Pincus, Encora is the preferred technology modernization and innovation partner to some of the world’s leading enterprise companies. It provides award-winning digital engineering services including Product Engineering & Development, Cloud Services, Quality Engineering, DevSecOps, Data & Analytics, Digital Experience, Cybersecurity, and AI & LLM Engineering. Encora's deep cluster vertical capabilities extend across diverse industries, including HiTech, Healthcare & Life Sciences, Retail & CPG, Energy & Utilities, Banking Financial Services & Insurance, Travel, Hospitality & Logistics, Telecom & Media, Automotive, and other specialized industries.
With over 9,000 associates in 47+ offices and delivery centers across the U.S., Canada, Latin America, Europe, India, and Southeast Asia, Encora delivers nearshore agility to clients anywhere in the world, coupled with expertise at scale in India. Encora’s Cloud-first, Data-first, AI-first approach enables clients to create differentiated enterprise value through technology