Join our engineering team to manage high-impact data ingestion pipelines and complex web scraping configurations. This is an opportunity for a mid-level engineer to take ownership of data flow reliability and scalability within a cloud-native environment.
- Maintain and manage website scraping configurations using Python-based frameworks.
- Proactively monitor scraping workflows for errors, crashes, and performance bottlenecks.
- Develop and maintain data ingestion pipelines to move and transform data into the Datalake.
- Collaborate with internal stakeholders to define and refine requirements for new scraping tasks.
- Prepare and share periodic reports regarding scraping activities, data health, and operational status.
- Proven experience in Data Engineering and ETL development.
- Strong proficiency in Python and SQL.
- Hands-on experience with orchestration tools like Apache Airflow.
- Experience using DBT for data transformations.
- Proficiency with Azure cloud services and infrastructure-as-code using Terraform.
- Solid understanding of version control with Git.
- Professional fluency in English (written and spoken).
- Experience with NoSQL databases.
- Fluency or working knowledge of French.
- Reduction in scraping configuration downtime and error rates.
- Timely delivery of data ingestion pipelines aligned with stakeholder needs.
- Maintenance of clean, documented, and efficient Python codebases.
We offer a comprehensive benefits package designed to support your professional growth and personal well-being.
This is a fully remote position that requires a self-motivated individual comfortable with asynchronous communication and high levels of technical documentation.
We are an equal opportunity employer and value diversity. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.