от компании (организации): Lamoda в городе (населённом пункте): Москва, Россия
в отрасли экономики "Информационные технологии, интернет, телеком" → "Электронная коммерция"
с заработной платой: по договоренности
Вакансия № 6808238 добавлена в базу данных сайта Работа в Москве и Московской области (МО, Подмосковье): Понедельник, 21 октября 2024 года.
Дата обновления вакансии № 6808238 на сайте Работа в Москве и Московской области (МО, Подмосковье): Пятница, 15 ноября 2024 года.
Обращаем Ваше внимание, что на момент обращения к работодателю вакансия № 6808238 может быть уже занята. Администрация сайта Работа в Москве и Московской области (МО, Подмосковье) приносит извинения за доставленные неудобства.
Требования к опыту работы:
1–3 года
Тип занятости:
полная занятость
График работы:
полный день
Дополнительные сведения о вакансии: Data engineer
Lamoda, tech.lamoda.ru - is a complex highload project, almost 300 specialists work on it, including developers, QAs, analysts, system administrators, product managers and designers.
We develop, test and support over 60 services. These are systems that automatize processes of the fulfillment center, contact centers, photo studio, delivery service and other operational systems. Apart from that we create high-loaded services: website, mobile API, catalogue, personalization and obviously develop website and mobile applications front-end.
What you’ll be doing:
- Design, build and maintain reliable and scalable enterprise level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
- Optimize jobs to utilize Kafka, Hadoop, Spark resources in the most efficient way
- Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
- Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
- Perform code review
- Provide mentorship and guidance to junior team members
Team Responsibilities:
- Performance tuning, monitoring, capacity planning of Hadoop cluster and other infrastructure
- Ingest, validate and process internal & third party data
- Create, maintain and monitor data flows for consistency, accuracy and lag time
- Maintain and enhance framework for jobs (Spark Streaming, Spark, MapReduce, aggregate jobs in Hive)
- Train Developers/Analysts on tools to pull data
- Tool evaluation/selection/implementation
- Review/Approval - DDL for database, Hive Framework jobs and Spark Streaming to make sure they meet our standards
Required Skills:
- BA/BS degree in Computer science or related field
- 5+ years of software engineering experience
- 3+ years of production development experience on Scala/Java
- Experience in Hadoop stack (YARN, HDFS, HBase, Hive)
- Experience in Spark
- Experience in Kafka
- Experience in Docker, Kubernetes is a plus
- Knowledge of version control systems (git)
- Proficiency in Linux
- Experience in Python is a huge plus
- Strong understanding of RDBMS, SQL;
- Passion for engineering and computer science around data
Technologies We Use:
- Luigi / Airflow - for job scheduling
- Docker - Packaged container image with all dependencies
- Hive - SQL data warehouse layer for data in HDFS
- Kafka - distributed commit log storage
- Yarn - Distributed cluster resource manager
- Spark Streaming - Near time aggregation
- SQL Server - Reliable OLTP RDBMS
What can you expect?
- Challenging and interesting atmosphere in a young team of professionals;
- Official employment, social package;
- Office in city center m. Paveletskaya.
- Competitive salary;
- Free English courses;
- Discount on the entire range of products.
- High exposure to different departments, data sources and business problems.
- Collaboration and discussion with our technology partners (e.g. Facebook, Google, Yandex, VK, Amazon).
- Exposure to the newest technologies in the data science space.
Откликнуться на эту вакансию: Data engineer
Предыдущая вакансия:
Вакансия № 6808236 на должность Ведущий специалист отдел смет и ценообразования от компании АО ДСК АВТОБАН в городе (населенном пункте) Москва