Data Engineer - Traffic Mart

About The Team

The Marketplace Intelligence and Data group focuses on creating sustainable and efficient data and intelligence products that support Shopee's business development. This team manages the data warehouse for Shopee's e-commerce platform, develops products for merchants and operations data, and handles comprehensive traffic data. They also work on product-related algorithms, such as product launch, control, and information optimization, as well as marketing algorithms including merchandising, product selection, recommendation systems, evaluation algorithms, and user profiling. Additionally, the team develops foundational AI technologies like machine translation, speech and image recognition algorithms, and real-person authentication.

Job Description

• Engage in the development and maintenance of the Marketplace data warehouse encompassing offline and real-time data collection, building the public data layer, constructing the data application layer, and enforcing data governance practices.
• Support data-related needs across various business sectors by collaborating with cross-functional teams to gather requirements, design efficient data architectures, and deliver optimized data solutions. These solutions cater to BI analysis, data products, and algorithm integration.
• Contribute to the exploration and innovation of key data technologies for the Marketplace. Be accountable for enhancing the current data architecture, improving data quality and productivity, managing massive and real-time data processing, and incorporating emerging technologies.

Candidate Requirements

• Possess a Bachelor's degree or higher in Computer Science or a related discipline.
• Have a minimum of three years’ experience in data engineering.
• Experience or understanding of Traffic Mart is a plus.
• Skilled in one or more big data technologies such as Apache Spark, Flink, Hadoop, HBase, Kafka, Druid, Clickhouse, among others.
• Proficient in programming languages including Java, Scala, Python, and SQL.
• Good grasp of data warehouse concepts, with practical experience in big data architecture design, data modeling, and performance optimization.