59日本XXXXXXXXX视频图片,391aa亚洲色欲

登錄/注冊我要招人

400-885-9898

更新于 11月10日

APP

舉報

Senior Data Engineer (Vendor)

1.1-1.8萬

天津西青區(qū)
5-10年
學歷不限
全職
招1人

職位描述

數(shù)據(jù)倉庫數(shù)據(jù)庫開發(fā)數(shù)據(jù)治理

osition: Senior Data Engineer – Vendor
Experience: 6–9 Years
Role Summary
We are seeking Senior Data Engineer resources to work on the migration of applications from our legacy Cloudera environment to the new Kubernetes-based data platform. The role requires strong hands-on development skills in data engineering, with the ability to deliver high-quality pipelines under guidance from internal leads.

Key Responsibilities

? Develop and optimize data pipelines using Spark 3.5 and Python/Scala.
? Migrate existing Hive, Spark, and Control-M jobs to Airflow and DBT-based workflows.
? Integrate data pipelines with messaging systems (Kafka, Solace) and object stores (S3, MinIO).
? Troubleshoot and optimize distributed jobs running in Kubernetes environments.
? Collaborate closely with internal leads and architects to implement best practices.
? Design and implement migration/acceleration framework to automate end to end migration.
? Continuous enhancements to the frameworks to ensure the stability, scalability and support for diverse use cases and scenarios.
? Work with various data applications to enable and support the migration process.
? Deliver assigned migration tasks within agreed timelines.

Required Skills

? 6–9 years of hands-on data engineering experience.
? Strong expertise in Apache Spark (batch + streaming) and Hive.
? Proficiency in Python, Scala, or Java.
? Knowledge of orchestration tools (Airflow / Control-M) and SQL transformation frameworks (DBT preferred).
? Experience working with Kafka, Solace, and object stores (S3, MinIO).
? Exposure to Docker/Kubernetes for deployment.
? Hands on experience of data Lakehouse formats (Iceberg, Delta Lake, Hudi).