As a Data Engineer, you will be instrumental in refining our data operations to enhance efficiency and quality across various systems. Your primary responsibilities will include:
In this role you will:
-
Lead Strategic Data Projects: develop innovative data solutions that process tens of terabytes from various sources.
-
Build Data Pipelines: Build end-to-end large-scale batch and real-time data pipelines and scalable architectures and services to efficiently handle large amounts of data
-
Work with state-of-the-art technology stack: utilize technologies/tools like Kafka/Kafka Connect, Flink, Airflow, Soda, Snowflake, BigQuery, AWS Analytics Ecosystem.
-
Manage Company Data: Organize storage and transfer of data across diverse platforms, including PostgreSQL, S3, BigQuery, and Snowflake.
-
Contribute to Data Quality: Design and implement a company-wide framework for data quality and data observability.
About you:
-
5+ years of experience working with & writing clean and efficient code in Python, SQL.
-
Deep understanding of Databases and Data Warehouses fundamentals.
-
Experience with cloud platforms (preferably AWS or GCP), and cloud DWH (preferably BigQuery or Snowflake).
-
Experience with Apache Kafka.
-
Experience with workflow orchestration tools (e.g. Apache Airflow).
It would be great you have:
-
Hands-on experience with modern ELT tools (dbt/Fivetran).
-
Hands-on experience with Kafka Connect or Debezium.
-
Hands-on experience in MLOps.
