Requirements
- Familiar with BigQuery, can code SQL.
- Familiar with Cloud composer / airflow.
- Familiar with IAM, service account.
- Familiar with Data catalog.
- Understand Infrastructure as Code.
- Good to have knowledge with Dataflow, K8s, Vertex AI pipeline, Kubeflow pipeline.
Required Skills
Python
- Deep knowledge about python programing, practice OOP, following coding best practice, know how to use flake8, mypy, black, SonarQube and pre-commit.
- Deep knowledge in unit test and end to end test, familiar with Pytest, fixtures, unittest etc.
Unix
- Familiar with popular Unix system, know how to install sth in docker.
- Familiar with shell.
Git
- Know how to create PR and solve merge conflict.
- Can create CI/CD pipeline in either Github Action or Azure DevOps using best practice
Docker
- Deep understanding with Docker
DBT
- Deep Knowledge in DBT, preferably with GCP.
SQL
- Deep knowledge of SQL
- Deep understanding with Data modeling, system design
