Staff Data Engineer

2. Staff Data Engineer


Key responsibilities

1. Design the Collections Snowflake and PySpark architecture.
2. Lead integration with commbank.data.
3. Model HPS card and account data for Collections use cases.
4. Define data contracts and schema-evolution standards.
5. Establish automated data-quality and reconciliation controls.
6. Design restartable, idempotent and recoverable pipelines.
7. Optimise Snowflake and PySpark workloads.
8. Establish production monitoring and data-pipeline observability.
9. Lead technical reviews and complex data investigations.
10. Mentor Senior Data Engineers and Data Engineers.
11. Work closely with Cards MOD, PCC and platform teams.

Essential skills

a) Strong production Snowflake experience.
b) Strong production PySpark / Spark experience.
c) Python.
d) Advanced SQL.
e) AWS.
f) S3.
g) Enterprise data engineering.
h) ETL/ELT and data modelling.
i) Data quality and reconciliation.
j) CI/CD and automated testing.
k) Production support.

Highly regarded

EMR / EMR Serverless.
Step Functions.
Lambda.
Cards or payments data.
Collections or lending.
PowerCurve Collections. 

 

Key responsibilities

 

1. Design the Collections Snowflake and PySpark architecture.

2. Lead integration with commbank.data.

3. Model HPS card and account data for Collections use cases.

4. Define data contracts and schema-evolution standards.

5. Establish automated data-quality and reconciliation controls.

6. Design restartable, idempotent and recoverable pipelines.

7. Optimise Snowflake and PySpark workloads.

8. Establish production monitoring and data-pipeline observability.

9. Lead technical reviews and complex data investigations.

10. Mentor Senior Data Engineers and Data Engineers.

11. Work closely with Cards MOD, PCC and platform teams.

 

Essential skills

 

a) Strong production Snowflake experience.

b) Strong production PySpark / Spark experience.

c) Python.

d) Advanced SQL.

e) AWS.

f) S3.

g) Enterprise data engineering.

h) ETL/ELT and data modelling.

i) Data quality and reconciliation.

j) CI/CD and automated testing.

k) Production support.

 

Highly regarded

 

EMR / EMR Serverless.

Step Functions.

Lambda.

Cards or payments data.

Collections or lending.

PowerCurve Collections.

 

When you apply, you voluntarily consent to the disclosure, collection and use of your personal data for employment/recruitment and related purposes in accordance with the Tech Aalto Privacy Policy, a copy of which is published at Tech Aalto’s website (https://www.techaalto.com/privacy/)

Confidentiality is assured, and only shortlisted candidates will be notified for interviews.