Data Engineer (Hadoop Stack)

Job Title: Data Engineer (Hadoop Stack)
Contract Type: Contract
Location: Melbourne, Victoria
Salary: Negotiable
Reference: 3797_1606792177
Contact Name: Mailangi Styles
Contact Email:
Job Published: December 01, 2020 14:09

Job Description

Ayan Infotech is in urgent need of numerous Data Engineers to take up an initial 6-month contract in Melbourne or Sydney. The role will require working on a complex business reengineering development projects and to solve organisational problems through IT solutions using Big Data concepts, Platform tools and software engineering languages as needed.

Due to the COVID19 pandemic, the work will be purely on a remote/WFH basis.

Key Responsibilities include:

* Designing, developing and implementing solutions to customers organisational problems;
* Providing high quality service delivery on engagements;
* Designing and developing data reconciliation & metadata driven data processing frameworks;"

Key Skills & Experience required:
*Minimum 2 years previous experience as a Data Engineer;
*2+ years of AWS hands on technical experience;
*2+ years experience in programming eg. Core Java/Python/Scala programming;
*2+ years experience in Hadoop Stack eg. Hive, Spark (Core and Streaming), Kafka, Flink, NIFI;
* Experience working on stream transform using Spark Streaming or Flink;
* Experience working on change data capture in a big data ecosystem;
* Experience in building real time or batch ingestion and transformation pipelines;
* Experience supporting a production service in a DevOps friendly environment;
* Experience working in Agile/Scrum teams;

Advantageous skills:
* Hands on experience in API and cloud-native architecture;
* Experience with Github and Jenkins;
* Experience with 'container' technology (e.g. Docker, Kubernetes);
* Experience wrangling data using library such as Pandas, Scikit-learn or Numpy;
* Experience using notebook such as Jupyter or Polyglot;
* Experience with test-driven development, test automation & continuous delivery;
* Experience in Java build automation technologies such as Gradle, Maven, Integration with Sonarqube etc.;
* Understanding of Financial Services Industry could be advantageous;

* Education level - Degree in Computer Science or related field;
* A minimum of 3 years industry experience;
* AWS certification would be advantageous;
* Candidate must have relevant VISA's, Australian PR or Australian Citizenship

Junior daily rates are on offer, and there is strong potential for the contract to be extended.

If interested, click the 'APPLY NOW' button or email your CV.

Contact - 02 9411 8794