Connecting...

W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9hewfulwluzm90zwnol2pwzy9iyw5uzxitzgvmyxvsdc5qcgcixv0

Data Engineer

Job Title: Data Engineer
Contract Type: Contract
Location: Melbourne, Victoria
Industry:
Salary: Negotiable
Reference: 4386_1632292739
Contact Name: Radha Krishnan
Contact Email: radha.k@ayaninfotech.com
Job Published: September 22, 2021 16:39

Job Description


Role: Data Engineer
Location: Melbourne Work From Home
Mode of Employment: 12 Months contact

JOB DESCRIPTION:

  • Build robust, efficient and reliable data pipelines consisting of diverse data sources to ingest and process data into cloud-based data platforms using Python, Scala, bash or Java as programming language.
  • Design and develop batch and streaming data ingestion pipelines from various source systems to cloud-based data and analytics platform
  • Transform data using data mapping and distributed data processing capabilities like Databricks, Apache beam, Apache Spark etc.
  • Collaborate with Security and Operations teams to develop and enforce proper data security and privacy practices
  • Collaborate with Architects to define the architecture and technology selection.?
  • Work with stakeholders including the Product Owner and data analyst teams to assist with data-related technical issues and support their data infrastructure needs
  • Share knowledge with immediate peers and build communities and connections that promote better data engineering practices across the organisation



Required skills:

  • Proven working experience as data engineer for 2+ years preferably in building data lake solution by ingesting and processing data from various source systems
  • Minimum one year of experience with cloud-based data components from one of the leading public cloud-based providers and/or Snowflake
  • Understanding of SDLC processes, experience of working in a large-scale program, data warehousing and ETL development?
  • Experience using DevOps methods and source control disciplines to build/release new features on enterprise data platforms?
  • Share knowledge and best practices with peers that promote better technical practices across the organisation??
  • Experience in building various frameworks for enterprise data lake is highly desirable??
  • Experienced in Agile / Scrum projects, ideally SAFE


Role: Data Engineer
Location: Melbourne Work From Home
Mode of Employment: 12 Months contact

JOB DESCRIPTION:

  • Build robust, efficient and reliable data pipelines consisting of diverse data sources to ingest and process data into cloud-based data platforms using Python, Scala, bash or Java as programming language.
  • Design and develop batch and streaming data ingestion pipelines from various source systems to cloud-based data and analytics platform
  • Transform data using data mapping and distributed data processing capabilities like Databricks, Apache beam, Apache Spark etc.
  • Collaborate with Security and Operations teams to develop and enforce proper data security and privacy practices
  • Collaborate with Architects to define the architecture and technology selection.?
  • Work with stakeholders including the Product Owner and data analyst teams to assist with data-related technical issues and support their data infrastructure needs
  • Share knowledge with immediate peers and build communities and connections that promote better data engineering practices across the organisation



Required skills:

  • Proven working experience as data engineer for 2+ years preferably in building data lake solution by ingesting and processing data from various source systems
  • Minimum one year of experience with cloud-based data components from one of the leading public cloud-based providers and/or Snowflake
  • Understanding of SDLC processes, experience of working in a large-scale program, data warehousing and ETL development?
  • Experience using DevOps methods and source control disciplines to build/release new features on enterprise data platforms?
  • Share knowledge and best practices with peers that promote better technical practices across the organisation??
  • Experience in building various frameworks for enterprise data lake is highly desirable??
  • Experienced in Agile / Scrum projects, ideally SAFE