AYAN InfoTech is looking for Data Bricks / Data Engineering to join an exciting project based in Sydney. The role offers you the opportunity to contribute towards an extremely well structured and mature environment, working on sophisticated enhancement projects.
Role: Data Bricks / Data Engineering
Location: Sydney
Contract Duration: 5 Months with high possible extensions
Experience: 7+ Years
Required Skills & Experience:
- Primary Skills - Azure DataBricks, Azure DevOps
- Databricks Development: Design and develop scalable data pipelines using Azure Databricks, PySpark, and SQL
Implement structured streaming, Delta Lake, and Unity Catalog.
Optimize Spark jobs and manage cluster performance.
Experience in a production support environment.
Expertise with Azure Databricks solutions. - DevOps Integration: Build and maintain CI/CD pipelines using Azure DevOps, GitHub, Jenkins, and Terraform.
- Automate deployment of notebooks, clusters, secrets, and data workflows.
- Manage version control, branching strategies, and code reviews.
- BODS ETL skills are a significant advantage as the scope of work includes supporting BODS ETL framework-based Oracle database reporting systems.
- Cloud & Security: Work with Azure services: Data Factory, Blob Storage, Key Vault, Synapse, EventHub, and Power BI.
- Implement RBAC/ABAC, encryption, and data masking for secure data handling.
- Collaboration & Documentation: Collaborate with cross-functional teams for requirement gathering and solution design.
- Document technical specifications, data dictionaries, and architecture blueprints.
Contact: 61-(02) 7207 6926 for more details.
Please note we will be able to contact only shortlisted candidates for this role. We thank you in advance for your interest.
