Ayan Infotech are looking for a Senior Databricks Developer for a long term contract job opportunity in Sydney.
Due to the nature of the work involved, we can only consider applicants who hold full unrestricted work rights in Australia and can work from the Sydney office two days a week at teh minimum.
Title: Senior Databricks Developer
Location: Sydney
Type: Contract (6 months +)
Required Skills and Experience:
- Minimum 5+ years of IT experience with 3+ years of hands-on experience in designing, developing, implementing, and maintaining data solutions with Databricks. This role requires close collaboration with various teams to gather data requirements, build scalable data pipelines, and ensure the overall solution is reliable, available, and optimized for high performance. Detailed skills are outlined below:
- Proven expertise with Lakehouse Architecture with Delta Tables - Delta tables, Schema Evolution, ACID compatibility, History capture etc.
- Integrating Databricks with various ETL/Orchestration tools and perform required operations.
- Strong expertise in fundamental areas like Spark ecosystems, Data frame API & Spark SQL with Python/Scala and SQL as language
- Hands on knowledge on Databricks native features like Workflows, Delta Live Tables, Unity Catalog etc.
- Expertise in workspace and cluster configuration, performance tuning, cost optimization, data security and log monitoring.
- Experience with Spark Structured Streaming jobs with Autoloader integrating with different streaming platforms.
- Experience in building optimal data pipeline architecture with knowledge and best practices from Databricks platform.
- Fair understanding of Data Modelling and defining conceptual logical and physical data models.
- Nice to have - exposure to AI/Gen AI skill and/or any associate or professional level Databricks/Cloud certifications.
We will be able to contact only shortlisted candidates. We thank you in advance for your interest.
www.ayaninfotech.com
