AYAN InfoTech is looking for Databricks Developer to join an exciting project based in Sydney. The role offers you the opportunity to contribute towards an extremely well structured and mature environment, working on sophisticated enhancement projects.
Role: Databricks Developer
Location: Sydney
Contract Duration: 6 Months with high possible extensions
Experience: 6+ Years
Required Skills & Experience:
- Minimum 5+ years of IT experience with 3+ years of hands-on experience in designing, developing, implementing, and maintaining data solutions with Databricks.
- This role requires close collaboration with various teams to gather data requirements, build scalable data pipelines, and ensure the overall solution is reliable, available, and optimized for high performance.
- Proven expertise with Lakehouse Architecture with Delta Tables - Delta tables, Schema Evolution, ACID compatibility, History capture etc.
- Integrating Databricks with various ETL/Orchestration tools and perform required operations.
- Strong expertise in fundamental areas like Spark ecosystems, Data frame API & Spark SQL with Python/Scala and SQL as language
- Hands on knowledge on Databricks native features like Workflows, Delta Live Tables, Unity Catalog etc.
- Expertise in workspace and cluster configuration, performance tuning, cost optimization, data security and log monitoring.
- Experience with Spark Structured Streaming jobs with Autoloader integrating with different streaming platforms.
- Experience in building optimal data pipeline architecture with knowledge and best practices from Databricks platform.
- Fair understanding of Data Modelling and defining conceptual logical and physical data models.
- Nice to have - exposure to AI/Gen AI skill and/or any associate or professional level Databricks/Cloud certifications.
Contact: 61-(02) 7207 6926 for more details.
Please note we will be able to contact only shortlisted candidates for this role. We thank you in advance for your interest.
