Senior Azure Data Engineer (Ingestion & Productisation)

About the Role
We are looking for a Senior Azure Data Engineer to play a key role in scaling and productising our Business Intelligence (BI) services. This position sits within our Ingestion team, where you will be responsible for designing and optimising data pipelines, ensuring seamless data integration, and productising our data platform services into scalable, repeatable solutions.

In this role, you will work alongside skilled professionals, leveraging MS Fabric, PySpark, and Azure technologies in an Agile environment. Your contributions will be instrumental in enhancing data architecture, driving efficiencies, and fostering a knowledge-sharing culture within the team.

Key Responsibilities
- Design, build, and optimize scalable data pipelines using PySpark within MS Fabric.
- Develop and manage API connections and authentication processes for seamless data integration.
- Implement and maintain robust ETL/ELT processes to ensure efficient data processing and transformation.
- Collaborate with business and technology teams to ensure data accessibility, quality, and security.
- Utilize Azure DevOps for CI/CD, pipeline automation, and project management.
- Work in an Agile/Scrum environment, managing sprints, backlogs, and development cycles efficiently.
- Play a key role in mentoring junior team members and fostering a knowledge-sharing culture within the team.

Required Skills & Experience

Technical Skills:
- 3+ years of experience in data engineering, working on commercial applications.
- Proven experience delivering at least one commercial project using MS Fabric.
- Strong understanding of Delta Lake architecture and Delta load frameworks.
- Strong hands-on experience with PySpark for data processing within Notebooks.
- Solid understanding of API connection and authentication processes for data integration.
- Experience with data analytics and Power BI for reporting and visualization is a plus.
- Familiarity with Agile/Scrum methodologies and sprint-based development cycles.

Soft Skills:
- Excellent communication skills, capable of articulating technical concepts to non-technical stakeholders.
- Ability to work with minimal direction and take initiative in problem-solving.
- Forthcoming and proactive in identifying challenges and suggesting improvements.
- Strong attention to detail, ensuring data accuracy and pipeline efficiency.
- Project management knowledge, including an understanding of sprints, backlog grooming, and resource planning.
- AI-optimized mindset, with an ability to leverage AI tools for improved productivity and efficiency.

Team & Work Environment
- You will be part of a newly formed data engineering team, focused on delivering high-quality, scalable data solutions.
- Work directly with the Australian CTO, contributing to strategic technical direction.
- Long-term opportunity to shape and optimize data architecture within a rapidly growing company.

Why Join Us?
- Work in a dynamic and innovative environment with cutting-edge technologies.
- Play a pivotal role in building a high-performing data engineering team.
- Enjoy long-term career growth opportunities with a forward-thinking company.
- Flexible work arrangements and a collaborative team culture.

If you are a driven and experienced Senior Azure Data Engineer looking for a challenging and rewarding role, we’d love to hear from you! Apply now and be part of our exciting journey in delivering world-class data solutions.