About Me
I'm a Senior Data Engineering Consultant with 10+ years of experience spanning data engineering, analytics, and actuarial domains. Based in Adelaide, South Australia, I specialise in Microsoft Fabric, Azure, and modern medallion Lakehouse architectures.
My expertise lies in designing and delivering scalable, metadata-driven data platforms that optimise cloud performance and cost while driving measurable business impact. I'm passionate about thought leadership in data engineering, producing blogs, case studies, and enablement content to promote modern practices and innovations.
Technical Stack
Platforms & Tools
Languages & Development
Data Architecture
Methodologies
AI & Development Tools
Experience
Senior Consultant & Capability Lead - Data Engineering
Promoted to Senior Consultant and Data Engineering Capability Lead in just over one year. Led the productisation of the Microsoft Fabric Accelerator, formalising patterns from client engagements into a reusable enterprise framework. Contributed to multi-million dollar tender, responding to 400+ requirements and presenting to CIO and executive leadership.
Consultant - Data Engineering
Built Microsoft Fabric data platforms implementing medallion Lakehouse architecture (Bronze, Silver, Gold). Developed metadata-driven ELT/ETL pipelines with performance optimisation for large-scale workloads. Reduced ingestion costs by 50% through pipeline optimisation. Achieved 80% cost savings by right-sizing capacities and optimising job scheduling. Designed CI/CD processes including Git branching strategy and automated deployment. Collaborated with Microsoft engineers to resolve platform stability issues and product bugs.
Manager - Data & Analytics
Led development of greenfield data analytics platform. Implemented new stack: Fivetran, GitHub, Azure SQL, dbt, Power BI, and Jupyter. Built executive dashboards for revenue, pricing, and budget tracking. Delivered machine utilisation and capacity analytics across 200+ devices spanning all clinics. Automated government KPI and radiologist CPD reporting. Conducted data requirements workshops with executives and clinical stakeholders.
Senior Manager Business Intelligence
Developed and implemented BI strategies aligned with organisational goals. Led cross-functional teams to design and deliver advanced analytics, data warehousing, and data solutions. Streamlined data collection and analysis processes, reducing processing times and improving data accuracy. Implemented new technologies and methodologies to enhance BI capabilities. Built partnerships with business stakeholders to expand BI reach and foster a data-driven culture across the organisation.
Featured Projects
Microsoft Fabric Accelerator
Production-grade, end-to-end data ecosystem featuring metadata-driven ELT, medallion Lakehouse architecture (Bronze/Silver/Gold), and built-in alerting, auditing, and logging. Built by productising patterns from client engagements into a reusable, enterprise-scale accelerator with cost optimisation and performance tuning capabilities.
Fabric Cost Optimisation Study
Led comprehensive analysis of Microsoft Fabric self-hosted data movement, identifying 10x cost discrepancy compared to Azure Data Factory. Provided actionable feedback to Microsoft PMs influencing product strategy. Achieved 50% cost reduction through pipeline optimisation and 80% savings through capacity right-sizing.
Healthcare Analytics Platform
Built greenfield analytics platform for Benson Radiology with 200+ medical devices. Implemented modern data stack with Fivetran, dbt, Azure SQL, and Power BI. Delivered executive dashboards for revenue tracking, machine utilisation analytics, and automated government KPI reporting.
Blog
Can You Actually Build a Production Fabric Platform with Private Links?
An honest assessment of Microsoft Fabric Private Links for enterprise deployments — what the limitations actually are, why most organisations land on Conditional Access instead, and when Private Links do make sense.
Delta Table Maintenance in Microsoft Fabric: A 2026 Practitioner's Guide
The definitive guide to Delta table maintenance in Microsoft Fabric — what the documentation says, where to find it, and what your notebooks should actually look like in 2026. Covers V-Order defaults, Auto-Compaction, Adaptive Target File Size, Fast Optimize, Liquid Clustering, and Deletion Vectors.
Building My Portfolio with Claude: A Personal Upskilling Journey
My journey from Level 1 (Code Completion) to Level 2 (Human-Directed Local Agents), using my portfolio website as the guinea pig. This post shares what I learned about repository-centric AI development, the benefits of full context, and how I'm keen to see these capabilities translate to professional work.
Certifications
Generative AI for Data Engineers
Generative AI: Elevate your Data Engineering Career
Generative AI: Introduction and Applications
Generative AI: Prompt Engineering Basics
SQL (Advanced) Certificate
SQL (Intermediate) Certificate
SQL (Basic) Certificate
Microsoft Certified: Fabric Analytics Engineer Associate
Foundations of Data Science
Customer Insights and Consumer Analytics for Organizations
Exploring and Preparing your Data with BigQuery
Getting Started with Amazon EMR
Introduction to Amazon Kinesis Analytics
Introduction to Amazon Kinesis Streams
Introduction to Python
Introduction to SQL
Student Member
Introduction to R
Education & Awards
Graduate Certificate in Management
Actuarial Exams
Graduate Certificate in Commerce
Bachelor of Journalism
One Young World Delegate
FSC/Deloitte Future Leaders Award - Finalist
Get In Touch
I'm always interested in hearing about new opportunities, collaborations, or discussing data engineering challenges. Based in Adelaide, South Australia. Feel free to reach out!
Email: bradcoles@outlook.com.au
Phone: +61 423 237 043
Location: Aldgate, South Australia