Does Australia have the national compute capability for widespread local AI use?

There's been a lot of attention on the potential benefits and risks of artificial intelligence for Australia - with the Department of Industry developing the Artificial Intelligence (AI) Ethics Framework and the DTA working on formal AI guidelines.

However comparative less attention is often placed on building our domestic AI compute capability - the local hardware required to operate AIs at scale.

The OECD also sees this as a significant issue - and has released the report 'A blueprint for building national compute capacity for artificial intelligence' specifically to help countries that lack sufficient national compute capability.

As an AI startup in Australia, we're been heavily reliant on leveraging commercial AI capabilities out of the US and Europe. This is because there's no local providers of generative AI at commercial prices. We did explore building our own local capability and found that the cost for physical hardware and infrastructure was approximately 10x the cost of the same configurations overseas.

There's been global shortages of the hardware required for large AI models for a number of years. Unfortunately, Australia tends to be at the far end of these supply chains, with even large commercial cloud vendors unable to provision the necessary equipment locally.

As such, while we've trained several large AI models ourselves, we've not been able to locate the hardware or make a commercial case for paying the costs of hosting them in Australia.

For some AI uses an offshore AI is perfectly acceptable, whether provided through a finetuned commercial service or custom-trained using an open-source model. However, there's also many other use cases, particularly in government with jurisdictional security requirements, where a locally trained and hosted AI is mandated.

A smaller AI model, such as Stable Diffusion, can run on a laptop, however larger AI models require significant dedicated resources, even in a cloud environment. Presently few organisations have the capability to pay the costs for sourcing the hardware and capability to run this within Australian jurisdictions.

And that's without considering the challenge in locating sufficient trained human resources to design, implement and manage such a service.

This is an engineering and production challenge, and will likely be resolved over time. However with the high speed of AI evolution, it is a significant structural disadvantage for Australian organisations that require locally hosted AI solutions.

If Australia's key services have to rely on AI technologies that are several generations behind world-standards, this will materially impact our global competitiveness and capability to respond.

As such, alongside worrying about the ethics and approaches for using AI, Australian governments should also reflect on what is needed to ensure that Australia has an evolving 'right size' national compute capability to support our AI requirements into the future.

Because this need is only going to grow.