It's time to treat (almost) every procurement as a procurement of AI

Key takeaways

Given the spread of AI—and the recent ban by the Commonwealth and the various state and territory governments on the use of DeepSeek on government devices—agencies urgently need to take the following actions:

• Acknowledge that virtually every procurement will involve AI, even if it is not explicitly requested.
• Update procurement policies, documentation, and contracts to:
  • apply each ban (noting that additional bans may be introduced in the future), and
  • manage the risks associated with AI.

Click here to access a short guide to the procurement changes that should be implemented to comply with the Voluntary AI Safety Standard (Voluntary Standard).

Introduction

Artificial Intelligence (AI), once the preserve of tech professionals, became a global phenomenon as of June 2020 with the release by Open AI’s AI model, ChatGPT-3. Since then, many AI models have emerged from companies such as Microsoft, Google, IBM, Amazon, Nvidia, and Anthropic, introducing terms like ‘LLMs (large language models)’, ‘Generative AI’, ‘AI Agent’ and ‘hallucination’ into our everyday language.   

While the advancements in AI offer ground breaking opportunities, they also come with significant risks. Governments worldwide are increasingly aware of these risks and are addressing them through various legislative measures.  

The Australian position

The Australian, state and territory governments, in line with their international counterparts, recognise the opportunities presented by AI but also acknowledge that the use of AI is not without risks.

Identified risks include bias in decision-making, privacy breaches, infringement of third-party IP rights, harm to individuals (both physical and psychological), reputational damage and regulatory non-compliance. AI can also amplify existing risks, such as cyber-attacks. Most recently, the Australian Government banned[1] DeepSeek products, applications, and web services due to an ‘unacceptable level of security risk’, with similar bans being implemented by states and territories[2].

Given these risks, it is unsurprising that public confidence and trust in AI usage is low[3]. The DeepSeek ban is unlikely to improve the situation. In response, the governments have committed to being ‘exemplars’ in the safe and responsible use of AI and have published various papers to guide AI users.  Notable among these are the National framework for the assurance of artificial intelligence in government[4] and the Voluntary Standard. The Voluntary Standard outlines 10 voluntary ‘guardrails’ for AI use, including specific guidelines on how to apply these guardrails to procurement processes.

What does this mean for procurement and contract managers?

Procurement and contract managers, along with their teams should:

• be aware of AI and comply with any relevant AI bans, and
• familiarise themselves with relevant government publications, including the two mentioned above, as well as any other publications specific to their state or territory where procurement occurs.

Given the rapid development of new and improved AI tools, we recommend professionals closely monitor advancements in this field by regularly checking websites of relevant agencies, such as the Department of Industry, Science and Resources, the Digital Transformation Agency and the National Artificial Intelligence Centre (NAIC), to name but a few.     

To ensure that AI is used safely and responsibly, procurement and contract managers and their teams need to adopt a proactive mindset to their tasks in the evolving AI landscape and rethink their approaches to procurement and contract management.

The management of AI must take priority. Simply put, procurement and contract managers along with their teams should:

Assume that virtually every procurement will involve AI as part of the solution, even if AI is not explicitly requested

Consequently, unless a procurement is solely for the supply of goods that could not contain AI, to comply with Australian government policy and manage AI-related risks, each procurement should:

• Ensure that any bans on AI are applied. As noted, the current ban on DeepSeek is unlikely to be the last.
• Apply the Voluntary Standard to almost every procurement process, to varying degrees based on the associated AI risks.

Procurement officers need to be particularly mindful that Australian, state and territory government procurement templates may not adequately address these risks. Therefore, it is essential to include appropriate clauses applying any AI bans and AI risk management in procurement documentation, including the contract itself. Relying solely on a template may not protect you or your agency if a banned form of AI is tendered or if an AI-related risk leads to adverse consequences.

Assume that every contactor is, or will soon be, using AI

Contract managers should determine whether their current contractors (including anyone within the supply chain[5]) are using a banned form of AI or intends to use AI in the future, and how this might impact existing or proposed contracts. In short, contract managers should:

• If a contractor is using AI, amend the contract to ensure compliance with each applicable ban and with the relevant contract requirements in the Voluntary Standard.
• If a contractor is not using AI, consider amending the contract to prohibit AI usage unless explicitly approved, which may be subject to certain conditions.

The ability to amend a contract as mentioned above will depend on the contract’s existing provisions. If the contract is silent on this matter, negotiating the required amendments will be necessary.

What’s next?

One of the key challenges facing procurement and contract managers and their teams in 2025 and beyond will be determining the best way to leverage AI in the procurement process, while managing associated risks. AI is no longer an ‘optional extra’ but a ‘must have’. It will not only allow procurement processes ‘to be done differently’ but ‘to do different things’.[6]

Procurement and contract managers along with their teams must remain informed about developments in the regulatory and governance environment, particularly regarding AI bans. The NAIC has indicated that in 2025, it will be looking to extend the Voluntary Standard to include additional practices and guidance for AI system developers, following online consultations in January and February 2025. One of the topics for discussion will be procurement guidance, as the NAIC aims to provide more details supported by a standalone procurement guide.

Further, it is highly likely there will be a legislative response to address the risks of AI, as foreshadowed by the Safe and responsible AI in Australia proposal paper for introducing mandatory guardrails for AI in high-risk settings.[7] Aligning procurement processes with the Voluntary Standard now will help agencies start to develop the practices required for a future regulatory framework and any upcoming AI bans.

[1] See Protective Security Policy Framework Direction 001-2025, by the Australian Department of Home Affairs, DeepSeek Products, Applications and Web Services

[2] Queensland, Western Australia, the ACT and the Northern Territory have imposed comparable bans on the use of DeepSeek on government devices

[3]  The Conversation, “80% of Australians think AI risk is a global priority. The government needs to step up”, published on 8 March 2024

[4] Released on 21 June 2024 at a meeting of Data and Digital Ministers from the Australian, state and territory governments

[5] The AI Voluntary Safety Standard applies throughout the AI supply chain.

[6] Procurement magazine, “How CPOs are Approaching AI in Procurement and Sourcing”, published on 23 August 2024

[7] Department of Industry, Science and Resources, published 5 September 2024