AI in City Procurement: 5 Pitfalls to Avoid

August 31, 2023 - (4 min read)

Technology

As city administrations consider incorporating emerging technologies like Artificial Intelligence (AI) into their systems, its usage in their procurement procedures should be carefully considered. AI is a powerful new tool that can be beneficial, but also comes with ethical and security issues that public agencies will need to face. Here are five potential pitfalls to avoid when considering using AI for procurement:

1. Dependence on Technology: The use of AI in procurement can significantly increase efficiency and streamline processes. For example, it can be used to write specifications for simple solicitations or evaluate bids, reducing the time and resources required. However, the information that AI produces is not always accurate, so city officials depending too heavily on this technology can lead to potential errors in the procurement process. No matter how accurate an AI-written specification is, it still requires knowledgeable procurement staff to assess it for city-specific requirements before it is published.

2. Security Concerns: AI can help improve security in procurement by leveraging machine learning algorithms and real-time data analysis to identify potential vulnerabilities in information technology (IT) purchases. Detecting potential issues in IT purchases can allow cities to proactively address any cybersecurity concerns that might have otherwise gone unrecognized. However, AI-powered tools are not without vulnerabilities of their own, and experienced cybercriminals can exploit them, potentially leading to security threats. City IT departments must work with procurement staff to implement robust security measures to protect sensitive information and ensure the integrity of city systems.

3. Bias and Discrimination: Unfortunately, because AI-powered tools are programmed by imperfect human beings with conscious and unconscious biases, they can perpetuate unethical discrimination in procurement. Certain algorithms may be programmed with biases that reflect societal, geographical, or economic prejudices, leading to unfair treatment of certain suppliers or products. For example, if an AI-generated solicitation is open to suppliers from any geographic area, but the procurement officer does not review the specifications to include that information, potential suppliers from beyond the city limits may incorrectly believe that their bids will not be considered. City procurement officers must carefully examine anything generated by AI to ensure that all procurements uphold their city’s ethical standards.

4. Cost Overruns: The use of AI can lead to significant cost savings by helping procurement staff identify more cost-effective solutions during the market research process. However, it can also result in cost overruns if not properly managed. AI-powered tools may recommend solutions that may initially seem less expensive but may not actually be cost-effective in the long run. AI-powered tools don’t consider that certain solutions have maintenance or upgrade expenses that will be required as technology advances. Take the iPhone for example. The first iterations of iPhones would not be capable of running the latest iOS, so continued use of those products would require a hardware upgrade. City procurement officers must carefully evaluate AI recommendations to ensure that they align with their budget and procurement goals.

5. Lack of Understanding: AI as an everyday tool is still relatively new and constantly evolving. There are many things that even early adopters don’t understand. Given that new technology tends to be adopted a bit slower in the public procurement realm, city procurement staff can lack the understanding to use AI effectively. AI algorithms may be difficult to interpret by even the most tech-savvy users, and procurement staff simply may not have the time in their day-to-day to get an in-depth understanding of how they work. City administrations should devise and implement thorough training programs before implementing any AI-powered tools or AI-generated documents into their policies, procedures, and publications.

While AI has the potential to revolutionize procurement for cities and other public entities, it is important to approach its use with a healthy dose of prudence to go along with interest in its benefits.