Most finance organizations are still working to understand the return on investment (ROI) that artificial intelligence (AI) provides. While its potential is high, according to research by Boston Consulting Group, only 26% of companies have advanced beyond the proof-of-concept stage to generate economic value from their AI initiatives.

For the majority, unclear ROI remains a key barrier to AI adoption, along with related challenges in expertise and strategy. AFP brought together finance and AI experts to share their insights on how companies can obtain that elusive value from their AI investments. Below are highlights from the AFP webinar.

What we mean by AI

The biggest thing people get wrong when talking about AI is confusing GenAI with classical AI machine learning. “We’ve been using machine algorithms for 15 years. They are deterministic and have been used to great success,” said Glenn Hopper, Head of AI Research and Development, Eventus Advisory Group. “But they were pretty geeky, and only people in the data world really knew what we were talking about.”

Up until two years ago, when GenAI gained prominence, investing in AI usually meant machine learning and data science platforms. “Now, it means working in GenAI, for which the toolset and effort are different,” said Hyoun Park, CEO of Amalgam Insights.

Ways to measure the ROI of AI

One of the most common ways of measuring the ROI of AI is through efficiency gains. Automation and faster task processing hold the potential to free up time and resources. But AI can also lead to new activities and insights beyond what we are doing today.

“If we just treat AI as a massive productivity enhancer, then we’re missing the point,” said Hopper.

“You want more accurate forecasts; you want to have more granular budgets; you want to be able to understand your business drivers; you want to be able to make suggestions about where to go next as a business,” added Park.

To look beyond efficiency metrics, Hopper suggests focusing on metrics that tie directly to business outcomes, such as:

• Revenue Impact: Look for specific ways in which AI initiatives have boosted revenue and track and quantify them. For example, has AI enhanced customer targeting or improved sales conversion rates?
• Customer Satisfaction: Use metrics like feedback scores or satisfaction ratings to highlight AI-driven improvements, e.g., more personalized experiences, quicker response times or better overall service.

“It's all part of a bigger mission of getting finance and accounting away from the idea of just being a cost center,” he said. “We're going to add value, not by reducing the number of people we have in the department, but by increasing the quality and value of the work they're doing.”

The importance of alignment and buy-in

It’s critically important to make sure that the goals of your AI project match the company’s goals. At a strategic level, “tie AI initiatives to business goals and prioritize the impactful use cases,” said Hopper, and build C-suite support to ensure focus and cross-team alignment.

Then, at a process level, take a step back and figure out where AI fits into the workflows. Data integration, application integration solutions that may already have workflows in place, and closed automation solutions are areas that lend themselves well to the use of AI.

“Can AI process these thousands and tens of thousands and millions of manual checks?” said Park. “Where are you trying to find needles in a haystack? That’s where AI can provide some real value.”

Fear of the unknown plays a part in people’s resistance to AI. However, Hopper added, “If we can figure out how and when to use GenAI and get comfortable with how it's generating results, then that fear dissipates.”

In addition to providing support, leadership should develop a usage policy on GenAI to prevent sensitive data from being made public.

AI implementation and scalability approaches

Before you can implement and scale your AI project, you have to get your data in order. Once that is done, Hopper advises starting with some enhanced BI; then, you can consider how AI can be applied within your organizational structures.

“The AI most of us are using is someone else's model — e.g., Microsoft Copilot, ChatGPT,” said Hopper. “We're not making a massive capital investment.”

There are four different tiers of AI implementation:

1. Top (rare): Companies developing their own foundation models
2. Mid: Companies fine-tuning existing open-source models
3. Lower: Companies using RAG (Retrieval Augmented Generation) to enhance existing models with company data
4. Basic: Companies using off-the-shelf AI tools across their workforce

Most companies aren't building their own LLMs. “Except for the biggest companies in the world that have an amount of data where it makes sense, most of us are not training these massive models,” said Hopper. “It's more about managing software cost. Scalability is not an issue we’re facing yet.”

When scaling AI, it's crucial to consider your classic computing or IT aspects in terms of storage and network to avoid overprovisioning. “You don't want too many duplicate resources all doing the same thing,” said Park. “You don't want to use a model that’s overkill for the type of use case you’re employing.” Smaller, custom-built agents are often more cost-effective than using large models (200-300 billion parameters) for simple tasks.

Also, be sure to consider storage costs for AI outputs, as necessitated by governance and compliance rules. “You need to take care of the storage because you're probably going to have to maintain the outputs,” said Park. “It's important to do that due diligence and just make sure that your AI approach does not lead to massive overruns on your intended budget.”

AI’s environmental impact

The environmental impact of running one question on GenAI is “an order of magnitude higher than it is to run it through a search engine,” said Hopper. As such, companies will need to consider the impact AI has on their carbon footprint.

For those that do ESG reporting, AI may be a significant contributor to Scope 2 reporting, explained Park. “There's still a lot of primary research being done at universities to figure out whether models can be made more efficient, whether we can use smaller models, etc.,” he said.

The environmental impact also comes with a financial cost. Hopper explained that GenAI is very expensive to deliver. “So expensive that we're having to rethink the energy delivery system around the world. It is something we need to be concerned about,” he said.

But Park noted that GenAI is still a relatively new technology, so “the cost and challenges we have right now may be very different two or three years down the road,” he said.

Real-world AI ROI business cases

For those looking for publicly available examples of AI implementations that demonstrated significant returns, Hopper compiled the following list:

• Walmart experienced 4.8% revenue growth and 21% growth in e-commerce, which they partially attributed to GenAI. They used AI to optimize inventory management and improve their product catalog, enhancing the customer shopping experience.
• Visa leveraged GenAI to more effectively identify and predict fraudulent activity, improving their payment security systems.
• HDFC ERGO, an Indian insurance company, utilized Vertex AI to create highly personalized offerings for consumers in specific geographical locations.
• Hiscox, an insurance company, developed an AI-enhanced lead underwriting model that reduced complex risk quoting time from three days to a few minutes.
• Loadsure implemented Google Cloud's Document AI and Gemini AI to automate insurance claims processing, resulting in faster processing times and improved customer satisfaction.
• Best Buy reported a 30-to-90-second reduction in average call time and after-call work by using Contact Center AI to generate real-time conversation summaries.
• Miinto, an e-commerce platform, saw a 40% increase in efficiency and a 20% improvement in conversion rates by using Vertex AI Vision to identify and merge duplicate product listings.
• An unnamed healthcare organization implementing an AI platform in radiology demonstrated a 451% ROI over five years, which increased to 791% when radiologist time savings were included.