How to Evaluate AI Software
Buying standard AI (artificial intelligence) software is a good first step for companies new to the technology. There should be no need to invest in technical infrastructure or resort to expensive data science. There will also be the benefit of getting a solution that has been tested by other customers. For the most part, the precision levels should be reliable, as the algorithms will likely be implemented correctly.
But there’s a lingering problem: there are plenty of AI apps out there, and it’s extremely difficult to determine which is the best option. After all, it seems like most tech vendors are touting their AI capabilities as a way to stand out from the crowd.
So what are the factors to consider when evaluating a new solution? Let’s take a look at the following:
Data connectors: AI is useless without data. It is the fuel for ideas.
But when it comes to a new AI solution, it can be difficult to find the right sources of data, scramble them and integrate them. So, when evaluating an app, you need to make sure that there are ways to manage this process.
“The most complex task in an AI solution is no longer to implement the machine learning algorithm – it is usually available as a set of functions in each tool – but to collect the data. “said Rosaria Silipo, PhD student. and a senior data scientist at KNIME. “That is, connect to various data sources, on-premises, on the web or in the cloud, and extract the data of interest. “
Flexibility: AI has no general scope. Instead, it focuses on particular use cases. This is called a “weak AI”.
That’s why it’s important to see if the app is designed to handle your vertical or your particular situation.
“Take research, for example, where AI can be used to reclassify results and improve relevance,” said Ciro Greco, vice president of artificial intelligence at Coveo. “When applied to e-commerce, the search looks for semi-structured records, such as products with little available text, and we can rely on reasonable amounts of behavioral data produced by users browsing the website. A strategy based on user behavior can be very effective because we can rely on enough data to learn. “
Yet AI research for customer service use cases is often very different. It is often a question of finding technical documents. “There’s a lot of unstructured text, like knowledge articles, and fewer behavioral data points because customer service websites are typically less visited than e-commerce platforms,” Greco said. “So in this case an NLP-based strategy for subject modeling is likely to be more effective, as we need to maximize the gain from the information we have, in this case free text.”
Ease of use: This is absolutely critical. The AI user is often a non-technical person. If the application is complex, there could easily be little adoption.
Ethical AI: Even if the application is correct, there may be risks. The data may have inherent biases, which could skew the results. That is why you should get an explanation of the data and how it is used.
“What many forget when evaluating an AI solution are the potential damage or risks it could pose to your organization,” said Michael Mazur, Founder and CEO of AI Clearing. “What if your organization is sued for deploying this solution? “
Fresh: “If you are the first customer in a specific industry for an AI vendor, then you are a very valuable customer and you can use that as a negotiating lever for a profitable contract,” said Brian Jackson, analyst. and research director at Info-Tech Research Group.
To M (@ttaulli) is an advisor / member of the board of directors of startups and author of Artificial Intelligence Basics: A Non-Technical Introduction, The Robotic Process Automation Handbook: A Guide to Implementing RPA Systems and Implementing AI Systems: Transform Your Business in 6 Steps. Hhas also developed various online courses, such as for the COBOL and Python Programming languages.