Button Text
Home
arrow
Blog
arrow
IDP
Aug 29, 2022
Min Read

4 Questions to Ask Your IDP Vendor

Share on TwitterShare on Twitter
Share on TwitterShare on Twitter
Share on TwitterShare on Twitter
Share on TwitterShare on Twitter
Manish Rai
VP of Marketing
SUMMARY

The global software market for Robotic Process Automation (RPA) is projected to reach $2.9B in 2022 (+19.5% YoY) according to Gartner. Although growth forecasts for RPA continue to predict double digit growth through 2023, the category is decelerating compared to previous years. For example, just four years ago Gartner estimated the RPA software market grew 63.1% YoY—which made it the fastest growing enterprise software market in 2018. The RPA category’s initial growth was driven by a compelling value proposition:

Enable business users to rapidly drive down costs by automating mundane, repetitive tasks.

This was a win-win for employers and employees. Companies could get the benefits of automation without hiring expensive, specialized (not to mention in short supply) talent and it eliminated the most mundane parts of many peoples’ jobs. Fast forward to today, and recent surveys suggest the COVID-19 pandemic stalled automation progress generally, as companies shifted focus to enabling remote work and adapting their business models. With the pandemic behind us, automation is back with a bang, driven by an ongoing labor shortage, macroeconomic headwinds, and pent-up demand to move stalled automation efforts forward.

Source: HFS Research

However, 90% of enterprise data is unstructured, and many processes involving structured data have already been automated. It’s because of unstructured data that renewed interest in automation is focused on Intelligent Automation (IA), which combines RPA with artificial intelligence (AI). These trends have caused Intelligent Document Processing (IDP) to become one of the fastest growing software categories, with Gartner estimating the IDP market will reach $4.8B in 2022, up from $1.2B (+300%) two years prior.

Venture capitalists (VC) and entrepreneurs have responded, with 75+ companies competing to help businesses automate processing of endless varieties of structured, semi-structured, and unstructured documents. What is the best way to evaluate IDP vendors in such a crowded environment?

Start with 4 questions to ask any IDP vendor

It is helpful to know that today most IDP vendors offer closed points solutions that excel at processing structured and semi-structured documents only, do not guarantee outcomes, and leave the heavy lifting that comes with setup, training, and ongoing post-processing to buyers. This has led to project delays and lower-than-expected savings for many IDP projects. To avoid these pitfalls, enterprises should ask IDP vendors the following four questions:

  1. Can you process 100% of my documents? Not 80%.  Not 90%.  Not even 95%.
  2. Do you guarantee data processing quality, cost, and speed?
  3. Can you process complex documents (unstructured, handwritten, etc.)?
  4. Can you process other data types (image, video, and audio)?

Now let's dive down into why each one of these questions is important when evaluating IDP vendors.

Q1: Can you process 100% of my documents?

For more than 20 years, starting with optical character recognition (OCR), businesses have been trying to automate document processing. Automation for documents with a fixed format, such as W-2, and W-4 forms, is relatively easy. However, variable format documents like invoices and purchase orders are far more difficult to automate.

Even the best IDP solutions struggle to automate more than 60-80% of complex documents, leaving businesses to worry about processing the remaining 20-40%. For enterprises receiving millions of invoices each year, these numbers can add up. Companies then have to worry about hiring, retraining, and managing a document processing workforce with high turnover or augment their IDP solutions with a Business Process Outsourcing (BPO) provider.

Look for vendors that offer a complete solution where AI, bots, and humans (often crowd-sourced) work hand in hand to process 100% of the documents. By using humans to train and improve AI models, costs are continually driven down as automation rates reach 100% over time. Ask vendors about their processes for onboarding and managing crowd-sourced resources.  They must include automated assessment during onboarding, quality control mechanisms that continuously assess human workers, and gamification to keep people motivated and engaged.

Q2: Do you guarantee data processing quality, cost, and speed?

Most IDP vendors provide confidence levels at the document and/or field level. This leaves businesses to set initial confidence levels based on trial and error, which in practice means continuously monitoring and tuning models as data evolves. Newer vendors allow businesses to make outcome trade-offs between quality, cost, and speed when building a project, and automatically select the AI confidence levels for various models involved to achieve the desired outcomes.

Q3: Can you process complex documents?

Most vendors promise the ability to process complex documents that may have some or all of the following characteristics:

  • Unstructured (e.g., contracts)
  • Information dense (e.g., bills of lading (BoL)
  • Include handwritten notes (many documents include them)
  • Stamps (approval/rejection)
  • Signatures

However, many of these same vendors build closed IDP software that utilizes a fixed set of OCRs and proprietary ML models. Unfortunately, this approach makes it difficult to adapt these solutions to new use cases. Look for vendors that have flexible AI platforms that can break a document extraction task into small steps, then leverage the best OCR, ML model, or software for each step, and finally combine the results into a unified output. This approach allows more modern platforms to quickly adapt to new and more complex document types after automating low-hanging fruit like invoices and purchase orders.

Q4: Can you process any data type?

Automation journeys become increasingly difficult over time. After the easy processes that rely on structured and semi-structured data and documents, companies must automate increasingly complex tasks to make further progress. This might include tasks involving not just complex documents, but also images, videos, and/or audio files.

It is important to select a platform architected for this natural evolution. Look for products with  an architecture that simplifies the process of training models on new data types. Closed IDP solutions with proprietary models simply aren’t setup for the variability that comes with automation. Selecting the right vendor now will prevent significantly higher costs that come with switching vendors, or the headache of stitching together multiple points solutions—each with its own setup, training, post-processing, and ongoing maintenance costs.

Learn more about Super.Redact, our AI-automated solution for redacting sensitive information at scale.

This is what super.AI was built for

Super.AI was built from day one to address these shortcomings.

A lesson learned when Kasparov lost to IBM’s Deep Blue is that the best chess players are not humans or AI, but humans and AI working together. This insight inspired us to launch our Data Processing Crowd, which works hand-in-hand with our unified AI platform to continuously improve quality and automation rates. Here’s how it works in a nutshell:

  • A sophisticated onboarding process verifies worker skills
  • 150+ quality assessment measures ensure output quality
  • Escalation rules automatically flag unclear or suspicious output
  • Gamification keeps human workers engaged
  • Strict security and privacy mechanisms keep your data safe

It’s this approach that allows us to process 100% of user data. Additionally, our platform was designed to hide the complexity of AI from business users. Simply define the priorities between quality, cost, and speed, and let the platform do the rest—including training, tuning, and testing to ensure we achieve desired outcomes with SLA guarantees.

Since our platform can break processing into smaller tasks and use the best available AI, software, or human for each one, we can process even the most complex documents that include handwritten notes, stamps, signatures, unstructured content, and more. Customers already leverage our platform to process a wide variety of data types, including documents, emails, images, videos, and audio. This functionality allows users to standardize on a single unified platform for all their unstructured data processing needs.

Other Tags:
IDP
Document Automation
Share on TwitterShare on Twitter
Share on FacebookShare on Facebook
Share on GithubShare on Github
Share on LinkedinShare on Linkedin

You might also like

close