Three questions to ask when evaluating a commerce protection solution for transparency

Commerce protection uses cross-merchant data networks, AI-powered machine learning, and ecommerce order automation so that you can focus on optimizing revenue rather than eliminating risk. Putting it a different way, this new breed of automated anti-fraud solutions minimizes the false positives that can cause you to turn down good customers’ money while pinpointing with ever-greater precision where the real fraud is coming from. It protects both you and your customers across the entire shoppers’ journey. And it’s actually enabling you to sell more, to more (and happier) customers.

How is this possible? In an abbreviation, AI. One thing all leading solutions claim is that their “secret sauce” for detecting fraud is based on their own particular artificial intelligence (AI) algorithms, or mathematical models, that can sniff out fraud far better than any human could.

This may be true, but what kind of AI algorithms? How do they work? What makes them so great? Peek under the hood (if the vendor will let you) and more than likely you’ll be staring into a black box — black-box AI. And that’s not a good thing.

Here is where transparency becomes really important.

Beware of the black box

Black box AI is any artificial intelligence system whose operations are not visible. It’s an impenetrable system. You don’t—and can’t—know how it works. Some variables go in one end, and some results are spit out the other, and you have no idea what happened in the middle.

Black box AI is in broad use today. When you search the web, when you use a credit card to buy a shirt, a phone or dinner delivered to your home, black box AI is likely to be involved.

Why design AI systems this way? Recent advances in AI technologies like machine learning, deep learning and computer vision have resulted in such complex systems that there is a widespread belief they are inherently uninterpretable. This means that humans—even the ones who have designed the systems—cannot understand how inputs relate to each other or are manipulated to reach a final decision or prediction.

Why this matters in fraud prevention/commerce protection

Why should you care? As long as a vendor keeps your chargebacks low, is it necessary to know how the sausage is made?

In the long run, yes. If you don’t know why certain automated decisions are being made — or even what data those decisions are based on—it’s impossible to evaluate the job your solution provider is doing and whether it’s providing value. You don’t know whether you should trust the models, and you can’t hold the vendor accountable should something go sideways.

Take approval rate guarantees. Certain vendors are promising to consistently deliver an agreed-upon approval rate. However, if you do not have insight into how a provider is delivering those approval rates—are they simply taking the hit of greater chargebacks to win your business?—then you have no assurance that they are making good decisions.

Machine learning that won’t leave you in the dark

Capabilities like Signifyd’s Agent Console and Insights Reporting give you both the micro and macro data you need to understand decisions and stay attuned to the true business performance of your vendor. In particular:

• Decision explainability and exploration—Providers should offer you an interface that allows you to search and review orders they process, giving you visibility into the data—and network insights—that determine the decision on a given order.
• Performance transparency and benchmarking—Commerce protection vendors should also provide a hub to monitor and track business performance across segments. And this should include the ability to customize reporting to your specific business needs.

Three things to ask your provider before committing

Here are some things to probe:

1. What are the number and depth of data points provided for each transaction decision?

Do you get visibility into the variables used to calculate decisions? Do you have indicators telling you which variables weighed most heavily—both positively and negatively? Providers should offer merchants an interface to search and review orders processed by the commerce protection provider and give insights into:

• Order decision
• Numerical score
• Payment / Shipping / Product details
• Billing and delivery locations
• People information
• Account information
• Phone information
• Network information

2. How many and what type of performance metrics are available in the reporting console?

Do you get a centralized hub in which all the numbers needed to analyze business performance are easily accessible?

• Business performance KPIs—including sales across segments like geographies, product lines and payment methods; total chargebacks; average chargeback amounts; and common chargeback codes
• Fraud metrics—such as fraud trends by segments, approval rate, chargeback rate and fulfillment time
• Chargeback recovery metrics—such as chargeback claim status, win rate of claim representment, leading chargeback reason codes and win rate by chargeback type.

3. How much visibility do your business stakeholders get?

Does your solution provide the ability to custom build, export and share metrics across your organization? Can your customer success team review solution effectiveness, including industry/sector comparisons to other retailers regarding ecommerce chargeback protection, trends, false positives and challenge rates?

Signifyd offers true visibility to put you in the driver’s seat

Signifyd believes that the essence of commerce protection is that the technology operates sight unseen, leveraging real-time network insights from thousands of merchants to process orders automatically. That it does this without disrupting your workflow or the shopper’s purchase journey is one of its chief benefits.

But it’s also essential to maintain accountability. That’s why Signifyd solutions offer merchant-facing modules that provide visibility into the platform network and decision engine as well as sufficient control levers to give you the final say over how you want to protect—or grow—your business.

Photo by Getty Images

Let’s talk transparency in commerce protection.