On Quantifying the Faithfulness of Explainations

Before I dive into the paper, I’ll start by providing some background on the domain.

Explainability algorithms (aka XAI Methods or Post-hoc XAI Methods) are a class of techniques that allow us to understand (“explain”) why a model has a particular output. A few common examples of such algorithms are “LIME” (Local Interpretable Model-agonostic Explainations), “SHAP” (SHapley Additive exPlanations), and “Grad-CAM” (Gradient-weighted Class Activation Mapping). Figure 1 provides the examples for outputs for each of them.

But these XAI Methods are generally defined only for unimodal models. Most of these algorithms do not directly scale up to Multimodal Models trivially and generally fail to account for the complex multimodal interactions.

Due to this reason, a new class of Multimodal XAI Methods (MXAI Methods) was created. Some prominent examples include “DIME”, “MultiSHAP”, “MultiViz”, “PixelSHAP and TokenSHAP”, “InterSHAP”, and “EMAP”.

A lot of these algorithms claimed to be state of the art of better than the pre-existing algorithms. While one had multiple algorithms to deploy while retriving explainations, there was no common benchmark or metric on which these algorithms could be compared.

Apart from this, we strongly believed in the need of a benchmark that could be utilized by the community to further improve and develop the MXAI Methods. To this end, we tackled the open problem of creating the benchmark, which I will be discussing in the blog.