There is no training required?

[saisusmitha]

As we are using pre-trained models - there is no training, right? Also, the dataset class, etc is not used anywhere for training, right? Kindly correct me if I am wrong. How to use evalaute.py and generate.py?

[aenglebert]

Hello !

The methods effectively does not requires a specific training, it's designed to explain an already trained model.

The generate.py is made to produce the explainability maps for our method or the other methods compared in the paper, while evaluate.py product the metrics to benchmark the methods.

They can be launched directly, without any arguments, and they will use the default configuration, or can be customized by editing the hydra yaml files or dynamicaly by overwriting them in the commad line.

By default generate.py produces maps for the tis method on a vit model for a subset of ilsvrc2012, while the evaluate compute the insertion metric on these maps.

Do you have a question concerning a specific part of the configuration? Or to adapt to a custom use case?

ps: I just saw that you changed the issue multiple times with different import errors before closing it and reopening with the current question, please don't forget to install the dependencies from the requirements.txt file.

[saisusmitha]

Thanks for the clarification @aenglebert - I am using a basic VIT trained on another dataset - so i load this trained model weights instead of the regular imagnet pretrained vit and use this explainability directly but for chefer1 , chefer2 etc it's not that simple I think - it has vit lrp kind of model to be loaded which is different from basic vit - so how to use those kinds of explainability methods for my vit(which is a basic vit)? Should i train Vit_lrp on the dataset i want then only i will be able to use those methods? if not kindly let me know how to use the basic vit model (what i have) for using chefer explainability methods?

[aenglebert]

In the case of our TIS method, we tried to made it easily usable on a vision transformer using a timm or a Torchvision implementation. There is a section "Import in any project" in the readme showing how to use it.

For the Chefer methods and similar, they use a modified model based on the timm implementation (they don't change the layers per say, the modifications are related to things like retriving intermediate values, so the output of the model is the same as the classical one if the weights are identical).

To use the same weights for all models in our benchmarks, we used the timm implementation and made a wrapper to uses the modified model for compared methods with the same weights as the normal timm vit (for exemple, see chefer2_wrapper.py ).
Maybe can you adapt our wrappers to fit your case ?

Please note that this repository is related to our TIS method and the other methods are imported for comparisons, but we don't make development for the other methods.
If you need to go more in depth into adapting other specific methods to your model, you may get more chance by asking directly in the repository of the individual methods you want to adapt ?

[saisusmitha]

@aenglebert why is that on every run the explanations change i.e how the map looks changes? what is the aspect with is randomly taken in each run?

[aenglebert]

Everything that is random in the network by itself, for exemple if dropout is enabled it will influence the result, if you use data augmentation,...
Did you put the network in eval mode?