Really appreciate your time! Thank you very much!

And why the overfitting depends on the number of comparisons, isn't the overfitting something relation to each model separately?

Thank's for the answer! I don't understand the separation you are doing btw training and validation. Didn't we have train/test and we applied cv to the train? The validation sets would be 1 fold at each cv iteration. What I am not understanding here?

Thank you very much for the answer!

One question, what do you mean in this context by estimates? Hyperparameters?

>But if you are comparing 3-4 estimates of the error on the test set to choose the best model class this is not a large comparison, and so the test set is "not so contaminated" by this comparison, and can be used for other purposes.

Could you explain this in another way pls, I'm not sure I am understanding it :(

Thank you for the reply! Step 5 is because I have to submit the predictions for a separated from which I don't know the labels. So my idea was to use all the data.

I don't have a clear opinion, I'm trying to learn and I'm proposing a situation and you're not listening. You are evaluating the performance of the model with the same accuracy you are selecting hyperparameters, this does not make sense.

Anyway, thank you for your help, really appreciate it.

What do you think about it?

Have a look at this: https://towardsdatascience.com/train-test-split-c3eed34f763b

Is not validation data, it is test data. You haven't checked the accuracy on the test data as another fold you average for getting the mean accuracy in the cross-validation. There you can see how the model is generalizing with the hyperparameters selected in the cross-validation.

Mmmm okay. But imagine you have 1000 datapoints and you want to compare a random forest and a DNN and select which one is the best to put it into production, how would you do it?

I don't know if you know what I mean, how can you test if the hyperparameters are overfitting if you have selected them as the ones that maximize the mean accuracy across all the folds.

Not really, because you are taking the average across all the folds, so at some point you are using that validation splits for training some of the folds, not with the test split.

Yeah, I understand that. But what's the point of separating the test set then? You are using cross-validation to select the hyperparameters, you are not seeing how they work in new data...

Okay, got two answers, one that says separate always a test set and the other one that is useless :(

But is this LIME's problem? I mean, it is the model that is not taking into account the correlated feature, not LIME. LIME just looks at the original model.

Damn! The paper is really interesting u/TenaciousDwight! Thanks :) Appreciate it.

Nice! Thank you :)

Thank you very much for the comment! I don't take it badly at all. Constructive criticism is always welcome. Regarding my accent, I really try to give the best of myself but it's innate haha, I'm sorry. I will try to improve it although I don't know how. Also, I recorded with the laptop, I have no micro :( However, I provided detailed subtitles in the video!!

Thank you very much for your opinion u/flinsypop! Appreciate it a lot!
I completely agree with what you say, and I'll take that in mind for the next videos.