The preliminary work presented here evaluates the robustness and bias of ML models by probing HPO behavior under different demographic conditions, which is critical for the development of clinically-usable methods. By examining the different hyperparameter distributions for a transformer based model along a male-female data split, we gain insight into the behavior and transferability of hyperparameters along imbalanced datasets in this area.