Machine learning (ML) models are widely used across various critical domains for their ability to process large-scale data and deliver high predictive accuracy. While traditional benchmarking of ML models focuses on performance metrics like accuracy and precision, these metrics often fall short in accounting for the model’s transparency and robustness when the models get complex with varying conditions. This research proposes a comprehensive framework for benchmarking supervised machine learning models that incorporate the model’s performance metrics, explainability techniques, and robustness assessments. The proposed framework combines traditional accuracy- and precision-based performance evaluation with explainability and robustness to ensure the model's efficiency, transparency, and stability in the presence of noise and data shifts. By holistically addressing performance, explainability, and robustness in tandem, the framework supports data-driven decision-making in selecting the model appropriate to organizational contexts, addressing stakeholder concerns related to model interpretability, trustworthiness, and resilience—particularly in critical domains such as healthcare.