Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Add McFadden's Pseudo R² Parity metric to ClassificationMetric#563

Open
Zheng-ZhongHeng wants to merge 12 commits into
Trusted-AI:mainTrusted-AI/AIF360:mainfrom
Zheng-ZhongHeng:feature/pseudo-r2-parityZheng-ZhongHeng/AIF360:feature/pseudo-r2-parityCopy head branch name to clipboard
Open

Add McFadden's Pseudo R² Parity metric to ClassificationMetric#563
Zheng-ZhongHeng wants to merge 12 commits into
Trusted-AI:mainTrusted-AI/AIF360:mainfrom
Zheng-ZhongHeng:feature/pseudo-r2-parityZheng-ZhongHeng/AIF360:feature/pseudo-r2-parityCopy head branch name to clipboard

Conversation

@Zheng-ZhongHeng

Copy link
Copy Markdown

Summary

This PR adds McFadden's Pseudo R² Parity as a new fairness metric
to the ClassificationMetric class, designed for binary classification
scenarios where scores are predicted probabilities (0~1) and labels
are binary ground-truth values (0 or 1).

Motivation

Existing fairness metrics in AIF360 do not measure whether a model's
explanatory power is equitable across groups. McFadden's Pseudo R²
provides a log-likelihood-based measure of model fit, and its parity
variant detects disparities between privileged and unprivileged groups.

New Methods

pseudo_r2(self, privileged=None)

$$R^2_{McFadden} = 1 - \frac{\ln L_{model}}{\ln L_{null}}$$

  • privileged=True: privileged group
  • privileged=False: unprivileged group
  • privileged=None: entire dataset

pseudo_r2_parity(self)

$$\Delta R^2 = R^2_{\text{unprivileged}} - R^2_{\text{privileged}}$$

  • 0 = parity
  • Positive = better fit for unprivileged group
  • Negative = better fit for privileged group

Testing

Unit tests added to tests/test_classification_metric.py:

  • test_pseudo_r2_overall
  • test_pseudo_r2_privileged
  • test_pseudo_r2_unprivileged
  • test_pseudo_r2_parity

Copilot AI and others added 12 commits April 2, 2026 08:42
…seudo-r2

Replace OLS R² with McFadden's Pseudo R² in `pseudo_r2()`
…lassificationMetric

Agent-Logs-Url: https://github.com/Zheng-ZhongHeng/AIF360/sessions/45a103f5-09f8-443e-8507-2fc4cf5b2f1e

Co-authored-by: Zheng-ZhongHeng <148734589+Zheng-ZhongHeng@users.noreply.github.com>
…hods

Move `pseudo_r2` / `pseudo_r2_parity` from `RegressionDatasetMetric` to `ClassificationMetric`
…lassificationMetric

Agent-Logs-Url: https://github.com/Zheng-ZhongHeng/AIF360/sessions/45a103f5-09f8-443e-8507-2fc4cf5b2f1e

Co-authored-by: Zheng-ZhongHeng <148734589+Zheng-ZhongHeng@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.