Skip to main content

performativeGYM on Credit Lending

This repository uses the performativeGYM library to train a model on the GiveMeSomeCredit dataset (the link gives a basic overview of the dataset), where the dataset has a performative effect, as described in Perdomo et al. (2020).

Any method implemented in the performativeGYM can be used to train the model. Use the optimizer=... parameter to select the method (see “Make a training run” below). The default is “RGD”.

The model predicts the binary feature SeriousDlqin2yrs from the GiveMeSomeCredit dataset. The model expects the input to be an array of 11 float32 numbers, corresponding to the features of GiveMeSomeCredit:

  1. RevolvingUtilizationOfUnsecuredLines
  2. age
  3. NumberOfTime30-59DaysPastDueNotWorse
  4. DebtRatio
  5. MonthlyIncome
  6. NumberOfOpenCreditLinesAndLoans
  7. NumberOfTimes90DaysLate
  8. NumberRealEstateLoansOrLines
  9. NumberOfTime60-89DaysPastDueNotWorse
  10. NumberOfDependents
  11. bias term which is always 1

The model expects this input to be shifted and scaled by scikit-learn’s preprocessing.scale().

This is an input example:

[
  -0.019939696392918062,
  -1.2677804884731523,
  -0.10910125023074325,
  -0.06213695057266494,
  -0.22038972657563877,
  -0.7265822667937595,
  0.2274215418328712,
  -0.9175569508977497,
  0.23555615821404854,
  -0.7417645465721154,
  1
]

Init project​

Install requirements.

pip install -r requirements.txt

Run a development environment​

Install requirements.

pip install -r requirements-dev.txt

If useful, install local version for tango-interfaces.

pip uninstall tango-interfaces
pip install -e <path>/tango-interfaces/

Run tracking server and model registry​

Run local tracking server and model registry.

mlflow ui

Prepare the environment for training/serving​

Export the following environment variables.

export MLFLOW_TRACKING_URI="http://127.0.0.1:5000"
export MLFLOW_EXPERIMENT_NAME="test"

Use these if the server requires authentication.

export MLFLOW_TRACKING_USERNAME=<username>
export MLFLOW_TRACKING_PASSWORD=<password>

Make a training run​

To make a run,

  • ensuring that you have correctly exported the environment variables as described above
  • execute the following command (this operation takes some time as creates environment).
mlflow run --env-manager local ./src

You can specify model parameters like this:

mlflow run --env-manager local ./src --param-list optimizer=RRM iterations=1000 base_optimizer=GD

Running local model server​

Install requirements.

pip install -r requirements-dev-modelserver.txt

Run a model server​

Access experiment from tracking server web UI at http://127.0.0.1:5000.

Access experiment

Copy experiment runid.

Copy run id

Use the runid to set the corresponding environment variable.

export MLFLOW_RUN_ID=<runid>

Serve the model identified by the runid.

mlflow models serve -m runs:/$MLFLOW_RUN_ID/model --enable-mlserver -p 5001