API Reference

Ranking Models

Classes to define statistical ranking models to train on data.

leaderbot.models.BradleyTerry(data[, k_cov])	Generalized Bradley-Terry model.
leaderbot.models.RaoKupper(data[, k_cov, k_tie])	Generalized Rao-Kupper model.
leaderbot.models.Davidson(data[, k_cov, k_tie])	Generalized Davidson model.

Model Evaluation

Evaluate metrics for model selection, goodness of fit, and generalization performance.

leaderbot.evaluate.model_selection(models[, ...])	Evaluate model selection.
leaderbot.evaluate.goodness_of_fit(models[, ...])	Evaluate metrics for goodness of fit.
leaderbot.evaluate.generalization(models[, ...])	Evaluate metrics for generalization performance.
leaderbot.evaluate.compare_ranks(models[, ...])	Compare ranking of various models.

Data

Load, convert, and split data.

leaderbot.data.load([filename, tie, ...])	Load data from JSON file or URL.
leaderbot.data.convert(input_file, output_file)	Convert Chatbot Arena's raw data.
leaderbot.data.split(data[, test_ratio, seed])	Split data to training and test data.
leaderbot.data.sample_whitelist	Built-in mutable sequence.

Types

Data types (for internal docstring only).

leaderbot.data.DataType

Standard data type for input data.