leaderbot.data.convert#

leaderbot.data.convert(input_file, output_file)#

Convert Chatbot Arena’s raw data.

Parameters:
input_filestr

A .json filename of the raw data. The filename can be the location on the local machine or a URL of a file on a remote server accessible via the HTTP or HTTPS protocol.

output_filestr

The output .json file to write the converted data.

See also

leaderbot.data.load

Notes

This function converts the raw JSON file from Chatbot Arena to another JSON file compatible as an input to leaderbot.data.load() function.

In particular, the output JSON file contains the following keys and values:

  • 'X':

    A list of tuple of two indices (i, j) representing a match between a pair of agents with the indices i and j.

  • 'Y':

    A list of tuples of four integers (n_win, n_loss, n_ties, n_both_bad) representing the frequencies of win, loss, ties, and ties as both bad, between agents i and j given by the corresponding tuple in X.

  • 'models': a list of the name of agents in the match.

Examples

>>> from leaderbot.data import convert, load

>>> # Input data
>>> input_file = 'https://storage.googleapis.com/arena_external_' + \
...              'data/public/clean_battle_20240814_public.json'

>>> # Convert data
>>> output_file = 'converted_data.json'
>>> convert(input_file, output_file)

>>> # Load the converted data
>>> data = load(output_file)