Hub methods

Methods for using the Hugging Face Hub:

Push to hub

evaluate.push_to_hub

( model_id: str task_type: str dataset_type: str dataset_name: str metric_type: str metric_name: str metric_value: float task_name: str = None dataset_config: str = None dataset_split: str = None dataset_revision: str = None dataset_args: typing.Dict[str, int] = None metric_config: str = None metric_args: typing.Dict[str, int] = None overwrite: bool = False )

Parameters

model_id (str) — Model id from https://hf.co/models.
task_type (str) — Task id, refer to https://github.com/huggingface/evaluate/blob/main/src/evaluate/config.py#L154 for allowed values.
dataset_type (str) — Dataset id from https://hf.co/datasets.
dataset_name (str) — Pretty name for the dataset.
metric_type (str) — Metric id from https://hf.co/metrics.
metric_name (str) — Pretty name for the metric.
metric_value (float) — Computed metric value.
task_name (str, optional) — Pretty name for the task.
dataset_config (str, optional) — Dataset configuration used in datasets.load_dataset(). See huggingface/datasets docs for more info: https://huggingface.co/docs/datasets/package_reference/loading_methods#datasets.load_dataset.name
dataset_split (str, optional) — Name of split used for metric computation.
dataset_revision (str, optional) — Git hash for the specific version of the dataset.
dataset_args (dict[str, int], optional) — Additional arguments passed to datasets.load_dataset().
metric_config (str, optional) — Configuration for the metric (e.g. the GLUE metric has a configuration for each subset)
metric_args (dict[str, int], optional) — Arguments passed during Metric.compute().
overwrite (bool, optional, defaults to False) — If set to True an existing metric field can be overwritten, otherwise attempting to overwrite any existing fields will cause an error.

Pushes the result of a metric to the metadata of a model repository in the Hub.

Example:

>>> push_to_hub(
...     model_id="huggingface/gpt2-wikitext2",
...     metric_value=0.5
...     metric_type="bleu",
...     metric_name="BLEU",
...     dataset_name="WikiText",
...     dataset_type="wikitext",
...     dataset_split="test",
...     task_type="text-generation",
...     task_name="Text Generation"
... )

Evaluate

Hub methods

Push to hub

evaluate.push_to_hub