API Reference

install

mim.commands.install.check_mim_resources() → None

Check if the mim resource directory exists.

Newer versions of the OpenMMLab packages have packaged the mim resource files into the distribution package, while earlier versions do not.

If the mim resources file (aka .mim) do not exists, log a warning that a new version needs to be installed.

mim.commands.install.filter_invalid_marker(extra_requires: List) → None

Filter out invalid marker in requirements parsed from METADATA.

More detail can be found at: https://github.com/pypa/pip/issues/11191

Parameters

extra_requires (list) – A list of Requirement parsed from distribution METADATA.

mim.commands.install.get_mmcv_full_find_link(mmcv_base_url: str) → str

Get the mmcv-full find link corresponding to the current environment.

Parameters

mmcv_base_url (str) – The base URL of mmcv find link.

Returns

The mmcv find links corresponding to the current torch version and cuda version.

Return type

str

mim.commands.install.get_mmdeps_from_mmpkg(mmpkg_name: str, mmpkg_version: str, index_url: Optional[str] = None) → str

Get ‘mim’ extra requirements for a given OpenMMLab package from mminstall.txt.

If there is a cached mminstall.txt, use the cache, otherwise download the source distribution package from pypi and extract mminstall.txt content.

Parameters

• mmpkg_name (str) – The OpenMMLab package name.

• mmpkg_version (str) – The OpenMMLab package version.

• index_url (str, optional) – The pypi index url that pass to get_mminstall_from_pypi.

Returns

The text content read from mminstall.txt, returns an empty string if anything goes wrong.

Return type

str

mim.commands.install.get_mminstall_from_pypi(mmpkg: str, index_url: Optional[str] = None) → str

Get the mminstall.txt content for a given OpenMMLab package from PyPi.

Parameters

• mmpkg (str) – The OpenMMLab package name, optionally with a version specifier. e.g. ‘mmdet’, ‘mmdet==2.25.0’.

• index_url (str, optional) – The pypi index url, if given, will be used in pip download.

Returns

The text content read from mminstall.txt, returns an empty string if anything goes wrong.

Return type

str

mim.commands.install.install(install_args: List[str], index_url: Optional[str] = None, is_yes: bool = False) → Any

Install packages via pip and add ‘mim’ extra requirements for OpenMMLab packages during pip install process.

Parameters

• install_args (list) – List of arguments passed to pip install.

• index_url (str, optional) – The pypi index url.

• is_yes (bool, optional) – Deprecated, will have no effect. Reserved for interface compatibility only.

mim.commands.install.patch_importlib_distribution(index_url: Optional[str] = None) → Generator

A patch for pip._internal.metadata.importlib.Distribution.

Since the old version of the OpenMMLab packages did not add the ‘mim’ extra requirements to the release distribution, we need to hack the Distribution and manually fetch the ‘mim’ requirements from mminstall.txt.

This patch works with ‘pip>=22.1,python>=3.11’.

Parameters

index_url (str, optional) – The pypi index url that pass to get_mminstall_from_pypi.

mim.commands.install.patch_pkg_resources_distribution(index_url: Optional[str] = None) → Generator

A patch for pip._vendor.pkg_resources.Distribution.

Since the old version of the OpenMMLab packages did not add the ‘mim’ extra requirements to the release distribution, we need to hack the Distribution and manually fetch the ‘mim’ requirements from mminstall.txt.

This patch works with ‘pip<22.1’ and ‘pip>=22.1,python<3.11’.

Parameters

index_url (str, optional) – The pypi index url that pass to get_mmdeps_from_mmpkg_pypi.

uninstall

mim.commands.uninstall.uninstall(uninstall_args: Union[str, List], confirm_yes: bool = True, requirements: Tuple = ()) → Any

Uninstall package.

Parameters

• uninstall_args (str or list) – A package name or a list of package names to uninstalled. You can also put some pip uninstal options here.

• confirm_yes (bool) – Don’t ask for confirmation of uninstall deletions. Default: True.

• requirements (tuple) – A tuple of requirements files to uninstalled.

Returns

The status code returned by pip uninstall.

list

mim.commands.list.list_package(all: bool = False) → List[Tuple[str, ...]]

List packages.

List packages of OpenMMLab projects or all the packages in the python environment.

Parameters

all (bool) – List all installed packages. If all is False, it just lists the packages installed by mim. Default: False.

search

mim.commands.search.convert2df(metadata: modelindex.models.ModelIndex.ModelIndex) → pandas.core.frame.DataFrame

Convert metadata into DataFrame format.

mim.commands.search.dump2json(dataframe: pandas.core.frame.DataFrame, json_path: str) → None

Dump data frame of meta data into JSON.

Parameters

• dataframe (DataFrame) – Data to be filtered.

• json_path (str) – Dump output to json_path.

mim.commands.search.filter_by_conditions(dataframe: pandas.core.frame.DataFrame, filter_conditions: Optional[str] = None) → pandas.core.frame.DataFrame

Filter rows with conditions.

Parameters

• dataframe (DataFrame) – Data to be filtered.

• filter_conditions (str, optional) – Conditions to filter. Default: None.

mim.commands.search.filter_by_configs(dataframe: pandas.core.frame.DataFrame, configs: Optional[List[str]] = None) → pandas.core.frame.DataFrame

Filter by configs.

Parameters

• dataframe (DataFrame) – Data to be filtered.

• configs (List[str], optional) – Config ids to query. Default: None.

mim.commands.search.filter_by_models(dataframe: pandas.core.frame.DataFrame, models: Optional[List[str]] = None) → pandas.core.frame.DataFrame

Filter by models.

Parameters

• dataframe (DataFrame) – Data to be filtered.

• models (List[str], optional) – Models to query. Default: None.

mim.commands.search.filter_by_training_datasets(dataframe: pandas.core.frame.DataFrame, datasets: Optional[List[str]]) → pandas.core.frame.DataFrame

Filter by training datasets.

Parameters

• dataframe (DataFrame) – Data to be filtered.

• datasets (List[str], optional) – Training datasets to query. Default: None.

mim.commands.search.get_model_info(package: str, configs: Optional[List[str]] = None, models: Optional[List[str]] = None, training_datasets: Optional[List[str]] = None, filter_conditions: Optional[str] = None, sorted_fields: Optional[List[str]] = None, ascending: bool = True, shown_fields: Optional[List[str]] = None, unshown_fields: Optional[List[str]] = None, local: bool = True, to_dict: bool = False, index_url: Optional[str] = None) → Any

Get model information like metric or dataset.

Parameters

• package (str) – Name of package to load metadata.

• configs (List[str], optional) – Config ids to query. Default: None.

• models (List[str], optional) – Models to query. Default: None.

• training_datasets (List[str], optional) – Training datasets to query. Default: None.

• filter_conditions (str, optional) – Conditions to filter. Default: None.

• sorted_fields (List[str], optional) – Sort output by sorted_fields. Default: None.

• ascending (bool) – Sort by ascending or descending. Default: True.

• shown_fields (List[str], optional) – Fields to be outputted. Default: None.

• unshown_fields (List[str], optional) – Fields to be hidden. Default: None.

• local (bool) – Query from local environment or remote github. Default: True.

• to_dict (bool) – Convert dataframe into dict. Default: False.

• index_url (str, optional) – The pypi index url, if given, will be used in pip download command for downloading packages when local is False. Default: None.

mim.commands.search.load_metadata(package: str, local: bool = True, index_url: Optional[str] = None) → Optional[modelindex.models.ModelIndex.ModelIndex]

Load metadata from local package or remote package.

Parameters

• package (str) – Name of package to load metadata.

• local (bool) – Query from local environment or remote github. Default: True.

• index_url (str, optional) – The pypi index url, if given, will be used in pip download command for downloading packages when local is False. Default: None.

mim.commands.search.load_metadata_from_local(package: str)

Load metadata from local package.

Parameters

package (str) – Name of package to load metadata.

Example

>>> metadata = load_metadata_from_local('mmcls')

mim.commands.search.load_metadata_from_remote(package: str, index_url: Optional[str] = None) → Optional[modelindex.models.ModelIndex.ModelIndex]

Load metadata from PyPI.

Download the model_zoo directory from PyPI and parse it into metadata.

Parameters

• package (str) – Name of package to load metadata.

• index_url (str, optional) – The pypi index url, if given, will be used in pip download command for downloading packages. Default: None.

Example

>>> # load metadata from latest version
>>> metadata = load_metadata_from_remote('mmcls')
>>> # load metadata from 0.11.0
>>> metadata = load_metadata_from_remote('mmcls==0.11.0')

mim.commands.search.print_df(dataframe: pandas.core.frame.DataFrame, display_width: int = 80) → None

Print Dataframe into terminal.

mim.commands.search.select_by(dataframe: pandas.core.frame.DataFrame, shown_fields: Optional[List[str]] = None, unshown_fields: Optional[List[str]] = None) → pandas.core.frame.DataFrame

Select by the fields.

When selecting some fields to be shown or be hidden, substring is supported. For example, if shown_fields is [‘epo’], all field contain ‘epo’ which will be chosen. So the new shown field will be [‘epochs’].

Parameters

• dataframe (DataFrame) – Data to be filtered.

• shown_fields (List[str], optional) – Fields to be outputted. Default: None.

• unshown_fields (List[str], optional) – Fields to be hidden. Default: None.

mim.commands.search.sort_by(dataframe: pandas.core.frame.DataFrame, sorted_fields: Optional[List[str]], ascending: bool = True) → pandas.core.frame.DataFrame

Sort by the fields.

When sorting output with some fields, substring is supported. For example, if sorted_fields is [‘epo’], the actual sorted fieds will be [‘epochs’].

Parameters

• dataframe (DataFrame) – Data to be sorted.

• sorted_fields (List[str], optional) – Sort output by sorted_fields. Default: None.

• ascending (bool) – Sort by ascending or descending. Default: True.

download

mim.commands.download.download(package: str, configs: List[str], dest_root: Optional[str] = None, check_certificate: bool = True) → List[str]

Download checkpoints from url and parse configs from package.

Parameters

• package (str) – Name of package.

• configs (List[str]) – List of config ids.

• dest_root (Optional[str]) – Destination directory to save checkpoint and config. Default: None.

• check_certificate (bool) – Whether to check the ssl certificate. Default: True.

train

mim.commands.train.train(package: str, config: str, gpus: int, gpus_per_node: Optional[int] = None, cpus_per_task: int = 2, partition: Optional[str] = None, launcher: str = 'none', port: Optional[int] = None, srun_args: Optional[str] = None, yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]]

Train a model with given config.

Parameters

• package (str) – The codebase name.

• config (str) – The config file path. If not exists, will search in the config files of the codebase.

• gpus (int) – Number of gpus used for training.

• gpus_per_node (int, optional) – Number of gpus per node to use (only applicable to launcher == “slurm”). Defaults to None.

• cpus_per_task (int, optional) – Number of cpus per task to use (only applicable to launcher == “slurm”). Defaults to None.

• partition (str, optional) – The partition name (only applicable to launcher == “slurm”). Defaults to None.

• launcher (str, optional) – The launcher used to launch jobs. Defaults to ‘none’.

• port (int | None, optional) – The port used for inter-process communication (only applicable to slurm / pytorch launchers). Default to None. If set to None, will randomly choose a port between 20000 and 30000.

• srun_args (str, optional) – Other srun arguments that might be used, all arguments should be in a string. Defaults to None.

• yes (bool) – Don’t ask for confirmation. Default: True.

• other_args (tuple, optional) – Other arguments, will be passed to the codebase’s training script. Defaults to ().

test

mim.commands.test.test(package: str, config: str, checkpoint: Optional[str] = None, gpus: Optional[int] = None, gpus_per_node: Optional[int] = None, cpus_per_task: int = 2, partition: Optional[str] = None, launcher: str = 'none', port: Optional[int] = None, srun_args: Optional[str] = None, yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]]

Test a model with given config.

Parameters

• package (str) – The codebase name.

• config (str) – The config file path. If not exists, will search in the config files of the codebase.

• checkpoint (str) – The path to the checkpoint file. Default to None.

• gpus (int) – Number of gpus used for testing (only applicable to launcher == “slurm”). Defaults to None.

• gpus_per_node (int, optional) – Number of gpus per node to use (only applicable to launcher == “slurm”). Defaults to None.

• cpus_per_task (int, optional) – Number of cpus per task to use (only applicable to launcher == “slurm”). Defaults to None.

• partition (str, optional) – The partition name (only applicable to launcher == “slurm”). Defaults to None.

• launcher (str, optional) – The launcher used to launch jobs. Defaults to ‘none’.

• port (int | None, optional) – The port used for inter-process communication (only applicable to slurm / pytorch launchers). Default to None. If set to None, will randomly choose a port between 20000 and 30000.

• srun_args (str, optional) – Other srun arguments that might be used, all arguments should be in a string. Defaults to None.

• yes (bool) – Don’t ask for confirmation. Default: True.

• other_args (tuple, optional) – Other arguments, will be passed to the codebase’s training script. Defaults to ().

run

mim.commands.run.run(package: str, command: str, yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]]

Run arbitrary command of a codebase.

This command assumes the command scripts have been put into the package/tools directory.

Parameters

• package (str) – The codebase name.

• command (str) – The command name.

• yes (bool) – Don’t ask for confirmation. Default: True.

• other_args (tuple, optional) – Other arguments, will be passed to the codebase’s script. Defaults to ().

gridsearch

mim.commands.gridsearch.gridsearch(package: str, config: str, gpus: int, gpus_per_node: Optional[int] = None, cpus_per_task: int = 2, max_jobs: int = 1, partition: Optional[str] = None, launcher: str = 'none', port: int = 29500, srun_args: Optional[str] = None, search_args: str = '', yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]]

Hyper parameter search with given config.

Parameters

• package (str) – The codebase name.

• config (str) – The config file path. If not exists, will search in the config files of the codebase.

• gpus (int) – Number of gpus used for training.

• gpus_per_node (int, optional) – Number of gpus per node to use (only applicable to launcher == “slurm”). Defaults to None.

• cpus_per_task (int, optional) – Number of cpus per task to use (only applicable to launcher == “slurm”). Defaults to None.

• partition (str, optional) – The partition name (only applicable to launcher == “slurm”). Defaults to None.

• max_jobs (int, optional) – The max number of workers. Applicable only if launcher == ‘slurm’. Default to 1.

• launcher (str, optional) – The launcher used to launch jobs. Defaults to ‘none’.

• port (int, optional) – The port used for inter-process communication (only applicable to slurm / pytorch launchers). Default to 29500.

• srun_args (str, optional) – Other srun arguments that might be used, all arguments should be in a string. Defaults to None.

• search_args (str, optional) – Arguments for hyper parameters search, all arguments should be in a string. Defaults to None.

• yes (bool) – Don’t ask for confirmation. Default: True.

• other_args (tuple, optional) – Other arguments, will be passed to the codebase’s training script. Defaults to ().

utils

mim.utils.args2string(args: dict) → str

Convert args dictionary to a string.

Parameters

args (dict) – A dictionary that contains parsed args.

Returns

A converted string.

Example

>>> args = {
    'arg1': [value1, value2],
    'arg2': [value3],
    'arg3': [value4]
}
>>> args2string(args)
'--arg1 value1 value2 --arg2 value3 --arg3 value4'

mim.utils.cast2lowercase(input: Union[list, tuple, str]) → Any

Cast input into lowercase.

Example

>>> cast2lowercase('Hello World')
'hello world'
>>> cast2lowercase(['Hello', 'World'])
['hello', 'world']

mim.utils.download_from_file(url: str, dest_path: str, hash_prefix: Optional[str] = None, check_certificate: bool = True) → None

Download object at the given URL to a local path.

Parameters

• url (str) – URL of the object to download.

• dest_path (str) – Path where object will be saved.

• hash_prefix (string, optional) – If not None, the SHA256 downloaded file should start with hash_prefix. Default: None.

• check_certificate (bool) – Whether to check the ssl certificate. Default: True.

mim.utils.ensure_installation(func)

A decorator to make sure a package has been installed.

Before invoking those functions which depend on installed package, the decorator makes sure the package has been installed.

mim.utils.extract_tar(tar_path: str, dst: str) → None

Extract file from tar.

Parameters

• tar_path (str) – Path for extracting.

• dst (str) – Destination to save file.

mim.utils.get_config(cfg, name)

Given the argument name, read the value from the config file.

The name can be multi-level, like ‘optimizer.lr’

mim.utils.get_content_from_url(url: str, timeout: int = 15, stream: bool = False, check_certificate: bool = True) → requests.models.Response

Get content from url.

Parameters

• url (str) – Url for getting content.

• timeout (int) – Set the socket timeout. Default: 15.

• check_certificate (bool) – Whether to check the ssl certificate. Default: True.

mim.utils.get_github_url(package: str) → str

Get github url.

Parameters

package (str) – Name of package, like mmcls.

Example

>>> get_github_url('mmcls')
'https://github.com/open-mmlab/mmclassification.git'

mim.utils.get_installed_path(package: str) → str

Get installed path of package.

Parameters

package (str) – Name of package.

Example

>>> get_installed_path('mmcls')
>>> '.../lib/python3.7/site-packages/mmcls'

mim.utils.get_installed_version(package: str) → str

Get the version of package from local environment.

Parameters

package (str) – Name of package.

mim.utils.get_latest_version(package: str, timeout: int = 15) → str

Get latest version of package.

Parameters

• package (str) – Package to get latest version.

• timeout (int) – Set the socket timeout. Default: 15.

Example

>>> get_latest_version('mmcv-full')
    '0.11.0'

mim.utils.get_package_info_from_pypi(package: str, timeout: int = 15) → dict

Get package information from pypi.

Parameters

• package (str) – Package to get information.

• timeout (int) – Set the socket timeout. Default: 15.

mim.utils.get_package_version(repo_root: str) → Tuple[str, str]

Get package and version from local repo.

Parameters

repo_root (str) – Directory of repo.

mim.utils.get_release_version(package: str, timeout: int = 15) → List[str]

Get release version from pypi.

The return list of versions is sorted by ascending order.

Parameters

• package (str) – Package to get version.

• timeout (int) – Set the socket timeout. Default: 15.

mim.utils.get_torch_cuda_version() → Tuple[str, str]

Get PyTorch version and CUDA version if it is available.

Example

>>> get_torch_cuda_version()
'1.8.0', '102'

mim.utils.is_installed(package: str) → bool

Check package whether installed.

Parameters

package (str) – Name of package to be checked.

mim.utils.module_full_name(abbr: str) → str

Get the full name of the module given abbreviation.

Parameters

abbr (str) – The abbreviation, should be the sub-string of one (and only one) supported module.

Returns

The full name of the corresponding module. If abbr is the

sub-string of zero / multiple module names, return empty string.

Return type

str

mim.utils.package2module(package: str)

Infer module name from package.

Parameters

package (str) – Package to infer module name.

mim.utils.parse_home_page(package: str) → Optional[str]

Parse home page from package metadata.

Parameters

package (str) – Package to parse home page.

mim.utils.parse_url(url: str) → Tuple[str, str]

Parse username and repo from url.

Parameters

url (str) – Url for parsing username and repo name.

Example

>>> parse_url('https://github.com/open-mmlab/mmcv.git')
'open-mmlab', 'mmcv'
>>> parse_ulr('git@github.com:open-mmlab/mmcv.git')
'open-mmlab', 'mmcv'

mim.utils.recursively_find(root: str, base_name: str, followlinks=False) → list

Recursive list a directory, return all files with a given base_name.

Parameters

• root (str) – The root directory to list.

• base_name (str) – The base_name.

• followlinks (bool) – Follow symbolic links. Defaults to False.

Returns

Files with given base_name.

mim.utils.set_config(cfg, name, value)

Given the argument name and value, set the value of the config file.

The name can be multi-level, like ‘optimizer.lr’

mim.utils.split_package_version(package: str) → Tuple[str, ...]

Split the package which maybe contains version info.

Parameters

package (str) – Name of package to split.

Example

>>> split_package_version('mmcls')
'mmcls', ''
>>> split_package_version('mmcls=0.11.0')
'mmcls', '0.11.0'
>>> split_package_version('mmcls==0.11.0')
'mmcls', '0.11.0'

mim.utils.string2args(text: str) → dict

Parse string to arguments.

Parameters

text (str) – The string to be parsed, which should be of the format: “–arg1 value1 value2 –arg2 value1 … –argn value1”. Using ‘=’ is also OK, like “–argn=value1”. It also support flag args like “–arg1”.

Returns

A dictionary that contains parsed args. Note that the type of values will all be strings.

Example

>>> text = '--arg1 value1 value2 --arg2 value3 --arg3 value4'
>>> string2args(text)
args = {
    'arg1': [value1, value2],
    'arg2': [value3],
    'arg3': [value4]
}