API Reference¶

install¶

mim.commands.install.check_mim_resources() → None[source]¶

Check if the mim resource directory exists.

Newer versions of the OpenMMLab packages have packaged the mim resource files into the distribution package, while earlier versions do not.

If the mim resources file (aka .mim) do not exists, log a warning that a new version needs to be installed.

mim.commands.install.filter_invalid_marker(extra_requires: List) → None[source]¶

Filter out invalid marker in requirements parsed from METADATA.

More detail can be found at: https://github.com/pypa/pip/issues/11191

Parameters: extra_requires (list) – A list of Requirement parsed from distribution METADATA.

mim.commands.install.get_mmcv_full_find_link(mmcv_base_url: str) → str[source]¶

Get the mmcv-full find link corresponding to the current environment.

Parameters: mmcv_base_url (str) – The base URL of mmcv find link.
Returns: The mmcv find links corresponding to the current torch version and cuda version.
Return type: str

mim.commands.install.get_mmdeps_from_mmpkg(mmpkg_name: str, mmpkg_version: str, index_url: Optional[str] = None) → str[source]¶

Get ‘mim’ extra requirements for a given OpenMMLab package from mminstall.txt.

If there is a cached mminstall.txt, use the cache, otherwise download the source distribution package from pypi and extract mminstall.txt content.

Parameters

mmpkg_name (str) – The OpenMMLab package name.
mmpkg_version (str) – The OpenMMLab package version.
index_url (str, optional) – The pypi index url that pass to get_mminstall_from_pypi.

Returns

The text content read from mminstall.txt, returns an empty string if anything goes wrong.

Return type

str

mim.commands.install.get_mminstall_from_pypi(mmpkg: str, index_url: Optional[str] = None) → str[source]¶

Get the mminstall.txt content for a given OpenMMLab package from PyPi.

Parameters

mmpkg (str) – The OpenMMLab package name, optionally with a version specifier. e.g. ‘mmdet’, ‘mmdet==2.25.0’.
index_url (str, optional) – The pypi index url, if given, will be used in pip download.

Returns

The text content read from mminstall.txt, returns an empty string if anything goes wrong.

Return type

str

mim.commands.install.install(install_args: List[str], index_url: Optional[str] = None, is_yes: bool = False) → Any[source]¶

Install packages via pip and add ‘mim’ extra requirements for OpenMMLab packages during pip install process.

Parameters

install_args (list) – List of arguments passed to pip install.
index_url (str, optional) – The pypi index url.
is_yes (bool, optional) – Deprecated, will have no effect. Reserved for interface compatibility only.

mim.commands.install.patch_importlib_distribution(index_url: Optional[str] = None) → Generator[source]¶

A patch for pip._internal.metadata.importlib.Distribution.

Since the old version of the OpenMMLab packages did not add the ‘mim’ extra requirements to the release distribution, we need to hack the Distribution and manually fetch the ‘mim’ requirements from mminstall.txt.

This patch works with ‘pip>=22.1,python>=3.11’.

Parameters: index_url (str, optional) – The pypi index url that pass to get_mminstall_from_pypi.

mim.commands.install.patch_pkg_resources_distribution(index_url: Optional[str] = None) → Generator[source]¶

A patch for pip._vendor.pkg_resources.Distribution.

Since the old version of the OpenMMLab packages did not add the ‘mim’ extra requirements to the release distribution, we need to hack the Distribution and manually fetch the ‘mim’ requirements from mminstall.txt.

This patch works with ‘pip<22.1’ and ‘pip>=22.1,python<3.11’.

Parameters: index_url (str, optional) – The pypi index url that pass to get_mmdeps_from_mmpkg_pypi.

uninstall¶

mim.commands.uninstall.uninstall(uninstall_args: Union[str, List], confirm_yes: bool = True, requirements: Tuple = ()) → Any[source]¶

Uninstall package.

Parameters

uninstall_args (str or list) – A package name or a list of package names to uninstalled. You can also put some pip uninstal options here.
confirm_yes (bool) – Don’t ask for confirmation of uninstall deletions. Default: True.
requirements (tuple) – A tuple of requirements files to uninstalled.

Returns

The status code returned by pip uninstall.

list¶

mim.commands.list.list_package(all: bool = False) → List[Tuple[str, ...]][source]¶

List packages.

List packages of OpenMMLab projects or all the packages in the python environment.

Parameters: all (bool) – List all installed packages. If all is False, it just lists the packages installed by mim. Default: False.

search¶

mim.commands.search.convert2df(metadata: modelindex.models.ModelIndex.ModelIndex) → pandas.core.frame.DataFrame[source]¶: Convert metadata into DataFrame format.

mim.commands.search.dump2json(dataframe: pandas.core.frame.DataFrame, json_path: str) → None[source]¶

Dump data frame of meta data into JSON.

Parameters

dataframe (DataFrame) – Data to be filtered.
json_path (str) – Dump output to json_path.

mim.commands.search.filter_by_conditions(dataframe: pandas.core.frame.DataFrame, filter_conditions: Optional[str] = None) → pandas.core.frame.DataFrame[source]¶

Filter rows with conditions.

Parameters

dataframe (DataFrame) – Data to be filtered.
filter_conditions (str, optional) – Conditions to filter. Default: None.

mim.commands.search.filter_by_configs(dataframe: pandas.core.frame.DataFrame, configs: Optional[List[str]] = None) → pandas.core.frame.DataFrame[source]¶

Filter by configs.

Parameters

dataframe (DataFrame) – Data to be filtered.
configs (List[str], optional) – Config ids to query. Default: None.

mim.commands.search.filter_by_models(dataframe: pandas.core.frame.DataFrame, models: Optional[List[str]] = None) → pandas.core.frame.DataFrame[source]¶

Filter by models.

Parameters

dataframe (DataFrame) – Data to be filtered.
models (List[str], optional) – Models to query. Default: None.

mim.commands.search.filter_by_training_datasets(dataframe: pandas.core.frame.DataFrame, datasets: Optional[List[str]]) → pandas.core.frame.DataFrame[source]¶

Filter by training datasets.

Parameters

dataframe (DataFrame) – Data to be filtered.
datasets (List[str], optional) – Training datasets to query. Default: None.

mim.commands.search.get_model_info(package: str, configs: Optional[List[str]] = None, models: Optional[List[str]] = None, training_datasets: Optional[List[str]] = None, filter_conditions: Optional[str] = None, sorted_fields: Optional[List[str]] = None, ascending: bool = True, shown_fields: Optional[List[str]] = None, unshown_fields: Optional[List[str]] = None, local: bool = True, to_dict: bool = False, index_url: Optional[str] = None) → Any[source]¶

Get model information like metric or dataset.

Parameters

package (str) – Name of package to load metadata.
configs (List[str], optional) – Config ids to query. Default: None.
models (List[str], optional) – Models to query. Default: None.
training_datasets (List[str], optional) – Training datasets to query. Default: None.
filter_conditions (str, optional) – Conditions to filter. Default: None.
sorted_fields (List[str], optional) – Sort output by sorted_fields. Default: None.
ascending (bool) – Sort by ascending or descending. Default: True.
shown_fields (List[str], optional) – Fields to be outputted. Default: None.
unshown_fields (List[str], optional) – Fields to be hidden. Default: None.
local (bool) – Query from local environment or remote github. Default: True.
to_dict (bool) – Convert dataframe into dict. Default: False.
index_url (str, optional) – The pypi index url, if given, will be used in pip download command for downloading packages when local is False. Default: None.

mim.commands.search.load_metadata(package: str, local: bool = True, index_url: Optional[str] = None) → Optional[modelindex.models.ModelIndex.ModelIndex][source]¶

Load metadata from local package or remote package.

Parameters

package (str) – Name of package to load metadata.
local (bool) – Query from local environment or remote github. Default: True.
index_url (str, optional) – The pypi index url, if given, will be used in pip download command for downloading packages when local is False. Default: None.

mim.commands.search.load_metadata_from_local(package: str)[source]¶

Load metadata from local package.

Parameters: package (str) – Name of package to load metadata.

Example

>>> metadata = load_metadata_from_local('mmcls')

mim.commands.search.load_metadata_from_remote(package: str, index_url: Optional[str] = None) → Optional[modelindex.models.ModelIndex.ModelIndex][source]¶

Load metadata from PyPI.

Download the model_zoo directory from PyPI and parse it into metadata.

Parameters

package (str) – Name of package to load metadata.
index_url (str, optional) – The pypi index url, if given, will be used in pip download command for downloading packages. Default: None.

Example

>>> # load metadata from latest version
>>> metadata = load_metadata_from_remote('mmcls')
>>> # load metadata from 0.11.0
>>> metadata = load_metadata_from_remote('mmcls==0.11.0')

mim.commands.search.print_df(dataframe: pandas.core.frame.DataFrame, display_width: int = 80) → None[source]¶: Print Dataframe into terminal.

mim.commands.search.select_by(dataframe: pandas.core.frame.DataFrame, shown_fields: Optional[List[str]] = None, unshown_fields: Optional[List[str]] = None) → pandas.core.frame.DataFrame[source]¶

Select by the fields.

When selecting some fields to be shown or be hidden, substring is supported. For example, if shown_fields is [‘epo’], all field contain ‘epo’ which will be chosen. So the new shown field will be [‘epochs’].

Parameters

dataframe (DataFrame) – Data to be filtered.
shown_fields (List[str], optional) – Fields to be outputted. Default: None.
unshown_fields (List[str], optional) – Fields to be hidden. Default: None.

mim.commands.search.sort_by(dataframe: pandas.core.frame.DataFrame, sorted_fields: Optional[List[str]], ascending: bool = True) → pandas.core.frame.DataFrame[source]¶

Sort by the fields.

When sorting output with some fields, substring is supported. For example, if sorted_fields is [‘epo’], the actual sorted fieds will be [‘epochs’].

Parameters

dataframe (DataFrame) – Data to be sorted.
sorted_fields (List[str], optional) – Sort output by sorted_fields. Default: None.
ascending (bool) – Sort by ascending or descending. Default: True.

download¶

mim.commands.download.download(package: str, configs: Optional[List[str]] = None, dest_root: Optional[str] = None, check_certificate: bool = True, dataset: Optional[str] = None) → Optional[List[str]][source]¶

Download checkpoints from url and parse configs from package.

Parameters

package (str) – Name of package.
configs (List[str], optional) – List of config ids.
dest_root (str, optional) – Destination directory to save checkpoint and config. Default: None.
check_certificate (bool) – Whether to check the ssl certificate. Default: True.
dataset (str, optional) – The name of dataset.

train¶

mim.commands.train.train(package: str, config: str, gpus: int, gpus_per_node: Optional[int] = None, cpus_per_task: int = 2, partition: Optional[str] = None, launcher: str = 'none', port: Optional[int] = None, srun_args: Optional[str] = None, yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]][source]¶

Train a model with given config.

Parameters

package (str) – The codebase name.
config (str) – The config file path. If not exists, will search in the config files of the codebase.
gpus (int) – Number of gpus used for training.
gpus_per_node (int, optional) – Number of gpus per node to use (only applicable to launcher == “slurm”). Defaults to None.
cpus_per_task (int, optional) – Number of cpus per task to use (only applicable to launcher == “slurm”). Defaults to None.
partition (str, optional) – The partition name (only applicable to launcher == “slurm”). Defaults to None.
launcher (str, optional) – The launcher used to launch jobs. Defaults to ‘none’.
port (int | None, optional) – The port used for inter-process communication (only applicable to slurm / pytorch launchers). Default to None. If set to None, will randomly choose a port between 20000 and 30000.
srun_args (str, optional) – Other srun arguments that might be used, all arguments should be in a string. Defaults to None.
yes (bool) – Don’t ask for confirmation. Default: True.
other_args (tuple, optional) – Other arguments, will be passed to the codebase’s training script. Defaults to ().

test¶

mim.commands.test.test(package: str, config: str, checkpoint: Optional[str] = None, gpus: Optional[int] = None, gpus_per_node: Optional[int] = None, cpus_per_task: int = 2, partition: Optional[str] = None, launcher: str = 'none', port: Optional[int] = None, srun_args: Optional[str] = None, yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]][source]¶

Test a model with given config.

Parameters

package (str) – The codebase name.
config (str) – The config file path. If not exists, will search in the config files of the codebase.
checkpoint (str) – The path to the checkpoint file. Default to None.
gpus (int) – Number of gpus used for testing (only applicable to launcher == “slurm”). Defaults to None.
gpus_per_node (int, optional) – Number of gpus per node to use (only applicable to launcher == “slurm”). Defaults to None.
cpus_per_task (int, optional) – Number of cpus per task to use (only applicable to launcher == “slurm”). Defaults to None.
partition (str, optional) – The partition name (only applicable to launcher == “slurm”). Defaults to None.
launcher (str, optional) – The launcher used to launch jobs. Defaults to ‘none’.
port (int | None, optional) – The port used for inter-process communication (only applicable to slurm / pytorch launchers). Default to None. If set to None, will randomly choose a port between 20000 and 30000.
srun_args (str, optional) – Other srun arguments that might be used, all arguments should be in a string. Defaults to None.
yes (bool) – Don’t ask for confirmation. Default: True.
other_args (tuple, optional) – Other arguments, will be passed to the codebase’s training script. Defaults to ().

run¶

mim.commands.run.run(package: str, command: str, yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]][source]¶

Run arbitrary command of a codebase.

This command assumes the command scripts have been put into the package/tools directory.

Parameters

package (str) – The codebase name.
command (str) – The command name.
yes (bool) – Don’t ask for confirmation. Default: True.
other_args (tuple, optional) – Other arguments, will be passed to the codebase’s script. Defaults to ().

gridsearch¶

mim.commands.gridsearch.gridsearch(package: str, config: str, gpus: int, gpus_per_node: Optional[int] = None, cpus_per_task: int = 2, max_jobs: int = 1, partition: Optional[str] = None, launcher: str = 'none', port: int = 29500, srun_args: Optional[str] = None, search_args: str = '', yes: bool = True, other_args: tuple = ()) → Tuple[bool, Union[str, Exception]][source]¶

Hyper parameter search with given config.

Parameters

package (str) – The codebase name.
config (str) – The config file path. If not exists, will search in the config files of the codebase.
gpus (int) – Number of gpus used for training.
gpus_per_node (int, optional) – Number of gpus per node to use (only applicable to launcher == “slurm”). Defaults to None.
cpus_per_task (int, optional) – Number of cpus per task to use (only applicable to launcher == “slurm”). Defaults to None.
partition (str, optional) – The partition name (only applicable to launcher == “slurm”). Defaults to None.
max_jobs (int, optional) – The max number of workers. Applicable only if launcher == ‘slurm’. Default to 1.
launcher (str, optional) – The launcher used to launch jobs. Defaults to ‘none’.
port (int, optional) – The port used for inter-process communication (only applicable to slurm / pytorch launchers). Default to 29500.
srun_args (str, optional) – Other srun arguments that might be used, all arguments should be in a string. Defaults to None.
search_args (str, optional) – Arguments for hyper parameters search, all arguments should be in a string. Defaults to None.
yes (bool) – Don’t ask for confirmation. Default: True.
other_args (tuple, optional) – Other arguments, will be passed to the codebase’s training script. Defaults to ().

utils¶

mim.utils.args2string(args: dict) → str[source]¶

Convert args dictionary to a string.

Parameters: args (dict) – A dictionary that contains parsed args.
Returns: A converted string.

Example

>>> args = {
    'arg1': [value1, value2],
    'arg2': [value3],
    'arg3': [value4]
}
>>> args2string(args)
'--arg1 value1 value2 --arg2 value3 --arg3 value4'

mim.utils.cast2lowercase(input: Union[list, tuple, str]) → Any[source]¶

Cast input into lowercase.

Example

>>> cast2lowercase('Hello World')
'hello world'
>>> cast2lowercase(['Hello', 'World'])
['hello', 'world']

mim.utils.download_from_file(url: str, dest_path: str, hash_prefix: Optional[str] = None, check_certificate: bool = True) → None[source]¶

Download object at the given URL to a local path.

Parameters

url (str) – URL of the object to download.
dest_path (str) – Path where object will be saved.
hash_prefix (string, optional) – If not None, the SHA256 downloaded file should start with hash_prefix. Default: None.
check_certificate (bool) – Whether to check the ssl certificate. Default: True.

mim.utils.ensure_installation(func)[source]¶

A decorator to make sure a package has been installed.

Before invoking those functions which depend on installed package, the decorator makes sure the package has been installed.

mim.utils.extract_tar(tar_path: str, dst: str) → None[source]¶

Extract file from tar.

Parameters

tar_path (str) – Path for extracting.
dst (str) – Destination to save file.

mim.utils.get_config(cfg, name)[source]¶

Given the argument name, read the value from the config file.

The name can be multi-level, like ‘optimizer.lr’

mim.utils.get_content_from_url(url: str, timeout: int = 15, stream: bool = False, check_certificate: bool = True) → requests.models.Response[source]¶

Get content from url.

Parameters

url (str) – Url for getting content.
timeout (int) – Set the socket timeout. Default: 15.
check_certificate (bool) – Whether to check the ssl certificate. Default: True.

mim.utils.get_github_url(package: str) → str[source]¶

Get github url.

Parameters: package (str) – Name of package, like mmcls.

Example

>>> get_github_url('mmcls')
'https://github.com/open-mmlab/mmclassification.git'

mim.utils.get_installed_path(package: str) → str[source]¶

Get installed path of package.

Parameters: package (str) – Name of package.

Example

>>> get_installed_path('mmcls')
>>> '.../lib/python3.7/site-packages/mmcls'

mim.utils.get_installed_version(package: str) → str[source]¶

Get the version of package from local environment.

Parameters: package (str) – Name of package.

mim.utils.get_latest_version(package: str, timeout: int = 15) → str[source]¶

Get latest version of package.

Parameters

package (str) – Package to get latest version.
timeout (int) – Set the socket timeout. Default: 15.

Example

>>> get_latest_version('mmcv-full')
    '0.11.0'

mim.utils.get_package_info_from_pypi(package: str, timeout: int = 15) → dict[source]¶

Get package information from pypi.

Parameters

package (str) – Package to get information.
timeout (int) – Set the socket timeout. Default: 15.

mim.utils.get_package_version(repo_root: str) → Tuple[str, str][source]¶

Get package and version from local repo.

Parameters: repo_root (str) – Directory of repo.

mim.utils.get_release_version(package: str, timeout: int = 15) → List[str][source]¶

Get release version from pypi.

The return list of versions is sorted by ascending order.

Parameters

package (str) – Package to get version.
timeout (int) – Set the socket timeout. Default: 15.

mim.utils.get_torch_cuda_version() → Tuple[str, str][source]¶

Get PyTorch version and CUDA version if it is available.

Example

>>> get_torch_cuda_version()
'1.8.0', '102'

mim.utils.is_installed(package: str) → bool[source]¶

Check package whether installed.

Parameters: package (str) – Name of package to be checked.

mim.utils.module_full_name(abbr: str) → str[source]¶

Get the full name of the module given abbreviation.

Parameters

abbr (str) – The abbreviation, should be the sub-string of one (and only one) supported module.

Returns

The full name of the corresponding module. If abbr is the: sub-string of zero / multiple module names, return empty string.

Return type

str

mim.utils.package2module(package: str)[source]¶

Infer module name from package.

Parameters: package (str) – Package to infer module name.

mim.utils.parse_home_page(package: str) → Optional[str][source]¶

Parse home page from package metadata.

Parameters: package (str) – Package to parse home page.

mim.utils.parse_url(url: str) → Tuple[str, str][source]¶

Parse username and repo from url.

Parameters: url (str) – Url for parsing username and repo name.

Example

>>> parse_url('https://github.com/open-mmlab/mmcv.git')
'open-mmlab', 'mmcv'
>>> parse_ulr('git@github.com:open-mmlab/mmcv.git')
'open-mmlab', 'mmcv'

mim.utils.recursively_find(root: str, base_name: str, followlinks=False) → list[source]¶

Recursive list a directory, return all files with a given base_name.

Parameters

root (str) – The root directory to list.
base_name (str) – The base_name.
followlinks (bool) – Follow symbolic links. Defaults to False.

Returns

Files with given base_name.

mim.utils.set_config(cfg, name, value)[source]¶

Given the argument name and value, set the value of the config file.

The name can be multi-level, like ‘optimizer.lr’

mim.utils.split_package_version(package: str) → Tuple[str, ...][source]¶

Split the package which maybe contains version info.

Parameters: package (str) – Name of package to split.

Example

>>> split_package_version('mmcls')
'mmcls', ''
>>> split_package_version('mmcls=0.11.0')
'mmcls', '0.11.0'
>>> split_package_version('mmcls==0.11.0')
'mmcls', '0.11.0'

mim.utils.string2args(text: str) → dict[source]¶

Parse string to arguments.

Parameters: text (str) – The string to be parsed, which should be of the format: “–arg1 value1 value2 –arg2 value1 … –argn value1”. Using ‘=’ is also OK, like “–argn=value1”. It also support flag args like “–arg1”.
Returns: A dictionary that contains parsed args. Note that the type of values will all be strings.

Example

>>> text = '--arg1 value1 value2 --arg2 value3 --arg3 value4'
>>> string2args(text)
args = {
    'arg1': [value1, value2],
    'arg2': [value3],
    'arg3': [value4]
}