coremltools

Use Core ML to integrate machine learning models into your app. Core ML provides a unified representation for all models. Your app uses Core ML APIs and user data to make predictions, and to train or fine-tune models, all on the user’s device.

Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption. Running a model strictly on the user’s device removes any need for a network connection, which helps keep the user’s data private and your app responsive.

Open Neural Network eXchange is a file format shared across many neural network training frameworks.

❗️

ONNX Converter is not recommended for PyTorch conversion

With coremltools-4.x, we recommend the use of the PyTorch converter for PyTorch model.
New features are not being added for ONNX to CoreML converter.

You can convert a model from ONNX to Core ML using the following code:

import coremltools as ct

# Convert from ONNX to Core ML
model  = ct.converters.onnx.convert(model='my_model.onnx')

The argument minimum_ios_deployment_target controls the set of Core ML layers used by the converter. When its value is set to 12, only the set of layers that were shipped in Core ML during the iOS 12, macOS 14 release cycle are used.

It is recommended to first use this setting. If successful, it produces a Core ML model that can be deployed to iOS 12 and higher. If it results in an error due to an unsupported operator or parameter, the target should be set to 13 so the converter can utilize all the layers (including control flow, recurrent layers etc) that were shipped in Core ML in iOS 13.

Operators

❗️

ONNX Version support

ONNX to Core ML supports ONNX Opset version 10 and lower

List of ONNX operators supported in Core ML 2.0 via the converter

List of ONNX operators supported in Core ML 3.0 via the converter

Some of the operators are partially compatible with Core ML, for example gemm with more than 1 non constant input is not supported in Core ML 2, or scale as an input for upsample layer is not supported in Core ML 3 etc.
For unsupported ops or unsupported attributes within supported ops, Core ML custom layers or custom functions can be used.
See the testing script tests/custom_layers_test.py on how to produce Core ML models with custom layers and custom functions.

Supported Models

Models from https://github.com/onnx/models that have been tested to work with this converter:

  • BVLC Alexnet
  • BVLC Caffenet
  • BVLC Googlenet
  • BVLC reference_rcnn_ilsvrc13
  • Densenet
  • Emotion-FERPlus
  • Inception V1
  • Inception V2
  • MNIST
  • Resnet50
  • Shufflenet
  • SqueezeNet
  • VGG
  • ZFNet

Usage

For PyTorch models, please use direct PyTorch to Core ML converter. Please read this section for some examples on PyTorch conversion.

For ONNX models, please checkout these Jupyter notebooks for example usage.

Conversion API Parameters

Following is a list of parameters supported by ONNX to Core ML conversion API and their respective definitions.

from coremltools.converters.onnx import convert

def convert(model,
            mode=None,
            image_input_names=[],
            preprocessing_args={},
            image_output_names=[],
            deprocessing_args={},
            class_labels=None,
            predicted_feature_name='classLabel',
            add_custom_layers=False,
            custom_conversion_functions={},
            minimum_ios_deployment_target='13')
"""
__model__: ONNX model | str
      An ONNX model with parameters loaded in onnx package or path to file
      with models.

__mode__: str ('classifier', 'regressor' or None)
      Mode of the converted coreml model:
      'classifier', a NeuralNetworkClassifier spec will be constructed.
      'regressor', a NeuralNetworkRegressor spec will be constructed.

__image_input_names__: list of strings
      Name of the inputs to be defined as image type. Otherwise, by default all inputs are MultiArray type.

__preprocessing_args__: dict
      Specify preprocessing parameters, that are be applied to all the image inputs specified through the "image_input_names" parameter.
      'is_bgr', 'red_bias', 'green_bias', 'blue_bias', 'gray_bias',
      'image_scale' keys with the same meaning as coremltools set_pre_processing_parameters: https://apple.github.io/coremltools/generated/coremltools.models.neural_network.builder.html#coremltools.models.neural_network.builder.NeuralNetworkBuilder.set_pre_processing_parameters

__image_output_names__: list of strings
      Name of the outputs to be defined as image type. Otherwise, by default all outputs are MultiArray type.

__deprocessing_args__: dict
      Same as 'preprocessing_args' but for the outputs.

__class_labels__: A string or list of strings.
      As a string it represents the name of the file which contains
      the classification labels (one per line).
      As a list of strings it represents a list of categories that map
      the index of the output of a neural network to labels in a classifier.

__predicted_feature_name__: str
      Name of the output feature for the class labels exposed in the Core ML
      model (applies to classifiers only). Defaults to 'classLabel'

__add_custom_layers__: bool
      If True, then ['custom'](https://developer.apple.com/documentation/coreml/core_ml_api/integrating_custom_layers?language=objc) layers will be added to the model in place of unsupported onnx ops or for the ops
      that have unsupported attributes.
      Parameters for these custom layers should be filled manually by editing the mlmodel
      or the 'custom_conversion_functions' argument can be used to do the same during the process of conversion

__custom_conversion_fuctions__: dict (str: function)
      Specify custom function to be used for conversion for given op.
        User can override existing conversion function and provide their own custom implementation to convert certain ops.
        Dictionary key must be string specifying ONNX Op name or Op type and value must be a function implementation available in current context.
        Example usage: {'Flatten': custom_flatten_converter, 'Exp': exp_converter}
        `custom_flatten_converter()` and `exp_converter()` will be invoked instead of internal onnx-coreml conversion implementation for these two Ops;
        Hence, User must provide implementation for functions specified in the dictionary. If user provides two separate functions for node name and node type, then custom function tied to node name will be used. As, function tied to node type is more generic than one tied to node name.
        `custom_conversion_functions` option is different than `add_custom_layers`. Both options can be used in conjuction in which case, custom function will be invoked for provided ops and custom layer will be added for ops with no respective conversion function.
        This option gives finer control to user. One use case could be to modify input attributes or certain graph properties before calling
        existing onnx-coreml conversion function. Note that, It is custom conversion function's responsibility to add respective Core ML layer into builder(coreml tools's NeuralNetworkBuilder).
        Examples: https://github.com/onnx/onnx-coreml/blob/master/tests/custom_layers_test.py#L43

__onnx_coreml_input_shape_map__: dict (str: List[int])
    (Optional)
    (only used if `minimum_ios_deployment_target` version is less than '13')
    A dictionary with keys corresponding to the model input names. Values are a list of integers that specify
    how the shape of the input is mapped to Core ML. Convention used for Core ML shapes is:
    0: Sequence, 1: Batch, 2: channel, 3: height, 4: width.
    For example, an input of rank 2 could be mapped as [3,4] (i.e. H,W) or [1,2] (i.e. B,C) etc.

__minimum_ios_deployment_target__: str
      Target Deployment iOS version (default: '12')
      Supported values: '11.2', '12', '13'
      Core ML model produced by the converter will be compatible with the iOS version specified in this argument.
      e.g. if `minimum_ios_deployment_target` = '12', the converter would only utilize Core ML features released till iOS12
      (equivalently macOS 10.14, watchOS 5 etc).
      Release notes:
      * iOS 11 / Core ML 1: https://github.com/apple/coremltools/releases/tag/v0.8
      * iOS 12 / Core ML 2: https://github.com/apple/coremltools/releases/tag/v2.0
      * iOS 13 / Core ML 3: https://github.com/apple/coremltools/releases/tag/v3.0-beta
"""

Updated 5 months ago


ONNX


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.