coremltools

Use Core ML to integrate machine learning models into your app. Core ML provides a unified representation for all models. Your app uses Core ML APIs and user data to make predictions, and to train or fine-tune models, all on the userโ€™s device.

Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption. Running a model strictly on the userโ€™s device removes any need for a network connection, which helps keep the userโ€™s data private and your app responsive.

Unified Conversion API

With coremltools 4.0 and newer versions, use the Unified Conversion API to convert to Core ML models from the following source frameworks:

๐Ÿšง

API compatibility

The Unified Conversion API supports only TensorFlow and PyTorch neural networks. For converting models using multi-backend Keras, Caffe, and ONNX, use the conversion APIs specific to those packages.

Supported formats

Formats supported by the Unified Conversion API include the following:

Framework

Supported Format

TensorFlow versions 1.x

TensorFlow versions 2.x

PyTorch

  • TorchScript object
  • TorchScript object saved as a .pt file

Load and convert a model

The typical conversion process with the Unified Conversion API is to load the model to infer its type, and then use the convert() method to convert it to the Core ML format. Follow these steps:

  1. Load the model.
import coremltools as ct

# Load TensorFlow model
import tensorflow as tf # Tf 2.2.0

tf_model = tf.keras.applications.MobileNet()
import coremltools as ct

# Load PyTorch model (and perform tracing)
torch_model = torchvision.models.mobilenet_v2()
torch_model.eval() 

example_input = torch.rand(1, 3, 256, 256)
traced_model = torch.jit.trace(torch_model, example_input)
  1. Convert the model using convert():
# Convert using the same API
model_from_tf = ct.convert(tf_model)
# Convert using the same API. Note that we need to provide "inputs" for pytorch conversion.
model_from_torch = ct.convert(traced_model,
                              inputs=[ct.TensorType(name="input", shape=example_input.shape)])

The conversion produces an MLModel object which you can use to make predictions, change metadata, or save to the Core ML format for use in Xcode. For more information, see the Core ML Overview.

๐Ÿ“˜

Conversion options

The convert() method tries to infer as much as possible from the source network, but some information may not be present, such as input names, types, shapes, and classifier options. For more information see Conversion Options.

Convert from TensorFlow 2

TensorFlow 2 models are typically exported as tf.Model objects in the SavedModel or HDF5 file formats. For additional TensorFlow formats you can convert, see the TensorFlow 2 Conversion page.

The following example demonstrates how to use the convert() method to convert an Xception model from tf.keras.applications:

import coremltools as ct 
import tensorflow as tf

# Load from .h5 file
tf_model = tf.keras.applications.Xception(weights="imagenet", 
                                          input_shape=(299, 299, 3))

# Convert to Core ML
model = ct.convert(tf_model)

Convert from TensorFlow 1

The conversion API can also convert models from TensorFlow 1. These models are generally exported with the extension .pb, in the frozen protobuf file format, using TensorFlow 1's freeze graph utility. You can pass this model directly into the convert() method. For details, see the TensorFlow 1 Conversion page.

The following example demonstrates how to convert a pre-trained MobileNet model in the frozen protobuf format to Core ML.

๐Ÿ“˜

Download for the following example

To run the following example, first download this pre-trained model.

import coremltools as ct

# Convert a frozen graph from TensorFlow 1 to Core ML
mlmodel = ct.convert("mobilenet_v1_1.0_224/frozen_graph.pb")

The MobileNet model in the previous example already has a defined input shape, so you did not need to provide it. However, in some cases the TensorFlow model does not contain a fully defined input shape. You can pass an input shape that is compatible with the model into the convert() method in order to provide the shape information, as shown in the following example.

๐Ÿ“˜

Download for the following example

To run the following example, first download this pre-trained model.

import coremltools as ct

# Needs additional shape information
mlmodel = ct.convert("mobilenet_v2_1.0_224_frozen.pb",
                    inputs=[ct.TensorType(shape=(1, 224, 224, 3))])

Convert from PyTorch

You can convert PyTorch models that are either traced or in the TorchScript format. For example, you can convert a model obtained using PyTorch's save and load APIs to Core ML using the same Unified Conversion API as the previous example:

import coremltools as ct
import torch
import torchvision

# Get a pytorch model and save it as a *.pt file
model = torchvision.models.mobilenet_v2()
model.eval()
example_input = torch.rand(1, 3, 224, 224)
traced_model = torch.jit.trace(model, example_input)
traced_model.save("torchvision_mobilenet_v2.pt")

# Convert the saved PyTorch model to Core ML
mlmodel = ct.convert("torchvision_mobilenet_v2.pt",
                    inputs=[ct.TensorType(shape=(1, 3, 224, 224))])

For more details on tracing and scripting to produce PyTorch models for conversion, see the PyTorch page.

Set the compute units

Normally you convert a model by using coremltools.convert() without using the compute_units parameter. In most cases you donโ€™t need it, because the converter picks the default optimized path for fast execution while loading the model. The default setting (ComputeUnit.ALL) uses all compute units available, including the Apple Neural Engine (ANE), the CPU, and the graphics processing unit (GPU). Whether you are using neural networks or ML programs, the defaults for conversion and prediction are picked to execute the model in the most performant way, as described in Typed Execution.

However, you may find it useful, especially for debugging, to specify the actual compute units when converting or loading a model by using the compute_units parameter. The parameter is based on the MLComputeUnits enumeration in the Swift developer language โ€” compute units are employed when loading a Core ML model, taking in MLmodelConfiguration which includes compute units. Therefore, both the coremltools.models.MLModel loader and the coremltools.convert() converter provide the compute_units parameter.

The compute_units parameter can have the following values:

  • coremltools.ComputeUnit.CPU_ONLY: Limit the model to use only the CPU.
  • coremltools.ComputeUnit.CPU_AND_GPU: Use both the CPU and GPU, but not the ANE.
  • coremltools.ComputeUnit.ALL: The default setting uses all compute units available, including the ANE, CPU, and GPU.

For example, the following converts the model and sets the compute_units to CPU only:

model = ct.convert(tf_model, compute_units=ct.ComputeUnit.CPU_ONLY)

For details on how to use this parameter with neural networks, see Neural network untyped tensors. For details on using it with ML programs, see ML program typed tensors.

API reference

For the full list of arguments supported by the coremltools.convert API, see the API reference page.

For common scenarios, see the pages on image inputs, classifiers, and flexible input shapes under Conversion Options.

Amazon SageMaker Neo

You can use Amazon SageMaker Neo to automatically convert TensorFlow and PyTorch models to Core ML using the Unified Conversion API. Amazon SageMaker Neo exposes the Unified Conversion API from its service in the AWS cloud. You can access the API using the AWS Command Line Interface (AWS CLI), the Amazon SageMaker console, or the SDK. For more information see the AWS Machine Learning blog.

Updated 24 days ago



Unified Conversion API


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.