coremltools

Use Core ML to integrate machine learning models into your app. Core ML provides a unified representation for all models. Your app uses Core ML APIs and user data to make predictions, and to train or fine-tune models, all on the user’s device.

Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption. Running a model strictly on the user’s device removes any need for a network connection, which helps keep the user’s data private and your app responsive.

Model Prediction

After conversion, you might want to verify that the predictions made by Core ML match up with the source framework. To facilitate programmatic verification of the conversion, we provide a way for you to easily evaluate Core ML models.

Make Predictions from a Model

The following example makes predictions for the HousePricer.mlmodel using the predict() method.

import coremltools as ct

# Load the model
model = ct.models.MLModel('HousePricer.mlmodel')

# Make predictions
predictions = model.predict({'bedroom': 1.0, 'bath': 1.0, 'size': 1240})

📘

macOS required for model prediction

For the prediction API, coremltools interacts with the Core ML framework which is available on macOS only. Hence the prediction API is not available on Linux.

Types of Inputs and Outputs

Core ML supports several feature types for inputs and outputs.

Let us look in detail the two feature types that are commonly used with neural network models: the ArrayFeatureType (which maps to the MLMultiArray Feature Value in Swift) and the ImageFeatureType (which maps to Image Feature Value in Swift). When using the Core ML model in your Xcode app, you would use an MLFeatureValue.

For evaluating a Core ML model in python using the predict method:

  • For multi-arrays inputs, a numpy array must be fed.
  • For an image input, a PIL image python object should be used.

Multi-array prediction

A ML Model that takes a MultiArray input requires a numpy array as an input during the predict() call. For example:

import coremltools as ct
import numpy as np

model = ct.models.MLModel('path/to/the/saved/model.mlmodel')

# print input description to get input shape
print(model.description.input)

input_shape = (...) # insert correct shape of the input

# call predict
output_dict = model.predict({'input_name': np.random.rand(*input_shape)}, useCPUOnly=True)

Image Prediction

A ML Model takes an image input requires a PIL image during the prediction call. For example:

import coremltools as ct
import numpy as np
import PIL.Image

# load a model whose input type is "Image"
model = ct.models.MLModel('path/to/the/saved/model.mlmodel')

Height = 20  # use the correct input image height
Width = 60  # use the correct input image width


# Scenario 1: load an image from disk
def load_image(path, resize_to=None):
    # resize_to: (Width, Height)
    img = PIL.Image.open(path)
    if resize_to is not None:
        img = img.resize(resize_to, PIL.Image.ANTIALIAS)
    img_np = np.array(img).astype(np.float32)
    return img_np, img


# load the image and resize using PIL utilities
_, img = load_image('/path/to/image.jpg', resize_to=(Width, Height))
out_dict = model.predict({'image': img})

# Scenario 2: load an image from a numpy array
shape = (Height, Width, 3)  # height x width x RGB
data = np.zeros(shape, dtype=np.uint8)
# manipulate numpy data
pil_img = PIL.Image.fromarray(data)
out_dict = model.predict({'image': pil_img})

Image Prediction for a MultiArray Model

If the Core ML model has a MultiArray input type but it really represents an image, the jpeg image can still be used to call predict on the model. To do this, the loaded image should first be converted to a numpy array. For example:

Height = 20  # use the correct input image height
Width = 60  # use the correct input image width

# assumption: the mlmodel's input is of type MultiArray and of shape (1, 3, Height, Width)
model_expected_input_shape = (1, 3, Height, Width) # depending on the model description, this could be (3, Height, Width)

# load the model
model = coremltools.models.MLModel('path/to/the/saved/model.mlmodel')

def load_image_as_numpy_array(path, resize_to=None):
    # resize_to: (Width, Height)
    img = PIL.Image.open(path)
    if resize_to is not None:
        img = img.resize(resize_to, PIL.Image.ANTIALIAS)
    img_np = np.array(img).astype(np.float32) # shape of this numpy array is (Height, Width, 3)
    return img_np

# load the image and resize using PIL utilities
img_as_np_array = load_image_as_numpy_array('/path/to/image.jpg', resize_to=(Width, Height)) # shape (Height, Width, 3)

# note that PIL returns an image in the format in which the channel dimension is in the end,
# which is different than Core ML's input format, so that needs to be modified
img_as_np_array = np.transpose(img_as_np_array, (2,0,1)) # shape (3, Height, Width)

# add the batch dimension if the model description has it
img_as_np_array = np.reshape(img_as_np_array, model_expected_input_shape)

# now call predict
out_dict = model.predict({'image': img_as_np_array})

Updated 5 months ago


Model Prediction


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.