coremltools

Use Core ML to integrate machine learning models into your app. Core ML provides a unified representation for all models. Your app uses Core ML APIs and user data to make predictions, and to train or fine-tune models, all on the user’s device.

Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption. Running a model strictly on the user’s device removes any need for a network connection, which helps keep the user’s data private and your app responsive.

Model Intermediate Language

Model Intermediate Language (MIL) is an intermediate representation (IR) that powers the converter. Behind the scenes, the converter takes a computation graph from a source framework (e.g., TensorFlow and PyTorch) and builds a MIL program. The converter performs a series of optimization over the MIL program, and eventually generates the ML Model from the optimized MIL program.

Notice that MIL is multi-level. This means that instead of a single set of operations, MIL has multiple sets of operations and optimization that may be invoked, depending on factors like the source frameworks (TensorFlow vs PyTorch).

For example, when converting a TensorFlow 1 program, you may use operations and optimization tailored for TensorFlow 1 in the MIL::TF1 "dialect" to facilitate the conversion process. All programs get converted to the MIL::Core dialect.

To learn more about MIL, you can refer to:

Building MIL Program with Builder

You can easily construct a MIL program using the Python builder. In this example, we create a simple program that takes an input that is (1, 100, 100, 3) and creates a few layers of the neural network. This program contains all the information needed to describe the neural network.

# import builder
from coremltools.converters.mil import Builder as mb

# Input to MIL program is a list of tensors. Here we have one input with
# shape (1, 100, 100, 3) and implicit dtype == fp32
@mb.program(input_specs=[mb.TensorSpec(shape=(1, 100, 100, 3)),])
def prog(x):
    # MIL operation takes named inputs (instead of positional inputs).
    # Here `name` argument is optional.
    x = mb.relu(x=x, name='relu')
    x = mb.transpose(x=x, perm=[0, 3, 1, 2], name='transpose')
    x = mb.reduce_mean(x=x, axes=[2, 3], keep_dims=False, name='reduce')
    x = mb.log(x=x, name='log')
    y = mb.add(x=1, y=2)
    return x
  
print(prog)

The output of the following program looks a bit like this.

main(%x: (1, 100, 100, 3, fp32)) {
  block0() {
    %relu: (1, 100, 100, 3, fp32) = relu(x=%x, name="relu")
    %transpose_perm_0: (4,i32)* = const(val=[0, 3, 1, 2], name="transpose_perm_0")
    %transpose: (1, 3, 100, 100, fp32) = transpose(x=%relu, perm=%transpose_perm_0, name="transpose")
    %reduce_axes_0: (2,i32)* = const(val=[2, 3], name="reduce_axes_0")
    %reduce_keep_dims_0: (bool)* = const(val=False, name="reduce_keep_dims_0")
    %reduce: (1, 3, fp32) = reduce_mean(x=%transpose, axes=%reduce_axes_0, keep_dims=%reduce_keep_dims_0, name="reduce")
    %log: (1, 3, fp32) = log(x=%reduce, name="log")
    %add_0_x_0: (i32)* = const(val=1, name="add_0_x_0")
    %add_0_y_0: (i32)* = const(val=2, name="add_0_y_0")
    %add_0: (i32)* = add(x=%add_0_x_0, y=%add_0_y_0, name="add_0")
  } -> (%log)
}

Note that main is a function. A MIL program contains one or more functions. The mb.program decorator simply creates a MIL program with a single function main. The input to main is an fp32 tensor with the shape specified in the Python code.

Each function contains exactly one top-level block. This main function in this example contains block0. The return values of top-level block (%log) is the return value of the function.

A block is a sequence of operations. Here we have 10 operations in block0, where 5 of them are simply const operations that produces constants like the permutation order (%transpose_perm_0).

  • MIL automatically derives the shapes and types from all operations. For example, the reduce_mean operation returns the (1, 3, fp32) tensor.
  • MIL also performs value inference whenever possible. For example, the result of the add operation is (i32)*, where asterisk * represents compile-time known value.

You can convert your MIL program to CoreML for execution.

# Note: This example continues from the above code.

# Convert to CoreML format
model = ct.convert(prog)

# Make a prediction with CoreML
prediction = model.predict({
  'x': np.random.rand(1, 100, 100, 3).astype(np.float32),
})

Updated 3 months ago


Model Intermediate Language


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.