## coremltools

Use Core ML to integrate machine learning models into your app. Core ML provides a unified representation for all models. Your app uses Core ML APIs and user data to make predictions, and to train or fine-tune models, all on the user’s device.

Core ML optimizes on-device performance by leveraging the CPU, GPU, and Neural Engine while minimizing its memory footprint and power consumption. Running a model strictly on the user’s device removes any need for a network connection, which helps keep the user’s data private and your app responsive.

# Model Intermediate Language

Model Intermediate Language (MIL) is an intermediate representation (IR) that powers the converter. Behind the scenes, the converter takes a computation graph from a source framework (e.g., TensorFlow and PyTorch) and builds a MIL program. The converter performs a series of optimization over the MIL program, and eventually generates the ML Model from the optimized MIL program.

Notice that MIL is multi-level. This means that instead of a single set of operations, MIL has multiple sets of operations and optimization that may be invoked, depending on factors like the source frameworks (TensorFlow vs PyTorch).

For example, when converting a TensorFlow 1 program, you may use operations and optimization tailored for TensorFlow 1 in the MIL::TF1 "dialect" to facilitate the conversion process. All programs get converted to the MIL::Core dialect.

## Building MIL Program with Builder

You can easily construct a MIL program using the Python builder. In this example, we create a simple program that takes an input that is (1, 100, 100, 3) and creates a few layers of the neural network. This program contains all the information needed to describe the neural network.

# import builder
from coremltools.converters.mil import Builder as mb

# Input to MIL program is a list of tensors. Here we have one input with
# shape (1, 100, 100, 3) and implicit dtype == fp32
@mb.program(input_specs=[mb.TensorSpec(shape=(1, 100, 100, 3)),])
def prog(x):
# MIL operation takes named inputs (instead of positional inputs).
# Here name argument is optional.
x = mb.relu(x=x, name='relu')
x = mb.transpose(x=x, perm=[0, 3, 1, 2], name='transpose')
x = mb.reduce_mean(x=x, axes=[2, 3], keep_dims=False, name='reduce')
x = mb.log(x=x, name='log')
return x

print(prog)


The output of the following program looks a bit like this.

main(%x: (1, 100, 100, 3, fp32)) {
block0() {
%relu: (1, 100, 100, 3, fp32) = relu(x=%x, name="relu")
%transpose_perm_0: (4,i32)* = const(val=[0, 3, 1, 2], name="transpose_perm_0")
%transpose: (1, 3, 100, 100, fp32) = transpose(x=%relu, perm=%transpose_perm_0, name="transpose")
%reduce_axes_0: (2,i32)* = const(val=[2, 3], name="reduce_axes_0")
%reduce_keep_dims_0: (bool)* = const(val=False, name="reduce_keep_dims_0")
%reduce: (1, 3, fp32) = reduce_mean(x=%transpose, axes=%reduce_axes_0, keep_dims=%reduce_keep_dims_0, name="reduce")
%log: (1, 3, fp32) = log(x=%reduce, name="log")
} -> (%log)
}


Note that main is a function. A MIL program contains one or more functions. The mb.program decorator simply creates a MIL program with a single function main. The input to main is an fp32 tensor with the shape specified in the Python code.

Each function contains exactly one top-level block. This main function in this example contains block0. The return values of top-level block (%log) is the return value of the function.

A block is a sequence of operations. Here we have 10 operations in block0, where 5 of them are simply const operations that produces constants like the permutation order (%transpose_perm_0).

• MIL automatically derives the shapes and types from all operations. For example, the reduce_mean operation returns the (1, 3, fp32) tensor.
• MIL also performs value inference whenever possible. For example, the result of the add operation is (i32)*, where asterisk * represents compile-time known value.

You can convert your MIL program to CoreML for execution.

# Note: This example continues from the above code.

# Convert to CoreML format
model = ct.convert(prog)

# Make a prediction with CoreML
prediction = model.predict({
'x': np.random.rand(1, 100, 100, 3).astype(np.float32),
})


Updated 3 months ago

# Model Intermediate Language

### Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.