ML SDK Model Converter

The ML SDK Model Converter is a command line application that translate TOSA ML
Models to VGF files. A VGF file is a model file containing SPIR-V™ modules and
constants that are required to execute the model through the ML extensions for
Vulkan®. The ML SDK Model Converter supports several different TOSA encodings
as inputs:

• TOSA FlatBuffers
• TOSA MLIR bytecode
• TOSA MLIR textual format

The ML SDK Model Converter can also produce TOSA FlatBuffers from its input,
without performing any conversion.

You can also use the ML SDK Model Converter to check that all tensors specified
in the input model are ranked and have fixed, non-dynamic shapes. If a dynamic
tensor is detected, the program will exit with an error.

The suggested workflow for this tool as part of the ML SDK for Vulkan® is:

1. A TOSA MLIR file is converted to a VGF file using the ML SDK Model Converter
   (this project).
2. The generated VGF file and VGF library VGF Dump Tool is used to create a
   JSON scenario template file. The template file is edited with the correct
   filenames and paths.
3. Using the generated VGF file and scenario file, the ML SDK Scenario Runner
   then dispatches the contained SPIR-V™ modules to the ML extensions for
   Vulkan®.

Cloning the repository

To clone the ML SDK Model Converter as a stand-alone repository, you can use
regular git clone commands. However, for better management of dependencies and
to ensure everything is placed in the appropriate directories, we recommend
using the git-repo tool to clone the repository as part of the ML SDK for
Vulkan® suite. Repo tool.

For a minimal build and to initialize only the ML SDK Model Converter and its
dependencies, run:

repo init -u https://github.com/arm/ai-ml-sdk-manifest -g model-converter

Alternatively, to initialize the repo structure for the entire ML SDK for
Vulkan®, including the ML SDK Model Converter, run:

repo init -u https://github.com/arm/ai-ml-sdk-manifest -g all

After the repo is initialized, you can fetch the contents with:

repo sync --no-clone-bundle

Cloning on Windows®

To ensure nested submodules do not exceed the maximum long path length, you must
enable long paths on Windows®, and you must clone close to the root directory
or use a symlink. Make sure to use Git for Windows.

Using PowerShell:

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1
git config --global core.longpaths true
git --version # Ensure you are using Git for Windows, for example 2.50.1.windows.1
git clone <git-repo-tool-url>
python <path-to-git-repo>\git-repo\repo init -u <manifest-url> -g all
python <path-to-git-repo>\git-repo\repo sync --no-clone-bundle

Using Git Bash:

cmd.exe "/c reg.exe add \"HKLM\System\CurrentControlSet\Control\FileSystem"" /v LongPathsEnabled /t REG_DWORD /d 1 /f"
git config --global core.longpaths true
git --version # Ensure you are using the Git for Windows, for example 2.50.1.windows.1
git clone <git-repo-tool-url>
python <path-to-git-repo>/git-repo/repo init -u <manifest-url> -g all
python <path-to-git-repo>/git-repo/repo sync --no-clone-bundle

Building the ML SDK Model Converter from source

The build system must have:

• CMake 3.25 or later.
• C/C++ 17 compiler: GCC, or optionally Clang on Linux and MSVC on Windows®.
• Python 3.10 or later. Required python libraries for building are listed in
  tooling-requirements.txt.
• Ninja 1.10 or later.

The following dependencies are also needed:

• Argument Parser for Modern C++.
• LLVM.
• TOSA Tools.

For the preferred dependency versions see the manifest file.

Building with the script

Arm® provides a python build script to make build configuration options easily
discoverable. When the script is run from a git-repo manifest checkout, the
script uses default paths and does not require any additional arguments.
Otherwise the paths to the dependencies must be specified.

To build on Linux, run:

SDK_PATH="path/to/sdk"
python3 ${SDK_PATH}/sw/model-converter/scripts/build.py -j $(nproc) \
    --vgf-lib-path ${SDK_PATH}/sw/vgf-lib \
    --flatbuffers-path ${SDK_PATH}/dependencies/flatbuffers \
    --argparse-path ${SDK_PATH}/dependencies/argparse \
    --tosa-tools-path ${SDK_PATH}/dependencies/tosa-tools \
    --external-llvm ${SDK_PATH}/dependencies/llvm-project

To build on Windows®, run:

$env:SDK_PATH="path\to\sdk"
$cores = [System.Environment]::ProcessorCount
python3 "$env:SDK_PATH\sw\model-converter\scripts\build.py" -j $cores `
    --vgf-lib-path "$env:SDK_PATH\sw\vgf-lib" `
    --flatbuffers-path "$env:SDK_PATH\dependencies\flatbuffers" `
    --argparse-path "$env:SDK_PATH\dependencies\argparse" `
    --tosa-tools-path "$env:SDK_PATH\dependencies\tosa-tools" `
    --external-llvm "$env:SDK_PATH\dependencies\llvm-project"

If the components are in their default locations, it is not necessary to specify
the --vgf-lib-path, --flatbuffers-path, --argparse-path,
--tosa-tools-path, and --external-llvm options.

Tests can be enabled and run with --test and linting by --lint. To enable
tests and documentation building python dependencies must be installed:

pip install -r requirements.txt
pip install -r tooling-requirements.txt

The documentation can be built with --doc. To build the documentation, sphinx
and doxygen must be installed on the machine.

You can install the project build artifacts into a specified location by passing
the option --install with the required path.

To create an archive containing the build artifacts, pass the --package-type
option with an archive type such as zip or tgz. Use --package-dir to
choose where the archive is written; by default, packages are written to the
build directory.

For more information, see the help output:

python3 scripts/build.py --help

Usage

To generate a VGF file, run:

./build/model-converter --input ${INPUT_TOSA} --output ${OUTPUT_VGF}

To generate a TOSA flatbuffer file, run:

./build/model-converter --tosa-flatbuffer --input ${INPUT_TOSA} --output ${OUTPUT_TOSA_FB}

To lower TOSA custom operations through the Arm.ExperimentalMLOperations CALL extended instruction, provide a domain to Opcode mapping:

./build/model-converter --input ${INPUT_TOSA} --output ${OUTPUT_VGF} \
    --custom-op-domain-to-opcode com.example.accel:42

The com.arm.bespoke domain is reserved for Opcode 0 and must be enabled with --enable-bespoke:

./build/model-converter --input ${INPUT_TOSA} --output ${OUTPUT_VGF} --enable-bespoke

For more information, see the help output:

./build/model-converter --help

PyPI

The ML SDK Model Converter is available on PyPI as the ai-ml-sdk-model-converter package.

Install the published package:

pip install ai-ml-sdk-model-converter

To build and install the host executable from an ML SDK checkout, run from this
repository root:

pip install .

Known Limitations

• Usage of the patches/llvm.patch file is temporary until the required changes
  can be upstreamed to main LLVM Project

License

The ML SDK Model Converter is distributed under the software licenses in
LICENSES directory.

Trademark notice

Arm® is a registered trademarks of Arm Limited (or its subsidiaries) in the US
and/or elsewhere.

Khronos®, Vulkan® and SPIR-V™ are registered trademarks of the
Khronos® Group.