Inference

Using an already trained model to produce an output from a new input.

For local AI, the model format, hardware support, memory requirements and runtime all affect what you can use.

Projects connected to this idea

Make room for a model on your own machine.

Editorial reportLocal AIDesktop / terminal

Use pretrained models through a shared library and model ecosystem.

Project notesBuild appsPython library

Run language-model inference across Metal, CUDA and Vulkan backends.

Project notesLocal AIInference engine