Your first algorithm

The built-in algorithms cover the common jobs: standardizing uploads, quality checks, anonymization, hand pose, episodes and the LeRobot v3 export. When your data or your team needs something they don't do, you write your own algorithm and drop it into a workflow next to the built-in ones. Yours run the same way theirs do.

This page starts with the few ideas you need, then walks you through a real algorithm from a blank editor to a saved version.

One algorithm, one job

Every algorithm belongs to one of seven categories. The category says what the algorithm does, and it also fixes what it reads and what it writes, so you never declare the main ports yourself. Pick the category first and the rest follows.

Most algorithms run once per recording. A check or an enrich algorithm can instead run once per episode, in parallel, and curate and export algorithms run once per dataset version. Besides its main input, an algorithm may take extra features inputs, such as a calibration, and most categories may also write checks.

The eight data types in the Reads and Writes columns are what connect steps in a workflow: an output can only feed an input of the same type. The interface reference describes each of them.

Where you write

The simplest place is the browser: the editor gives you a working starter for each category, completes the SDK as you type and lets you try the code on a recording. This page uses it.

If you prefer your own editor, the command line does the same steps locally, and an AI assistant can drive them through the MCP tools. Code that isn't Python can ship as a container image; see Image algorithms.

Write a motion energy algorithm

You'll write an enrich algorithm that measures how much the first camera's picture changes from one frame to the next. Low values mark idle stretches, which a later step can cut or drop. You need a project with at least one processed recording; see Upload recordings if you don't have one yet.

1Open a new algorithm

In the sidebar, open Algorithms and choose New algorithm.

2Pick what it does

Under What does it do? choose Enrich, and under Where does it run? keep Recording. Choose Start writing, and the editor opens with a working starter.

3Paste the code

Replace the starter with the code below. It's the example packages/sdk-python/examples/v1/enrich-motion-energy/main.py from the SDK.

"""Enrich: the mean absolute difference between consecutive frames of the first camera, per frame.

In:  the canonical recording.
Out: one new per-frame feature "motion_energy"; the recording itself is not changed.
Why: idle stretches (the arm waits, the person stands still) show as low motion, so a later step can
     segment or drop them. A feature is how an algorithm adds knowledge without rewriting the recording.
"""

from dataclasses import dataclass

import numpy as np

from embodiflow import Context, Feature, FeatureWriter, Recording, algorithm, param


@dataclass
class Params:
    step: int = param(1, minimum=1, title="Frame step")


@algorithm.enrich(
    name="motion-energy",
    title="Motion energy",
    description="How much the first camera's picture changes from one frame to the next.",
    features=[Feature("motion_energy", type="scalar", per="frame", dtype="float32", unit="none")],
    params=Params,
)
def run(recording: Recording, out: FeatureWriter, ctx: Context[Params]) -> None:
    cam = recording.cameras[0]
    energy = np.zeros((recording.num_frames, 1), np.float32)
    prev = None
    for frame in cam.frames(step=ctx.params.step, gray=True):
        current = frame.image.astype(np.float32)
        if prev is not None:
            energy[frame.index, 0] = np.abs(current - prev).mean()
        prev = current
    out.per_frame("motion_energy", energy)

Its test is packages/sdk-python/examples/v1/enrich-motion-energy/test_main.py: it runs this code through the real runner on a small fixture and checks the values it writes. To start from it, run embodiflow algorithm init my-algorithm --category enrich, which writes main.py and test_main.py; then run embodiflow algorithm test my-algorithm (needs Python with pytest and the SDK).

There are four parts to notice. Params holds the settings a workflow may change; here step is the frame step, at least 1. The @algorithm.enrich(...) decorator is the interface: it declares one feature, motion_energy, with one number per frame. The run function does the work, and its arguments are bound by name, so take only the ones you need, in any order. Finally, out.per_frame(...) writes the feature, exactly one row per frame.

4Check the interface

The Interface panel on the right updates as you type. Check that Features lists motion_energy and that no line in the code is marked with a problem.

The platform reads the decorator without running your code, which is why decorator arguments and parameter defaults must be literals: strings, numbers, lists and the SDK constructors such as Feature and param.

5Try it on a recording

In Test run, choose a Project and a Recording. The Window is the first 30 seconds by default; choose Whole recording to run on all of it. Choose Run test, and a plot of motion_energy appears when it finishes.

A test run uses the real runtime on real data but saves nothing, and the window is at most 600 seconds. If a run can't start, the editor says why, and you can still save the algorithm.

6Save a version

Choose Save version at the top right. The first version is 1.0.0, and a saved version never changes. Later, when you save a change, the platform suggests the next version number for you: a change to the interface, such as the ports, the declared features or a parameter type, needs a new major version.

Put it in a workflow

A saved version can now be a step in any workflow of your team. Here's how to add motion energy to a copy of a standard workflow:

1. Open Workflows, find Robot standard or Human egocentric standard, choose Duplicate to edit, pick your project and choose Create copy.
2. In the editor, type motion-energy into Search algorithms and add it. Team algorithms carry the Team label. The editor connects its recording input to the nearest output that fits.
3. Select the step, open Connections and choose the output of the Anonymize step as its recording. Every step that reads the recording after standardizing must read the anonymized one, otherwise the workflow is refused with WORKFLOW_PRIVACY_BARRIER.
4. Connect the step's features output to the Output node, so the feature is merged into the saved recording.
5. When Problems is empty, choose Test on sample, pick a recording, set Window in seconds to something short such as 30 and choose Run test. This test saves nothing either.
6. Choose Publish, confirm, and bind the workflow to the project so new uploads use it.

Binding a workflow to a project needs the project permission to manage workflows (workflow:manage), and starting a run needs workflow:run. Use your algorithm in a workflow covers the editor in depth.

The same steps on the command line

If you'd rather work locally, these four commands do the same thing. init writes a starter main.py and its test, recordings pull downloads a processed recording to try it on, and save uploads the result as a new version.

embodiflow algorithm init motion-energy --category enrich
embodiflow recordings pull <recording-id> --project <team>/<project> --as recording --out ./rec
embodiflow algorithm test motion-energy --input recording=./rec --window 30s
embodiflow algorithm save motion-energy

What you have now

The saved version shows up in the algorithm library with the Team label, and a workflow refers to it as team/motion-energy@1. Its feature is stored under a name that belongs to your team, so it never collides with a built-in feature.

If a line is marked with ALGORITHM_FILE_INVALID, a decorator argument isn't a literal or a rule is broken; the message on that line says which. If the save is refused with FEATURE_ROW_COUNT_MISMATCH, write exactly one row per frame. Other codes are explained in Troubleshooting.

Next

• Enrich algorithms, to go further with features
• Use your algorithm in a workflow
• Interface reference