Annotate data

Label your data so a model can learn from it. For a vision model, label images with tags for classification or bounding boxes for object detection. For a robot policy, tag each recorded sequence with the task it demonstrates. Organizing your data into a dataset makes bulk annotation and training workflows easier.

Tag images for classification

Tags are labels that apply to an entire image. Use them when you are building a classification model – for example, labeling images as “good-part” or “defective-part”.

  1. In the DATA tab, click an image to open it in the detail view.
  2. On the right side panel, find the Tags section.
  3. Click the + button next to Tags.
  4. Type a tag name (for example, good-part) and press Enter.
  5. The tag is saved immediately. Repeat for each image.

To tag multiple images at once, use an SDK.

async def main():
    viam_client = await connect()
    data_client = viam_client.data_client

    binary_data_ids = ["binary-data-id-1", "binary-data-id-2"]

    await data_client.add_tags_to_binary_data_by_ids(
        tags=["good-part"],
        binary_ids=binary_data_ids,
    )
    print("Tags added.")

    viam_client.close()

binaryIDs := []string{"binary-data-id-1", "binary-data-id-2"}

err = dataClient.AddTagsToBinaryDataByIDs(
    ctx,
    []string{"good-part"},
    binaryIDs,
)
if err != nil {
    logger.Fatal(err)
}
fmt.Println("Tags added.")

You can get binary data IDs by querying for images using the data client’s binary_data_by_filter method, which returns objects that include the binary data ID.

Choose tag names that are clear, consistent, and descriptive. Use lowercase with hyphens (for example, good-part, defective-part, no-part). Avoid vague names like type1 or other.

Draw bounding boxes for object detection

Bounding boxes mark the location of specific objects within an image. Use them when you are building an object detection model – for example, detecting packages on a conveyor belt.

  1. Open your dataset and click an image to open the detail view.
  2. In the side panel, click the Actions tab.
  3. Click Annotate to enter annotation mode.
  4. Select or create a label (for example, package).
  5. Hold Cmd (macOS) or Ctrl (Windows/Linux) and click-and-drag on the image to draw a rectangle around the object.
  6. The bounding box appears with your selected label. Adjust the box edges by dragging them if needed.
  7. Repeat for every object in the image that should be detected.
  8. Move to the next image and repeat.
async def main():
    viam_client = await connect()
    data_client = viam_client.data_client

    bbox_id = await data_client.add_bounding_box_to_image_by_id(
        binary_id="binary-data-id-1",
        label="package",
        x_min_normalized=0.15,
        y_min_normalized=0.20,
        x_max_normalized=0.85,
        y_max_normalized=0.90,
    )
    print(f"Added bounding box: {bbox_id}")

    viam_client.close()

bboxID, err := dataClient.AddBoundingBoxToImageByID(
    ctx,
    "binary-data-id-1",
    "package",
    0.15, // x_min_normalized
    0.20, // y_min_normalized
    0.85, // x_max_normalized
    0.90, // y_max_normalized
)
if err != nil {
    logger.Fatal(err)
}
fmt.Printf("Added bounding box: %s\n", bboxID)

Draw tight bounding boxes that closely fit the object. Do not include excessive background. If an image contains multiple objects, draw a separate bounding box for each one.

From code, coordinates are normalized 0.0-1.0, where (0.0, 0.0) is the top-left corner and (1.0, 1.0) is the bottom-right corner.

Tag sequences

A sequence’s tags label the whole sequence, such as the task a robot policy demonstration shows. You usually set them when you record, through the capture-control module’s start_sequence command or CreateSequence. See Sequences.

If you didn’t tag sequences while recording, add a tag from code. The Viam app and the CLI can’t edit a sequence.

The update replaces the sequence’s tags, so read the current tags first and pass them back with the new one.

sequence = await data_client.get_sequence("<sequence-id>")
await data_client.update_sequence(
    id="<sequence-id>",
    sequence_tags=list(sequence.sequence_tags) + ["pick-up-cup"],
)

The SDK builds the field mask from the arguments you pass, so you don’t set it yourself.

sequence, err := dataClient.GetSequence(ctx, "<sequence-id>")
if err != nil {
    logger.Fatal(err)
}
err = dataClient.UpdateSequence(ctx, "<sequence-id>", &app.UpdateSequenceOptions{
    SequenceTags: append(slices.Clone(sequence.SequenceTags), "pick-up-cup"),
    FieldMask:    []string{"sequence_tags"},
})
if err != nil {
    logger.Fatal(err)
}

The field mask is required. It names the fields to change.

const sequence = await dataClient.getSequence("<sequence-id>");
await dataClient.updateSequence(
  "<sequence-id>",
  undefined, // resources
  [...sequence.sequenceTags, "pick-up-cup"],
  undefined, // startTime
  undefined, // endTime
  { paths: ["sequence_tags"] },
);

The field mask is required. It names the fields to change.

To confirm, get the sequence again: its sequence_tags include the new tag. Use the same tag for every demonstration of the same task. To check a sequence dataset’s tags, see Verify dataset quality.

Troubleshooting

Bounding boxes not saving
  • Annotation mode not active. You must click the Annotate button before drawing boxes. Without annotation mode, click-and-drag does nothing.
  • No label selected. You must select or create a label before drawing a bounding box. If no label is selected, the box will not persist.
  • Box too small. Very small bounding boxes (a few pixels) may not register. Draw boxes that clearly encompass the object.
Tags applied to wrong images
  • Remove incorrect tags. In the Viam app, click the image, find the tag in the Tags section, and click the x next to it to remove it.
  • Programmatically remove tags. Use remove_tags_from_binary_data_by_ids in the SDK to remove tags from specific images.

What’s next