Table of contents

Segments API

The Segments API allows you to search through segments that are automatically generated during the harvest process for video files. Segments contain enrichment data for frames extracted at key moments, scene changes and periodic intervals throughout the video timeline.

Search segments

POST /api/data/segments/search

The method post body must include filter parameters. The query, limit, page, only and sort_fields parameters are Optional

  • filters.exact_terms - (array) Array containing multiple field-value pairs to search on segments
  • query - (string) Optional Defaults to empty query string
  • limit - (integer) Optional Value used to limit the number of results returned. Defaults to 20
  • page - (integer) Optional Current index of all pages. Defaults to 0
  • only - (array) Optional Array containing fields limiting results to certain fields
  • sort_fields.field - (string) Optional Field used for sorting
  • sort_fields.asc - (bool) Optional Set to true to sort ascending

A successful response will return with a 200 OK status. The response body will include the original body of the POST including all parameters used in search. The response body also contains additional result information.

  • total_hits - (integer) Number of segments return that match your query
  • results - (array) List of results
  • results.result._id - (string) Segment id

Get all segments for item


This example shows how to search for all segments for item id, 8a503972d0454f0bd0c33880177da283. Results are sorted by time ascending.


POST /api/data/segments/search


  "query": "",
  "limit": 20,
  "page": 0,
  "types": [
  "fields": null,
  "only": null,
  "filters": {
    "exact_terms": [
        "field": "item_id",
        "value": "c7af513ad0a2f2571282640d06fc7651"
    "multi_terms": null,
    "ranges": null,
    "exists": null
  "sort_fields": [
      "field": "time",
      "ascending": false
  "disable_aggr": false,
  "total_hits": 88,
  "results": [...]

Get segment by id

This example shows how to retrieve data for a single segment of data that corresponds to a frame within the video.

GET /api/data/segments/{id}
  • {id} - (string) ID of the segment


Below is an example of how to retrieve segment data using a segment id, 1f12b7212ddadb42c7aea47e6ad57d2c, from the response to the POST to /segments/search.


GET /api/data/segments/1f12b7212ddadb42c7aea47e6ad57d2c


  • _id - (string) Segment id
  • item_id - (string) Id of the item that the segment belongs
  • summary.tags - (array) Descriptive tags of objects found within the item
  • summary.tags[].name - (string) The tag string
  • summary.tags[].confidence - (number) How confident the system is that the tag is accurate. Zero indicates unknown confidence.
  • summary.tags[].source - (string) Source denotes the cognitive service used to detect tag
  • summary.people - (array) People recognized within the item
  • summary.people[].id - (string) Persons unique id
  • summary.people[].name - (string) Person’s name, if not Unknown.
  • summary.people[].face_attributes - (object) Age, gender, emotions and other attributes associated with person
  • summary.people[].face_landmarks - (object) Facial landmarks associated with person
  • summary.people[].face_rectangle - (object) Face rectangle contains top, left, height and width to describe bounding box of person’s face
  • summary.people[].thumbnail_path - (string) Thumbnail or frame which person’s face is found
  • summary.people[].source - (string) Source denotes the cognitive service used to detect person
  • summary.description.text - (string) Description of item that contains the highest confidence
  • summary.description.source - (string) Source of description with the highest confidence
  • summary.description.confidence - (float64) Confidence of description
  • summary.descriptions - (array) All descriptive phrases retrieved from all cognitive services for the item
  • summary.descriptions[].text - (string) Descriptive phrase for the item
  • summary.descriptions[].confidence - (number) How confident the system is that the description is accurate. Zero indicates unknown confidence.
  • summary.descriptions[].source - (string) Source denotes the cognitive service used to generate descriptive text
  • summary.logos - (array) Logos detected within the item
  • summary.logos[].name - (string) Name of the logo found
  • summary.logos[].size - (string) Size category of the logo found (tiny, small, medium, large)
  • summary.logos[].logoquadrilateral - (array) Contains the coordinates of the quadrilateral that encloses the logo
  • summary.logos[].confidence - (number) How confident the system is that the logo is accurate. Zero indicates unknown confidence.
  • summary.logos[].source - (string) Source denotes the cognitive service used to generate logo information
  • summary.locations - (array) Landmarks and locations detected within the item
  • summary.locations.feature_collection - (array) Feature collection of all feature objects in GeoJson format
  • summary.locations.feature_collection.type - (string) Feature type
  • summary.locations.feature_collection.geometry - (object) Feature collection of all feature objects
  • summary.locations.feature_collection.geometry.coordinates - (array) Latitude and longitude coordinates
  • summary.locations.feature_collection.geometry.type - (string) Type of geometry
  • - (object) Custom properties for feature
  • - (string) Name of location
  • - (number) Confidence of location
  • - (string) Source denotes the cognitive service used to determine location information
  • summary.ocr - (array) Summary of OCR text extracted from the item
  • summary.ocr[].text - (string) OCR text
  • summary.ocr[].source - (string) Source denotes the cognitive service used to gather OCR data
  • summary.nsfw - (object) Object indicating the result of adult or NSFW content detection
  • summary.nsfw.source - (string) A key indicating the source of the decision
  • summary.nsfw.nsfw - (bool) Whether the item is considered not suitable for work or not
  • summary.nsfw.nsfw_score - (number) A value from 0 to 1 indicating how unsuitable the item is. 0 means safe, 1 means unsafe.
  • time - (float) Time in seconds indicating the location of the frame within video timeline

Only fields

The only fields limit results to certain fields.

This documentation is generated from the latest version of GrayMeta Platform. For documentation relevant to your own deployed version, please use the documentation inside the application.