Overview of sink types

Operator attributes and sink types

There are many forms of information you can get about traffic data passing through a certain operator. You may want statistical data about speed or acceleration of all passing traffic, or maybe a speed heatmap. From the standpoint of the server’s HTTP response, the sink’s type denotes the contents of each sink’s data object – the names and data types of its member variables. It also denotes additional parameters that may be a part of sink output requests. The attribute that the sink is attached to decides the values of these members. The following table shows a list of all attributes that all operators have along with their associated sink type.

Filter attribute	Sink type
Object count	Value
Level of service	Value
Speed	Statistical value
Stationary time	Statistical value
Duration of occurrence	Statistical value
Trajectory list	Table
Trajectory view	Trajectory view
Category distribution	Distribution
Color distribution	Distribution
Speed map	Heatmap
Acceleration map	Heatmap
Occupancy map	Heatmap
OD matrix	OD matrix
Raw trajectory data	Raw trajectories

Light vs. heavy data

The sinks are categorized as light data or heavy data sinks. This reflects how large their data responses tend to be in general. To prevent sink data responses from growing out of proportion, it's only possible to request outputs from one heavy data sink at a time. In other words, when requesting for a heavy data sink output, no other light data or heavy data sink output can be requested.

Details of sink types

In this section, you will find detailed descriptions of each sink type. The properties data_validity and object_count are common for all sink types. They are both explained in the previous article.

Value

JSON type identifier: value
History supported: Yes
Output heaviness: Light data
Additional request parameters: None
Response's data object contents: A single numerical value
Example of response's data contents:

"data": {
  "data_validity": "ok",
  "object_count": 57,
  "value": 57
}

Statistical value

JSON type identifier: statistical_value
History supported: Yes
Output heaviness: Light data
Additional request parameters: None
Response's data object contents: A collection of key/value pairs. The values are always a single number. The keys designate: Minimum value, maximum value, median, average, mode, variance, and size (number of samples). Size is currently redundant and has the same value as object_count.
Example of response's data contents:

"data": {
  "data_validity": "ok",
  "object_count": 624,
  "minimum": 0.0,
  "maximum": 51.6,
  "median":  41.1,
  "average": 40.2,
  "size": 624
}

Table

Tables are currently limited to hold up to 10,000 records of closed trajectories. There's no limit for additional open trajectories. Sending all of them at once can be demanding on the hardware and bandwidth. That's why each response will hold a maximum of 100 table records. In order to obtain further rows, you need to include further parameters in your data request. By default, tables are ordered by the "Trajectory start" column in ascending order.

JSON type identifier: table
History supported: No
Output heaviness: Heavy data
Additional request parameters:

number_of_rows—maximum number of rows you want to receive in one page. The actual received number can be lower if there are not enough rows in the table, or if you provided a number greater than 100.
first_row_index—the row index at which the returned page shall start
order_by—the name of column by which you want the page to be ordered. You can see column names in the response example below in the header array. The table is ordered before a page is extracted.
order—either ascending, or descending

(All of these parameters are optional.)

Example request body:

{
  "sequence_number": "11", 
  "sinks": [ 
    { 
      "id": 0, 
      "number_of_rows": 100, 
      "first_row_index": 200, 
      "order_by": "Maximum speed", 
      "order": "descending"  
    } 
  ] 
}

Response's data object contents:

table_size—total number of rows in the table; currently redundant and synonymous with object_count
number_of_rows—number of rows in the current page
header—an array of strings designating the names of table columns
units—the units and time formats used in each column
data_start—the value of the column of the first row in the returned page, always as a string. If the table is ordered by "Trajectory start", then this property is instead named data_start_time.
data_end—the value of the column of the last row in the returned page, always as a string. If the table is ordered by "Trajectory start", then this property is instead named data_end_time.
order—either ascending, or descending
data—an array of arrays, each containing one table row

The Category fields will have one of the following values: car, light, heavy, bus, motorcycle, bicycle, pedestrian, unknown.

Example of response's data contents:

"data": {
  "data_validity": "ok",
  "object_count": 16,
  "data_end_time": "1618991959204",
  "data_start_time": "1618990159105",
  "first_row_index": 0,
  "number_of_rows": 16,
  "order": "ascending",
  "table_size": 16,
  "header": [ "ID", "License plate", "Category", "Color", 
              "Trajectory start", "Trajectory end", "Average speed",
              "Minimum speed", "Maximum speed", "Duration of occurence",
              "Stationary duration" ],
  "units": [ "", "", "", "", "ms", "ms", "km/h", "km/h", 
             "km/h", "s", "s" ],
  "data": [
    [ "197", "Undefined", "bicycle", "undefined", "1618990434162",
      "1618990442603", "42.67", "39.63", "92.23", "8.44", "8.11" ],
    [ "363", "Undefined", "car", "black", "1618990681890",
      "1618990705947", "48.95", "0.03", "175.54", "24.05", "15.05" ],
    // ...
  ]
}

Distribution

JSON type identifier: distribution
History supported: Yes
Output heaviness: Light data
Additional request parameters: None
Response's data object contents: A collection of key/value pairs, where keys are histogram categories (classes), and values are single numerical values. There are eight possible categories: car, light, heavy, bus, motorcycle, bicycle, pedestrian, unknown.
Example of response's data contents for a category distribution :

"data": {
   "data_validity": "ok",
   "object_count": 130,
   "categories": [
      {
         "category": "bicycle",
         "count": 13
      },
      {
         "category": "bus",
         "count": 3
      },
      {
         "category": "car",
         "count": 74
      },
      {
         "category": "light",
         "count": 11
      },
      {
         "category": "pedestrian",
         "count": 29
      },
      {
         "category": "unknown",
         "count": 0
      }
   ]
}

Example of response’s data contents for a color distribution:

"data": {
   "data_validity": "ok",
   "object_count": 14,
   "colors": [
      {
         "color": "black",
         "count": 4
      },
      {
         "color": "blue",
         "count": 0
      },
      {
         "color": "brown",
         "count": 0
      },
      {
         "color": "red",
         "count": 1
      },
      {
         "color": "silver",
         "count": 2
      },
      {
         "color": "undefined",
         "count": 7
      }
   ]
}

Trajectory view

JSON type identifier: trajectory_view
History supported: No
Output heaviness: Heavy data
Additional request parameters: None
Response's data object contents:

rendered—denotes whether the outputs are PNG images, or raw data. This will always be set to true.
rendered_trajectories—an array of objects, which contain:
- data—a string containing a Base64 representation of a PNG image. This image is a visualization of all selected trajectories of the corresponding category over a transparent background.
- category—a string denoting a category whose trajectories are rendered in the corresponding image. The possible values and the corresponding rendered color are:
  - unknown—#000000
  - car—#7CFF2E
  - light (Light truck)—#00C800
  - heavy (Heavy truck)—#007D00
  - bus—#07FBFF
  - motorcycle—#0000FF
  - bicycle—#6400FF
  - pedestrian—#FF00FF
  - e-cart—#195FFF

If there are no objects to be rendered for a given category, the whole object will be omitted in the rendered_trajectories array. E.g. in the example below, categories bus, motorcycle, and e-cart have been omitted.

Example of response's data contents:

"data": {
  "data_validity": "ok",
  "object_count": 17,
  "timestamp": "1619697039715",
  "rendered": true,
  "rendered_trajectories": [
    {
      "category": "unknown",
      "data": "iVBORw0KGgoAAAANSUhEUgAA", // ...
    },
    {
      "category": "car",
      "data": "iVBORw0KGgoAAAANSUhEUgAA", // ...
    },
    {
      "category": "light",
      "data": "iVBORw0KGgoAAAANSUhEUgAA", // ...
    },
    {
      "category": "heavy",
      "data": "iVBORw0KGgoAAAANSUhEUgAA", // ...
    },
    {
      "category": "bicycle",
      "data": "iVBORw0KGgoAAAANSUhEUgAA", // ...
    },
    {
      "category": "pedestrian",
      "data": "iVBORw0KGgoAAAANSUhEUgAA", // ...
    }
  ]
}

Example of all trajectory view images overlaid over a video snapshot:

Comments:

It might take a long time for a block to respond to a trajectory view sink data request. Due to this, all trajectory view sink data requests must be sent separately—without requests for any other sink data in the same message.

Heatmap

Decode the string from Base64.
Unzip with the ZLIB library's flate algorithm.
Parse the first four 32-bit integers representing the number of rows, number of columns, OpenCV element type (always 32-bit float), and number of channels (always 1) respectively.
Parse the 1D array of 32-bit floats representing the matrix data.

The matrices are represented as a raw 1D array of 32-bit floats, but the matrices are of the same total size as the video image (i.e. the array's size is image_height*image_width). Each element is a value that relates to the corresponding pixel in the original image—elements of the sum_data matrix represent a sum of a value selected during the sink's creation (e. g. current speed) and elements of the count_data matrix represent the number of samples.

For example for an image with resolution 1920x1080 and Speed map type of heatmap, the second element of sum_data would contain a sum of speeds of all trajectories that crossed the second pixel in the first row, and the second element of the count_data matrix would contain the number of those trajectories. Therefore by dividing the sum value by the count value, you can obtain the average value for the corresponding pixel in the input image.

For another example, if the image has a resolution of 1920x1080 px, then the matrices would be represented as 1D arrays containing 1920*1080 = 2073600 elements. Element 68372 of the array would correspond to the image pixel in a row floor(68372/1920) = 35 and column 68372%1920 = 1172 where % represents the modulus operation.

The matrices are quite large, so each of them is compressed with the ZIP (deflate) algorithm of the ZLIB library and encoded in Base64 before being placed into JSON response and sent.

Example of response’s data contents:

"data": {
  "object_count": 420, 
  "data_validity": "ok",
  "map_type": "Heatmap",              
  "count_data": "TWFuIGlzIGRpc3RpbmXNoZWQsIG5vdCB5IGhpcyByZWF", // ...
  "sum_data": "WQsIG5vdCBvbmx5IGJ5IGhpcyByZWd1aXNoZWQsIG55as" // ...
}

Example of a heatmap converted to a grayscale image:

For illustration purposes, here is an example of a processed heatmap overlaid over a corresponding video snapshot:

Comments:

It might take a long time for a block to respond to a heatmap sink data request. Due to this, all heatmap sink data requests must be sent separately—without requests for any other sink data in the same message.

Gridmap

Technically, gridmap is a subtype of the heatmap, but its output is a bit different, so it'll be explained here in a separate section. See the explanation of a gridmap widget to learn what it is. It's the same with a gridmap sink.

JSON type identifier: heatmap
History supported: No
Output heaviness: Heavy data
Additional request parameters: None
Response's data object contents: A map_type property, which can have either the Heatmap, or Gridmap value. This section will concern the Gridmap type—Heatmap is described in the previous one. Then there is tile_size, which denotes how many pixels are aggregated in one grid tile in each dimension. For example, a tile_size of 50 means that each grid aggregates 50x50=2500 pixels. Moreover, for an image of size 1920x1080, the grid has ceil(1920/50)=39 columns and ceil(1080/50)=22 rows. (ceil() is an operation of rounding up to the nearest integer.) Finally, the response's data object contains another data property, which contains a JSON string with the encoded grid. A quick guide to decode it:

Decode the string from Base64.
Unzip with the ZLIB library's flate algorithm.
Parse the array of 32-bit floats representing the matrix data.

Tiles are stored in a row-major order. Each tile comprises of 5 32-bit floats: Minimum, maximum, average, median, and object count. These statistical values are tied to the operator attribute that's been set for the sink: speed, acceleration, object count etc.

Example of response’s data contents:

"data": {
  "object_count": 420, 
  "data_validity": "ok",
  "map_type": "Gridmap",
  "tile_size": 50,
  "data": "eJzsnHlQFVcWhzsKQUFxQ1xxgR" // ...
}

Example of a gridmap displayed as a widget in FLOW Insights:

OD matrix

OD matrix functionality is explained in a dedicated how-to article.

Due to a bug, OD matrix sinks are currently not included in sink listings. You can find them in widget listings instead: /cubes/{cube_id}/analytics/{analytic_id}/widgets

JSON type identifier: od_matrix
History supported: Yes (in all time modes), the number of records is limited to 1000
Output heaviness: Light data
Additional request parameters: None
Response's data object contents:

origins—an array of entry gates (user name and ID)
destinations—an array of exit gates (user name and ID)
turning_movements—rows of turning counts

The data have a row-oriented format where the rows represent entry/origin gates. The columns are exit/destination gates. The order of the elements in the headers (origins/destinations) corresponds to the order of the data in the turning_movements section. Maximum size is 8x8.

Example of response’s data contents:

(the size of the OD matrix is 1x2 i.e. 1 entry and 2 exits)

"data": {
    "data_validity": "ok",
    "origins": [             // OD origins/entries
        {
           "id": 1,          // id of the entry gate 
           "name": "In"      // name of the entry gate 
        }
    ],
    "destinations": [         // OD destinations/exits
        {
            "id": 3,          // id of the exit gate (generated by FLOW)
            "name": "Out 1"   // name of the gate (defined by user)
        },  
        {
            "id": 4,
            "name": "Out 2"
        }
    ],
    "turning_movements": [   // data is entry/row-oriented
         {
             "category": "car",
             "data": [
                 [ 0, 5025]  // In -> Out 1: 0; In -> Out 2: 5025
             ]
         },
         {
             "category": "pedestrian",
             "data": [
                 [ 0, 19]  // In -> Out 1: 0; In -> Out 2: 19
             ]
         },
         // ...the rest of all supported categories 
      ]
   ]
}

Raw trajectories

This sink shows you the evolution of traffic objects' trajectories.

JSON type identifier: raw_trajectories
History supported: No
Output heaviness: Heavy data
Additional request parameters: None
Response's data object contents:

{
    "data_validity": "ok",
    "object_count": 1329,
    "units": {
        "duration": "ms",
        "map_positions": "m",
        "map_speeds": "m/s",
        "sensor_positions": "px",
        "timestamp": "ms",
        "timestamps": "ms",
        "wgs84_positions": "deg"
    },
    "trajectories": [{
        "color": "white",
        "duration": 39142,
        "id": "8",
        "license_plate": "Undefined",
        "timestamp": "1650529144653",
        "category": "car",
        "state_data": {
            "map_positions": [
                [615951.5625, 5453988.5],
                [615951.8125, 5453992],
                // ...
            ],
            "map_speeds": [
                0.09022672474384308, 
                0.021549025550484657, 
                // ...
            ],
            "sensor_positions": [
                [916, 729],
                [921, 726],
                // ...
            ],
            "timestamps": [
                0, 
                368,
                // ... 
            ],
            "wgs84_positions": [
                [16.59258449695335, 49.22769412627738],
                [16.592588941067113, 49.22772555017891],
                // ...
            ]
        }       
    }, {
        "color": "silver",
        "duration": 14783,
        "id": "9",
        "license_plate": "Undefined",
        "timestamp": "1650529144653",
        "category": "car",
        "state_data": {
            "map_positions": [
                [615953.375, 5453987.5],
                [615953.625, 5453990.5],
                // ...
            ],
            "map_speeds": [
                0.02577352710068226, 
                0.474840372800827,
                // ... 
            ],
            "sensor_positions": [
                [584, 1059],
                [583, 1057],
                // ...
            ],
            "timestamps": [
                0, 
                4204, 
                // ...
            ],
            "wgs84_positions": [
                [16.592609092302204, 49.22768479133908],
                [16.592613391888513, 49.22771171934915],
                // ...
            ]
        }       
    },
    // ...
    ]
}

Description:

units - contains information about which values use which measurement units
trajectories - information about the traffic objects outputted by the sink
- color - the general color of the traffic object
- duration - how long the traffic object has been in the scene
- id - the traffic object's unique identifier within the analytics
- license_plate - the license plate contents of the traffic object. If none have been detected, the value is "Undefined"
- timestamp - the timestamp of the moment when this object has been detected for the first time
- type - the traffic object's category
- state_data - data about the traffic object's properties that change with time
  - map_positions - Only available in georegistered analytics. This array contains the traffic object's positions within its corresponding UTM zone. The first element of the position array is the x coordinate, the second element is the y coordinate.
  - map_speeds - Only available in georegistered analytics. This array contains the traffic object's speeds in real-world measurement units.
  - timestamps - the amount of time that has passed between the moment when this object has been detected for the first time and the moment that the other state_data properties correspond to.
  - wgs84_positions - the position of the traffic object within the WGS 84 coordinate system. Only available in georegistered analytics. The first array element is the longitude, the second is latitude. They're expressed as positive or negative numbers. Positive latitude is above the equator (N), and negative latitude is below the equator (S). Positive longitude is east of the prime meridian, while negative longitude is west of the prime meridian (a north-south line that runs through a point in England).
  - sensor_positions - the position of the traffic object within the sensor's coordinate space. The first array element is the x coordinate, the second is the y coordinate. E.g. in a camera image, the coordinates [0,0] correspond to the upper left corner.

Trajectory file sink

This is a special type of sink that lets users of FLOW Insights export observed trajectories into a .tlgx file. It's not possible to interact with it through the REST API—requesting data of this sink returns an empty result.

JSON type identifier: file

Measurement units guide

What units do different sinks and attributes give you?

Sink type	Attribute	Unit
Value	Object count	Pure number (unitless)
	Level of service	Pure number (unitless)
Statistical value	Duration of occurrence	[m/s]
	Stationary duration	[m/s]
	Speed	[kpx/s] (kilopixels per second) if using non-georegistered footage and [m/s] for georegistered footage
Table	Trajectory list	Table has a descriptive header. All time data is in [s] except Trajectory start and Trajectory ends which have the [datetime] format. Speed in [kpx/h] if not georegistered or [km/h] when georegistered.
Distribution	Color	Pure number (unitless)
	Category	Pure number (unitless)
Heatmap	Speed	Heatmap has a set of 2 values for each pixel – `sum_data` value and a `count_data` value. `sum_data` is the sum of speeds recorded in a given pixel and uses [kpx/s] for non-georegistered footage and [m/s] for georegistered footage `count_data` is the number of objects that passed through that pixel and is, therefore, a unitless value
	Acceleration	Same as with speed, but the units are [kpx/s^2] and [m/s^2].
	Occupancy	`sum_data` is the number of objects that passed through the pixel `count_data` is equal to 1 if at least one object has been detected there, otherwise, the value is 0 Both of these values are unitless.
Raw trajectories	Raw trajectory data	See the `units` property inside the `data` response.

What next?

Learn more about sinks' data history.

Need more help or have some questions? Contact us!