Device¶

Device represents an OAK camera. On all of our devices there’s a powerful vision processing unit (VPU), called Myriad X. The VPU is optimized for performing AI inference algorithms and for processing sensory inputs (eg. calculating stereo disparity from two cameras).

Device API¶

Device object represents an OAK device. When starting the device, you have to upload a Pipeline to it, which will get executed on the VPU. When you create the device in the code, firmware is uploaded together with the pipeline and other assets (such as NN blobs).

pipeline = depthai.Pipeline()

# Create nodes, configure them and link them together

# Upload the pipeline to the device
with depthai.Device(pipeline) as device:
  # Print Myriad X Id (MxID), USB speed, and available cameras on the device
  print('MxId:',device.getDeviceInfo().getMxId())
  print('USB speed:',device.getUsbSpeed())
  print('Connected cameras:',device.getConnectedCameras())

  # Input queue, to send message from the host to the device (you can receive the message on the device with XLinkIn)
  input_q = device.getInputQueue("input_name", maxSize=4, blocking=False)

  # Output queue, to receive message on the host from the device (you can send the message on the device with XLinkOut)
  output_q = device.getOutputQueue("output_name", maxSize=4, blocking=False)

  while True:
      # Get a message that came from the queue
      output_q.get() # Or output_q.tryGet() for non-blocking

      # Send a message to the device
      cfg = depthai.ImageManipConfig()
      input_q.send(cfg)

Multiple devices¶

If you want to use multiple devices on a host, check Multiple DepthAI per Host.

Device queues¶

After initializing the device, one has to initialize the input/output queues as well. These queues will be located on the host computer (in RAM).

outputQueue = device.getOutputQueue("output_name")
inputQueue = device.getInputQueue("input_name")

When you define an output queue, the device can push new messages to it at any point in time, and the host can read from it at any point in time. Usually, when the host is reading very fast from the queue, the queue (regardless of its size) will stay empty most of the time. But as we add things on the host side (additional processing, analysis, etc), it may happen that the device will be writing to the queue faster than the host can read from it. And then the messages in the queue will start to add up - and both maxSize and blocking flags determine the behavior of the queue in this case. You can set these flags with:

# When initializing the queue
queue = device.getOutputQueue(name="name", maxSize=5, blocking=False)

# Or afterwards
queue.setMaxSize(10)
queue.setBlocking(True)

Specifying arguments for `getOutputQueue` method¶

When obtaining the output queue (example code below), the maxSize and blocking arguments should be set depending on how the messages are intended to be used, where name is the name of the outputting stream.

Since queues are on the host computer, memory (RAM) usually isn’t that scarce. But if you are using a small SBC like RPI Zero, where there’s only 0.5GB RAM, you might need to specify max queue size as well.

with dai.Device(pipeline) as device:
  queueLeft = device.getOutputQueue(name="manip_left", maxSize=8, blocking=False)

If only the latest results are relevant and previous do not matter, one can set maxSize = 1 and blocking = False. That way only latest message will be kept (maxSize = 1) and it might also be overwritten in order to avoid waiting for the host to process every frame, thus providing only the latest data (blocking = False). However, if there are a lot of dropped/overwritten frames, because the host isn’t able to process them fast enough (eg. one-threaded environment which does some heavy computing), the maxSize could be set to a higher number, which would increase the queue size and reduce the number of dropped frames. Specifically, at 30 FPS, a new frame is received every ~33ms, so if your host is able to process a frame in that time, the maxSize could be set to 1, otherwise to 2 for processing times up to 66ms and so on.

If, however, there is a need to have some intervals of wait between retrieving messages, one could specify that differently. An example would be checking the results of DetectionNetwork for the last 1 second based on some other event, in which case one could set maxSize = 30 and blocking = False (assuming DetectionNetwork produces messages at ~30FPS).

The blocking = True option is mostly used when correct order of messages is needed. Two examples would be:

matching passthrough frames and their original frames (eg. full 4K frames and smaller preview frames that went into NN),
encoding (most prominently H264/H265 as frame drops can lead to artifacts).

Blocking behaviour¶

By default, queues are blocking and their size is 30, so when the device fills up a queue and when the limit is reached, any additional messages from the device will be blocked and the library will wait until it can add new messages to the queue. It will wait for the host to consume (eg. queue.get()) a message before putting a new one into the queue.

Note

After the host queue gets filled up, the XLinkOut.input queue on the device will start filling up. If that queue is set to blocking, other nodes that are sending messages to it will have to wait as well. This is a usual cause for a blocked pipeline, where one of the queues isn’t emptied in timely manner and the rest of the pipeline waits for it to be empty again.

Non-Blocking behaviour¶

Making the queue non-blocking will change the behavior in the situation described above - instead of waiting, the library will discard the oldest message and add the new one to the queue, and then continue its processing loop (so it won’t get blocked). maxSize determines the size of the queue and it also helps to control memory usage.

For example, if a message has 5MB of data, and the queue size is 30, this queue can effectively store up to 150MB of data in the memory on the host (the messages can also get really big, for instance, a single 4K NV12 encoded frame takes about ~12MB).

Some additional information¶

Decreasing the queue size to 1 and setting non-blocking behavior will effectively mean “I only want the latest packet from the queue”.
Queues are thread-safe - they can be accessed from any thread.
Queues are created such that each queue is its own thread which takes care of receiving, serializing/deserializing, and sending the messages forward (same for input/output queues).
The Device object isn’t fully thread-safe. Some RPC calls (eg. getLogLevel, setLogLevel, getDdrMemoryUsage) will get thread-safe once the mutex is set in place (right now there could be races).

Reference¶

class depthai.Device

Represents the DepthAI device with the methods to interact with it. Implements the host-side queues to connect with XLinkIn and XLinkOut nodes

class Config: Device specific configuration

addLogCallback(self: depthai.DeviceBase, callback: Callable[[dai::LogMessage], None]) → int

Add a callback for device logging. The callback will be called from a separate thread with the LogMessage being passed.

Parameter callback:: Callback to call whenever a log message arrives

Returns: Id which can be used to later remove the callback

close(self: depthai.DeviceBase) → None: Closes the connection to device. Better alternative is the usage of context manager: with depthai.Device(pipeline) as device:

flashCalibration(self: depthai.DeviceBase, calibrationDataHandler: depthai.CalibrationHandler) → bool

Stores the Calibration and Device information to the Device EEPROM

Parameter calibrationObj:: CalibrationHandler object which is loaded with calibration information.

Returns: true on successful flash, false on failure

static getAllAvailableDevices() → List[depthai.DeviceInfo]

Returns all connected devices

Returns: Vector of connected devices

static getAnyAvailableDevice(*args, **kwargs)

Overloaded function.

getAnyAvailableDevice(timeout: datetime.timedelta) -> Tuple[bool, depthai.DeviceInfo]

Waits for any available device with a timeout

Parameter timeout:: duration of time to wait for the any device

Returns: Tuple of bool and DeviceInfo. Bool specifies if device was found. DeviceInfo specifies the found device

getAnyAvailableDevice() -> Tuple[bool, depthai.DeviceInfo]

Gets any available device

Returns: Tuple of bool and DeviceInfo. Bool specifies if device was found. DeviceInfo specifies the found device

getCameraSensorNames(self: depthai.DeviceBase) → Dict[depthai.CameraBoardSocket, str]

Get sensor names for cameras that are connected to the device

Returns: Map/dictionary with camera sensor names, indexed by socket

getChipTemperature(self: depthai.DeviceBase) → depthai.ChipTemperature

Retrieves current chip temperature as measured by device

Returns: Temperature of various onboard sensors

getCmxMemoryUsage(self: depthai.DeviceBase) → depthai.MemoryInfo

Retrieves current CMX memory information from device

Returns: Used, remaining and total cmx memory

getConnectedCameras(self: depthai.DeviceBase) → List[depthai.CameraBoardSocket]

Get cameras that are connected to the device

Returns: Vector of connected cameras

getDdrMemoryUsage(self: depthai.DeviceBase) → depthai.MemoryInfo

Retrieves current DDR memory information from device

Returns: Used, remaining and total ddr memory

static getDeviceByMxId(mxId: str) → Tuple[bool, depthai.DeviceInfo]

Finds a device by MX ID. Example: 14442C10D13EABCE00

Parameter mxId:: MyraidX ID which uniquely specifies a device

Returns: Tuple of bool and DeviceInfo. Bool specifies if device was found. DeviceInfo specifies the found device

getDeviceInfo(self: depthai.DeviceBase) → depthai.DeviceInfo

Get the Device Info object o the device which is currently running

Returns: DeviceInfo of the current device in execution

static getEmbeddedDeviceBinary(*args, **kwargs)

Overloaded function.

getEmbeddedDeviceBinary(usb2Mode: bool, version: depthai.OpenVINO.Version = <Version.???: 5>) -> List[int]

Gets device firmware binary for a specific OpenVINO version

Parameter usb2Mode:: USB2 mode firmware
Parameter version:: Version of OpenVINO which firmware will support

Returns: Firmware binary

getEmbeddedDeviceBinary(config: depthai.Device.Config) -> List[int]

Gets device firmware binary for a specific configuration

Parameter config:: FW with applied configuration

Returns: Firmware binary

static getFirstAvailableDevice() → Tuple[bool, depthai.DeviceInfo]

Gets first available device. Device can be either in XLINK_UNBOOTED or XLINK_BOOTLOADER state

Returns: Tuple of bool and DeviceInfo. Bool specifies if device was found. DeviceInfo specifies the found device

getInputQueue(*args, **kwargs)

Overloaded function.

getInputQueue(self: depthai.Device, name: str) -> depthai.DataInputQueue

Gets an input queue corresponding to stream name. If it doesn’t exist it throws

Parameter name:: Queue/stream name, set in XLinkIn node

Returns: Smart pointer to DataInputQueue

getInputQueue(self: depthai.Device, name: str, maxSize: int, blocking: bool = True) -> depthai.DataInputQueue

Gets an input queue corresponding to stream name. If it doesn’t exist it throws. Also sets queue options

Parameter name:: Queue/stream name, set in XLinkOut node
Parameter maxSize:: Maximum number of messages in queue
Parameter blocking:: Queue behavior once full. True: blocking, false: overwriting of oldest messages. Default: true

Returns: Smart pointer to DataInputQueue

getInputQueueNames(self: depthai.Device) → List[str]

Get all available input queue names

Returns: Vector of input queue names

getLeonCssCpuUsage(self: depthai.DeviceBase) → depthai.CpuUsage

Retrieves average CSS Leon CPU usage

Returns: Average CPU usage and sampling duration

getLeonCssHeapUsage(self: depthai.DeviceBase) → depthai.MemoryInfo

Retrieves current CSS Leon CPU heap information from device

Returns: Used, remaining and total heap memory

getLeonMssCpuUsage(self: depthai.DeviceBase) → depthai.CpuUsage

Retrieves average MSS Leon CPU usage

Returns: Average CPU usage and sampling duration

getLeonMssHeapUsage(self: depthai.DeviceBase) → depthai.MemoryInfo

Retrieves current MSS Leon CPU heap information from device

Returns: Used, remaining and total heap memory

getLogLevel(self: depthai.DeviceBase) → depthai.LogLevel

Gets current logging severity level of the device.

Returns: Logging severity level

getLogOutputLevel(self: depthai.DeviceBase) → depthai.LogLevel

Gets logging level which decides printing level to standard output.

Returns: Standard output printing severity

getMxId(self: depthai.DeviceBase) → str

Get MxId of device

Returns: MxId of connected device

getOutputQueue(*args, **kwargs)

Overloaded function.

getOutputQueue(self: depthai.Device, name: str) -> depthai.DataOutputQueue

Gets an output queue corresponding to stream name. If it doesn’t exist it throws

Parameter name:: Queue/stream name, created by XLinkOut node

Returns: Smart pointer to DataOutputQueue

getOutputQueue(self: depthai.Device, name: str, maxSize: int, blocking: bool = True) -> depthai.DataOutputQueue

Gets a queue corresponding to stream name, if it exists, otherwise it throws. Also sets queue options

Parameter name:: Queue/stream name, set in XLinkOut node
Parameter maxSize:: Maximum number of messages in queue
Parameter blocking:: Queue behavior once full. True specifies blocking and false overwriting of oldest messages. Default: true

Returns: Smart pointer to DataOutputQueue

getOutputQueueNames(self: depthai.Device) → List[str]

Get all available output queue names

Returns: Vector of output queue names

getQueueEvent(*args, **kwargs)

Overloaded function.

getQueueEvent(self: depthai.Device, queueNames: List[str], timeout: datetime.timedelta = datetime.timedelta(days=-1, seconds=86399, microseconds=999999)) -> str

Gets or waits until any of specified queues has received a message

Parameter queueNames:: Names of queues for which to wait for
Parameter timeout:: Timeout after which return regardless. If negative then wait is indefinite. Default is -1

Returns: Queue name which received a message first

getQueueEvent(self: depthai.Device, queueName: str, timeout: datetime.timedelta = datetime.timedelta(days=-1, seconds=86399, microseconds=999999)) -> str

Gets or waits until specified queue has received a message

Parameter queueNames:: Name of queues for which to wait for
Parameter timeout:: Timeout after which return regardless. If negative then wait is indefinite. Default is -1

Returns: Queue name which received a message

getQueueEvent(self: depthai.Device, timeout: datetime.timedelta = datetime.timedelta(days=-1, seconds=86399, microseconds=999999)) -> str

Gets or waits until any queue has received a message

Parameter timeout:: Timeout after which return regardless. If negative then wait is indefinite. Default is -1

Returns: Queue name which received a message

getQueueEvents(*args, **kwargs)

Overloaded function.

getQueueEvents(self: depthai.Device, queueNames: List[str], maxNumEvents: int = 18446744073709551615, timeout: datetime.timedelta = datetime.timedelta(days=-1, seconds=86399, microseconds=999999)) -> List[str]

Gets or waits until any of specified queues has received a message

Parameter queueNames:: Names of queues for which to block
Parameter maxNumEvents:: Maximum number of events to remove from queue - Default is unlimited
Parameter timeout:: Timeout after which return regardless. If negative then wait is indefinite - Default is -1

Returns: Names of queues which received messages first

getQueueEvents(self: depthai.Device, queueName: str, maxNumEvents: int = 18446744073709551615, timeout: datetime.timedelta = datetime.timedelta(days=-1, seconds=86399, microseconds=999999)) -> List[str]

Gets or waits until specified queue has received a message

Parameter queueName:: Name of queues for which to wait for
Parameter maxNumEvents:: Maximum number of events to remove from queue. Default is unlimited
Parameter timeout:: Timeout after which return regardless. If negative then wait is indefinite. Default is -1

Returns: Names of queues which received messages first

getQueueEvents(self: depthai.Device, maxNumEvents: int = 18446744073709551615, timeout: datetime.timedelta = datetime.timedelta(days=-1, seconds=86399, microseconds=999999)) -> List[str]

Gets or waits until any queue has received a message

Parameter maxNumEvents:: Maximum number of events to remove from queue. Default is unlimited
Parameter timeout:: Timeout after which return regardless. If negative then wait is indefinite. Default is -1

Returns: Names of queues which received messages first

getSystemInformationLoggingRate(self: depthai.DeviceBase) → float

Gets current rate of system information logging (“info” severity) in Hz.

Returns: Logging rate in Hz

getUsbSpeed(self: depthai.DeviceBase) → depthai.UsbSpeed

Retrieves USB connection speed

Returns: USB connection speed of connected device if applicable. Unknown otherwise.

getXLinkChunkSize(self: depthai.DeviceBase) → int

Gets current XLink chunk size.

Returns: XLink chunk size in bytes

isClosed(self: depthai.DeviceBase) → bool: Check if the device is still connected`

isPipelineRunning(self: depthai.DeviceBase) → bool

Checks if devices pipeline is already running

Returns: True if running, false otherwise

readCalibration(self: depthai.DeviceBase) → depthai.CalibrationHandler

Fetches the EEPROM data from the device and loads it into CalibrationHandler object

Returns: The CalibrationHandler object containing the calibration currently flashed on device EEPROM

removeLogCallback(self: depthai.DeviceBase, callbackId: int) → bool

Removes a callback

Parameter callbackId:: Id of callback to be removed

Returns: True if callback was removed, false otherwise

setLogLevel(self: depthai.DeviceBase, level: depthai.LogLevel) → None

Sets the devices logging severity level. This level affects which logs are transferred from device to host.

Parameter level:: Logging severity

setLogOutputLevel(self: depthai.DeviceBase, level: depthai.LogLevel) → None

Sets logging level which decides printing level to standard output. If lower than setLogLevel, no messages will be printed

Parameter level:: Standard output printing severity

setSystemInformationLoggingRate(self: depthai.DeviceBase, rateHz: float) → None

Sets rate of system information logging (“info” severity). Default 1Hz If parameter is less or equal to zero, then system information logging will be disabled

Parameter rateHz:: Logging rate in Hz

setXLinkChunkSize(self: depthai.DeviceBase, sizeBytes: int) → None

Sets the chunk size for splitting device-sent XLink packets. A larger value could increase performance, and 0 disables chunking. A negative value is ignored. Device defaults are configured per protocol, currently 64*1024 for both USB and Ethernet.

Parameter sizeBytes:: XLink chunk size in bytes

startPipeline(*args, **kwargs)

Overloaded function.

startPipeline(self: depthai.DeviceBase) -> None

Starts the execution of the devices pipeline

Returns: True if pipeline started, false otherwise

startPipeline(self: depthai.DeviceBase, arg0: depthai.Pipeline) -> bool

Starts the execution of a given pipeline

Parameter pipeline:: OpenVINO version of the pipeline must match the one which the device was booted with.

Returns: True if pipeline started, false otherwise

class dai::Device : public dai::DeviceBase ¶

Represents the DepthAI device with the methods to interact with it. Implements the host-side queues to connect with XLinkIn and XLinkOut nodes

Public Functions

Device(const Pipeline &pipeline)¶