Vision sensors rendering performance issue

[avena_robotics]

Hello,

we are writing plugin which is responsible to publish data to ROS2. ROS2 part is ready and it's working. The problem we have is with performance of rendering data in vision sensors.

We use 2 RGBs, 2 depths and 2 point clouds (which are obtained using simVision plugin). We turned on Explicit handling for RGBs and Depths inside vision sensor properties because when they were running all the time, Coppelia was lagging.

When we want to take data from vision sensors inside C++ plugin, function simHandleVisionSensor(sim_handle_all, nullptr, nullptr) is called and it takes a lot of time. Is there any option to speed it up.

Reducing resolution is not possible because we want to have exactly resolution we intend to have on real cameras. One option is to use other renderer but we have no experience with choosing the proper one. Another option is to render data for every sensor and read the data from them on separate threads. But is it possible? We don't know if Coppelia allows running Regular API functions on separate threads without causing any problems.

Best regards,
Avena

[coppelia]

Hello Avena,

The correct approach as you mention is to enabled the explicit handling flag for all vision sensors, and to handle them, i.e. call sim.handleVisionSensor on them explicitely, only when needed.

Additionally, each sensor can ignore the data that it doesn't require (e.g. Ignore depth info (faster), in the vision sensor dialog), so that there won't be any transfer for that data between GPU and CPU.

From your plugin, you should only access the API from the simulation or main thread.

You can also write your own, faster renderer, based on this plugin if required: simExtExternalRenderer

Cheers

[avena_robotics]

Thanks for answer,

Are default renderers using GPU on default?

Cheers

[coppelia]

Default renderers are using GPU, yes.

Cheers