Vision sensor

[pluckz]

Hello, I want to use a vision sensor to extract some coordinates from a object to move the end-effector of the robot to it. I need to detect semi circles in the image and I feel like the algorithms in VREP are not complex enough for this operation. If I'm correct, is there a way I export the image for the api and do the image processing outside of the simulator?

Thank you

[fferri]

You can write a plugin in C++ providing one or more custom Lua commands, and use whatever image processing algorithm (e.g. OpenCV, TensorFlow, ...) to extract the info you need.

[pluckz]

Thanks! Is there any example of this kind that I can look for? I'm using remote API C++ already, can I just use opencv in the api?

Thank you again!

[fferri]

The Image plugin uses OpenCV 3 to perform simple operations and drawing on images, and already can read/write to/from vision sensors as well as capture devices (cameras).

I think it is a good starting point to extend with further OpenCV functions.

To implement new Lua commands remember to edit also the callbacks.xml file accordingly.