Access the camera in CoppeliaSim, and applied in software developed by Qt.

Typically: "How do I... ", "How can I... " questions
Post Reply
MaJiamu
Posts: 14
Joined: 19 Jan 2021, 03:52

Access the camera in CoppeliaSim, and applied in software developed by Qt.

Post by MaJiamu »

Hello, thanks for this great platform, it help me a lot during my everyday research work. Recently, I was wondering whether the camera information can obtained and applied in my software.
Here is my question:
How can I access one of the camera in a CoppeliaSim sence, and obtain the video streaming, then apply the video streaming in my software which is developed in Qt v5.9.8?
The purpose for this work is that I have already created a simple robot model in CoppeliaSim, and I also have a same physical robot. Besides I connected the simulation model with physical robot, that they can move in sync. Therefore, I want to control the physical robot with my software(Qt developed) and show the simulation model real-time state in my software at the same time.
Looking forward to your reply! Thanks a lot!

fferri
Posts: 1193
Joined: 09 Sep 2013, 19:28

Re: Access the camera in CoppeliaSim, and applied in software developed by Qt.

Post by fferri »

You can create a vision sensor with a pose similar to the default camera.
Then you can deliver the vision sensor image data (raw RGB data) to your application in various ways:

Remote API:
Your application can use the Remote API to call sim.getVisionSensorCharImage and get the image data, as often as needed.

ZeroMQ Publisher:
In a script (an embedded script, a customization script, ...) you can use the ZMQ plugin to setup a ZeroMQ publisher socket (see simZMQ.bind) and send the image (simZMQ.send) continuously; your application will use libzmq to connect to it with a ZeroMQ subscriber socket which will receive the data and display the image.

ROS Publisher:
If you already have experience with ROS, a similar approach as above but using a ROS publisher/subscriber may be simple as well.

Other:
There is an experimental add-on (you can find some discussion in recent forum messages) that would stream 3D information (object meshes, infos, poses, etc...) so that you can reconstruct a 3D visualization on the client side, but this is probably more work for you (in terms of functionality needed to be implemented in the client) than streaming a vision sensor image, unless your client can use HTML5 and JavaScript, in which case a client is already available for that.

coppelia
Site Admin
Posts: 10336
Joined: 14 Dec 2012, 00:25

Re: Access the camera in CoppeliaSim, and applied in software developed by Qt.

Post by coppelia »

Here the beta version and the experimental add-on. This would require very little effort if you can imbed a browser into your Qt application.

Cheers

Post Reply