Camera stream of a real Intel Realsense reversed

Typically: "How do I... ", "How can I... " questions
Post Reply
Marcelbgit
Posts: 10
Joined: 01 Dec 2021, 12:27

Camera stream of a real Intel Realsense reversed

Post by Marcelbgit »

Dear CoppeliaSim community,

right now I am trying to subscribe a camera stream via ROS as you can see in the image below. However, as can be seen in the picture, the real camera (using ROS API) is inverted in contrast to the upper virtual camera (can also be seen in the writing of the pen). To correct this, I can't find any options in the vision sensor settings, etc. Do you have any idea how I can solve this? The stream outputs a "correctly aligned" image, as opening the "camera/color/image_raw" topic in RVIZ (ROS) confirms this. Additionally also my camera settings as further information. One more thing, is there any way to resize a 640x480 pixel camera resolution image to 128x128 via CoppeliaSim?

https://we.tl/t-uaqCbusx2T

I am very grateful for your help!
Cheers,
Marcel

fferri
Posts: 1193
Joined: 09 Sep 2013, 19:28

Re: Camera stream of a real Intel Realsense reversed

Post by fferri »

What can be seen in the image?
I see two very different pictures in the floating views. Is something missing?

Anyways, the reference frame for cameras and vision sensors in CoppeliaSim is set so that \(Y\) points upwards, and \(Z\) points in the direction the object is looking at, therefore the \(X\) axis has to be \(Y \times Z\) (to make a right-handed frame). This usually turns out to be the opposite if what is commonly found in other visualization systems such as WebGL and ROS, so you end up with a flipped image (a flip option in the object could be useful perhaps).

The image however can be transformed via sim.transformImage and scaled via sim.getScaledImage.

Marcelbgit
Posts: 10
Joined: 01 Dec 2021, 12:27

Re: Camera stream of a real Intel Realsense reversed

Post by Marcelbgit »

Thanks for your answer and your help!

The image was meant to show the difference between the image of the "real" and "virtual" camera. The real one was inverted what I didn't get in the past. Now I got the different frame approaches between ROS and CoppeliaSim.

Regarding the sim.transformImage. Does it only work for publishing images? It worked fine for the publishing part but not to flip the image in the subscribed image. I have tried to put it into the callback function to flip the subscribed image like the following lines without success.

Code: Select all

function imageMessage_callback(msg)
    local data,w,h=sim.getVisionSensorCharImage(passiveVisionSensor)
    sim.transformImage(data,{w,h},2)
    sim.setVisionSensorCharImage(passiveVisionSensor,msg.data)
end

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Camera stream of a real Intel Realsense reversed

Post by coppelia »

Hello,

in your code above, you should transform msg.data instead of data, e.g.:

Code: Select all

function imageMessage_callback(msg)
    sim.transformImage(msg.data,{msg.width,msg.height},2)
    sim.setVisionSensorCharImage(passiveVisionSensor,msg.data)
end
Cheers

Marcelbgit
Posts: 10
Joined: 01 Dec 2021, 12:27

Re: Camera stream of a real Intel Realsense reversed

Post by Marcelbgit »

thank you so much!
Cheers

Post Reply