kinect depth data question

Typically: "How do I... ", "How can I... " questions
Post Reply
lyz
Posts: 57
Joined: 07 Mar 2019, 07:34

kinect depth data question

Post by lyz »

Hi,
In the past, I wrote the code of the camera publishing depth according to the forum, but now I suddenly found a bug that caused me unable to use the camera sensor. Here's the question:
kinect_depth.lua

Code: Select all

function sysCall_init()
end

function sysCall_vision(inData)
    -- callback function automatically added for backward compatibility
    -- (vision sensor have no filters anymore, but rather a callback function where image processing can be performed)
    local retVal={}
    retVal.trigger=false
    retVal.packedPackets={}
    simVision.sensorDepthMapToWorkImg(inData.handle)
    simVision.verticalFlipWorkImg(inData.handle)
    simVision.intensityScaleOnWorkImg(inData.handle,1.000000,0.000000,false)
    simVision.workImgToSensorDepthMap(inData.handle,false)
    return retVal
end
rviz Results: https://share.weiyun.com/5jjK2hf

The depth image is correct, but the point cloud shown by rviz is wrong.

Then, I modify the kinect_depth code:

Code: Select all

function sysCall_vision(inData)
    -- callback function automatically added for backward compatibility
    -- (vision sensor have no filters anymore, but rather a callback function where image processing can be performed)
    local retVal={}
    retVal.trigger=false
    retVal.packedPackets={}
    simVision.sensorImgToWorkImg(inData.handle)
    --simVision.verticalFlipWorkImg(inData.handle)
    simVision.workImgToSensorImg(inData.handle)
    return retVal
end
result: https://share.weiyun.com/5l3HvNT
The point cloud is opposite to the left and the right, and the depth image is opposite to the up and the down.

scene image: https://share.weiyun.com/5FAnIOs

In any case, the point cloud image shown by rviz is wrong. What is the problem? It feels like a bug in the software?

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: kinect depth data question

Post by coppelia »

You have two locations in your scene where you read the vision sensor. One location you have shown. The other location is where you send the image to ROS. So if you need to flip the image or modify the image for RVIZ, then do it just before sending it.

Cheers

lyz
Posts: 57
Joined: 07 Mar 2019, 07:34

Re: kinect depth data question

Post by lyz »

Hi coppelia,
I'm very glad to see your answer.
First, I understand your idea. I used this code

Code: Select all

 simVision.verticalFlipWorkImg(inData.handle)
to rotate vision image data to publish them to ros and the final displayed image is the same as the image in simulation environment. The problem I have now is that I publish the depth data '/depth/image_raw' to ros, and its' image displayed right in rviz(kinect.rviz Image). However, the DepthCloud in rviz generated by depth data '/depth/image_raw' transformation does not correspond to the actual data(kinect.rviz DepthCloud). And the image of physical camera displayed in rviz does not have such problems. Is the inconsistent question a bug? You can test it yourself. this is my scene:
kinect.ttt & kinect.rviz & kinect.png: https://1drv.ms/u/s!AtF26DukM1J6hAtIjRZ ... 2?e=DKw8Tn

lyz
Posts: 57
Joined: 07 Mar 2019, 07:34

Re: kinect depth data question

Post by lyz »

Hi,
Thank you anyway. I have solved this problem by many tests. I convert work image to sensor image.Then rotate camera coordinate system. Finally reconstruct the getVisionSensorDepthBuffer buff data to solve it. Now rviz and image are all right.

I find if you want to use workImgToSensorImg at last , verticalFlipWorkImg and horizontalFlipWorkImg functions seem to be a problem. And workImgToSensorDepthMap function has not this problem.

Post Reply