Getting Depth from Vision Sensor

[Raveena]

I am trying to get the x, y and z coordinates of some objects using a vision sensor. I used the blob detection filter and got the blob position x and y to get the x and z coordinates of the object. Then I used the depth buffer to get the depth. But it is constantly giving 1 as the value, which, according to the help manual, refers to the far clipping plane. How do I get the correct depth? I am using V-REP PRO EDU 3.6.2 version. Here's the code for vision sensor :

Code: Select all

function sysCall_threadmain()
    VisionSensor = sim.getObjectHandle("Vision_sensor")
    while sim.getSimulationState()~=sim.simulation_advancing_abouttostop do
        result,pack1,pack2 = sim.handleVisionSensor(VisionSensor)
        result,pack1,pack2 = sim.readVisionSensor(VisionSensor)
        n = pack2[1]
        vCnt = pack2[2]
        if n == 1 then
            xval=(pack2[5]*0.9)-1.55
            zval=(pack2[6]*0.9)+0.13
            depth=sim.getVisionSensorDepthBuffer(VisionSensor,pack2[5],pack2[6])
            yval=(0.01+(depth[1]*0.99))-0.9
            print("\nX : "..xval.."  Y : "..yval.."  Z : "..zval)
        else
            if n == 2 then
                x1val=(pack2[5]*0.9)-1.55
                z1val=(pack2[6]*0.9)+0.13
                depth1=sim.getVisionSensorDepthBuffer(VisionSensor,pack2[5],pack2[6])
                y1val=(0.01+(depth1[1]*0.99))-0.9
                i = 3+vCnt+2
                x2val=(pack2[i]*0.9)-1.55
                z2val=(pack2[i+1]*0.9)+0.13
                depth2=sim.getVisionSensorDepthBuffer(VisionSensor,pack2[i],pack2[i+1])
                y2val=(0.01+(depth2[1]*0.99))-0.9
                print("\nBLOB 1 >> X : "..x1val.."  Y : "..y1val.."  Z : "..z1val)
                print("\nBLOB 2 >> X : "..x2val.."  Y : "..y2val.."  Z : "..z2val)
            else
                print("\nNO WHITE BOX DETECTED")
            end
        end
    end
end

depth[1] is always 1 for some reason. How do I fix this?

Here's the scene : https://github.com/Tejal-19/simbotix/bl ... hDepth.ttt

[coppelia]

Hello,

I think you are retrieving a large portion of the depth buffer. You should specify the size of the portion you want to retrieve, something like:

Code: Select all

depth=sim.getVisionSensorDepthBuffer(VisionSensor,pack2[5],pack2[6],1,1)

Additionally, the blob detection filter retrieves the x/y position of the blob with values between 0 and 1 (not related to the resolution of the vision sensor). So you'd have to multiply that with the resolution in X and Y.

Something similar for the depth values: they are between 0 and 1. If however you you specifiy:

Code: Select all

sim.getVisionSensorDepthBuffer(VisionSensor+sim.handleflag_depthbuffermeters,...)

then the values are directly in meters.

Cheers

[Raveena]

It worked! Thank you!

[coppelia]

I recommend that you switch to CoppeliaSim V4.1.0 (beta available here), or at least to version CoppeliaSim V4.0.0, since there are a few changes how the vision sensors are handled (i.e. the filters of the vision sensors). When doing so, you will notice that the system automatically attaches a customization script to your vision sensor. Open that script, copy the sysCall_vision function, and paste it into your threaded child script attached to the same vision sensor. Then you can remove that customization script.

The reason you need to make that change is specific to your scene where you explicitely call sim.handleVisionSensor from a threaded child script (then the vision callback also needs to be sitting in the same script).

Cheers