relationship of perspective and orthogonal sensors

Typically: "How do I... ", "How can I... " questions
Post Reply
mohammadhosseink
Posts: 20
Joined: 11 Sep 2022, 06:48

relationship of perspective and orthogonal sensors

Post by mohammadhosseink »

Hello,

consider a point in the environment of coppeliasim. This point's coordinates are represented by a pixel in an image from a vision sensor. The pixel in an image from an orthogonal sensor, (x1,y1). The pixel of the same point in a perspective image is (x2,y2). Is there a mathematical or coppelisim way to relate (x1,y1) to (x2,y2)?

cheers,

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: relationship of perspective and orthogonal sensors

Post by coppelia »

Have a look at the demo model in modelLibrary/components/sensors/Camera pixels to 3D positions.ttm: in there you will see the relationship between world coordinates and image coordinates, for a vision sensor in perspective projection mode. The sensor offers 2 calculation methods. Method2 is more intuitive, but slowlier. This is the relevant code:

Code: Select all

function sysCall_sensing()
    if not builtInCalculation then
        -- with method 2:
        local m=sim.getObjectMatrix(sensor,sim.handle_world)
        sim.addDrawingObjectItem(pointContainer,nil)
        local depthMap=sim.getVisionSensorDepth(sensor,1)
        depthMap=sim.unpackFloatTable(depthMap)
        local cols
        if pointsHaveColor then
            cols=sim.getVisionSensorImg(sensor)
            cols=sim.unpackUInt8Table(cols)
        end
        for y=0,resY-1,1 do
            for x=0,resX-1,1 do
                local ind=y*resY+x+1
                local d=depthMap[ind]
                local coord={0,0,d}
                coord[1]=d*math.tan(xAngle*0.5)*(0.5-(x/(resX-1)))/0.5
                coord[2]=d*math.tan(yAngle*0.5)*((y/(resY-1))-0.5)/0.5
                coord=sim.multiplyVector(m,coord)
                if pointsHaveColor then
                    coord[4]=cols[3*(ind-1)+1]
                    coord[5]=cols[3*(ind-1)+2]
                    coord[6]=cols[3*(ind-1)+3]
                end
                sim.addDrawingObjectItem(pointContainer,coord)
            end
        end
    end
end
For a vision sensor in orthogonal mode, things are easier.

Cheers

mohammadhosseink
Posts: 20
Joined: 11 Sep 2022, 06:48

Re: relationship of perspective and orthogonal sensors

Post by mohammadhosseink »

I see. So this is it right?

Code: Select all

coord[1]=d*math.tan(xAngle*0.5)*(0.5-(x/(resX-1)))/0.5
coord[2]=d*math.tan(yAngle*0.5)*((y/(resY-1))-0.5)/0.5
could you please elaborate on parameters

Code: Select all

xAngle, yAngle, resX, resY
?
thanks

mohammadhosseink
Posts: 20
Joined: 11 Sep 2022, 06:48

Re: relationship of perspective and orthogonal sensors

Post by mohammadhosseink »

Hello Coppeliasim,

I'm sorry for the inconvenience but I really can't understand the d, xAngle, and yAngle parameters in the answer you provided for me. Could you please explain them?

Thanks

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: relationship of perspective and orthogonal sensors

Post by coppelia »

xAngle: that is the horizontal view angle
yAngle: the vertical view angle
d: the distance to the object corresponding to that specific pixel, in meter

The idea is to iterate over all pixels, and via simple trigonometric calculations, extract for each pixel a 3D position in the world.

Cheers

Post Reply