Coordinate transformation from camera frame to world frame

[Uio96]

Hi there,

I'm trying to transform coordinates from camera frame to world frame. I first use the Lua script to get the transformation matrix between the camera and the world origin, e.g. Trans=sim.getObjectMatrix(cameraHandle,-1). Then I transform the point cloud obtained by the vision sensor in Matlab, e.g. ptCloud = pctransform(ptCloud, affine3d(Trans_Transpose)).

I think everything seems pretty straightforward and should be alright. However, the result seems not. There is an angle error between the processed point cloud and the ground truth. Does anybody have good ideas? Thank you very much.

[Uio96]

I just double-checked my code. The problem is with my point cloud input. The other stuff is good. Thanks anyway.

[jiuling]

Hello, excuse me, I want to know how to convert the pixel coordinates in the vision sensor to the world coordinate system, I tried to implement it using the python API, but it failed, and the problem mainly occurred in obtaining the internal parameters of the vision sensor.

[coppelia]

Hello,

have a look at the demo model in Models/components/sensors/Camera pixels to 3D positions.ttm, which does exactly that.

Cheers

[jiuling]

Thank you for your immediate reply, but the demo file is a 3D point cloud method, I have now obtained the RGB-D 3D coordinate data, can it be converted, and how to obtain the focal length and light point of the vision sensor? Because I now only need to simply grab a point, instead of doing a 3D reconstruction, I need color information.