How to map from 3D coordinates to 2D coordinates in image

[Uio96]

Hi there,

After I get the 3D coordinates of the object in the vision sensor frame, I want to extract their corresponding 2D coordinates in the output image.

I try to use some codes like

Code: Select all

cam_intrinsics = np.hstack((self.cam_intrinsics, np.zeros((self.cam_intrinsics.shape[0], 1), dtype=self.cam_intrinsics.dtype)))
p = np.matmul(cam_intrinsics, camera_vertices)
p[0] = p[0] / (p[2])
p[1] = p[1] / (p[2])

p[0] and p[1] would be the output 2D coordinates (X/Y)

However, there remains an offset from the ground truth to my calculations, I'm not sure what is wrong with my thoughts.

I also found something similar to my question, however, I did not understand the answer completely.
viewtopic.php?t=5435

If you need to x/y position of the target object in the vision sensor's reference frame, then simply do:

Code: Select all

transformMatrix = simGetObjectMatrix(shapeObj, frontSen)
xRel=transformMatrix[4]
yRel=transformMatrix[8]
zRel=transformMatrix[12]

The reference frame of a vision sensor has Y pointing up, x pointing left, and z pointing in the viewing direction.

Cheers

[Uio96]

I finally solve the problem. It was my mistake. Previously, I set up camera's intrinsics matrix but did not update the vision sensor.