vision sensor calibration
vision sensor calibration
hello, I have an intel real sense d435 camera in the real world. I want to do some surface grasping problems both in simulation and the real world. To simulate real-world noise, I plan to rotate and place the camera with some noise and do calibration, could you give me some hints on how to do this or is there any procedure that I should follow? I think the key point I need is just to map the image pixel position to a world coordinate system so that the robot knows where to go. Thank you!
Re: vision sensor calibration
Hello,
for each vision sensor image pixel, you can compute the corresponding 3D position in the world. For that, you need to read the vision sensor's depth map with sim.getVisionSensorDepthBuffer. Then you can select one specific pixel, or iterate over all pixels and read the depth value. From that, and the resolution of the sensor and the pose of the sensor, you can easily compute 3D positions. Have a look at the demo model Models/components/sensors/Blob to 3D position.ttm
Cheers
for each vision sensor image pixel, you can compute the corresponding 3D position in the world. For that, you need to read the vision sensor's depth map with sim.getVisionSensorDepthBuffer. Then you can select one specific pixel, or iterate over all pixels and read the depth value. From that, and the resolution of the sensor and the pose of the sensor, you can easily compute 3D positions. Have a look at the demo model Models/components/sensors/Blob to 3D position.ttm
Cheers
Re: vision sensor calibration
Thank you very much! What if the vision sensor is somewhat rotated? I remember seeing some papers say they do a projection to get their heightmap or something.coppelia wrote: ↑16 Feb 2022, 13:42 Hello,
for each vision sensor image pixel, you can compute the corresponding 3D position in the world. For that, you need to read the vision sensor's depth map with sim.getVisionSensorDepthBuffer. Then you can select one specific pixel, or iterate over all pixels and read the depth value. From that, and the resolution of the sensor and the pose of the sensor, you can easily compute 3D positions. Have a look at the demo model Models/components/sensors/Blob to 3D position.ttm
Cheers
Re: vision sensor calibration
That's exactly what is done in the model I mention. Did you try it?What if the vision sensor is somewhat rotated?
cheers
Re: vision sensor calibration
yes, I see the model and moved it in a new scene. It seems that it captures all red color, is this normal? What's the difference between this model and a regular vision sensor? I mean, there is a script with it, but how can I try the function of this model to really understand camera calibration in coppeliasim? Thanks again!