How to implement a (more realistic) depth sensor with shadows?

[Uli_v]

Hi,

I saw that V-REP supports rendering with shadows. Can this also work for depth sensors?

I would like to have a depth sensor (based on a structured light, like Intel RealSense SR300) that produces shadows. Shadows (missing depth information) are caused because the location of the IR projector and IR camera on the sensor is different. This causes that some surfaces in the IR camera are visible that do not have IR pattern (I call these "shadows"). In the depth image, these pixels won't have any depth information (e.g. depth value = 0). How can I easily implement a more realistic depth sensor with shadows in V-REP? Thanks.

Uli

[coppelia]

Hello Uli,

ok, I see what you are trying to achieve...
Basically, if there was a possibility to render a scene with the OpenGL3 mode, but by selecting only one light source and rendering a binary image (1=lit, 0=shadow), then one could modify the depth map appropriately (i.e. by clearing depth map areas where the RGB image is 0). Is that correct?
Currently this is not possible. Probably that could be implemented with not too much effort. Also, in next release, the vision sensor filters will be gone. Instead, there will be a vision callback that allows to handle image processing in a much more flexible way than currently.

Cheers

[Uli_v]

Yes, the description your approach is correct. I had something like that in mind. Could that be added in the next release? When is the next release due? Thanks.