## Synchronize with external ImageSensor

Typically: "How do I... ", "How can I... " questions
SkySurfer1
Posts: 10
Joined: 14 Mar 2016, 14:34

### Synchronize with external ImageSensor

Hi,

I use the remote API to modify the output of an VisionSensor externally and pass it back to a second VisionSensor which is set to externally triggered. What I try now is to compare the blob detection rate between the original (gt) and the modified image.

The setup is like this:
-+ BlobDetectionCamera
+- BlobDetectionCamera_body
+- BlobDetectionCamera_camera (streams to external)
+- BlobDetectionCamera_camera1 (external triggered)

Both cameras show their image in a floating view. camera1 has a notable delay but also shows the blob detection frame round the parts.

I created a litle external application for testing purposes which run synchronous, fetches the image from camera, pass it back to camera1 and triggers the next step.

BlobDetectionCamera has a non-threaded child script:

Code: Select all

if (sim_call_type==sim.syscb_init) then
camera=sim.getObjectHandle('blobDetectionCamera_camera1')
cameraGT=sim.getObjectHandle('blobDetectionCamera_camera')
end

if (sim_call_type==sim.syscb_actuation) then
t=sim.getSimulationTime()
imageAnalysisFrequency=sim.getScriptSimulationParameter(sim.getScriptAssociatedWithObject(baseObject),'imageAnalysisFrequency')
if (t-lastImageAnalysisTime>1/imageAnalysisFrequency) then
lastImageAnalysisTime=t

sim.handleVisionSensor(cameraGT)

sim.handleVisionSensor(camera)
result,t0,t1=sim.readVisionSensor(camera)  --> here result is -1

if (result==-1) then
print(sim.getLastError())
end

end
end

Problem is that readVisionSensor(camera) returns -1 and getLastError returns nil.
Docs said that readVisionSensor may return -1 when handleVisionSensor is not called but it is.

So I suppose it is a timing issue? Is there a way to synchronize it?

coppelia
Posts: 7396
Joined: 14 Dec 2012, 00:25

### Re: Synchronize with external ImageSensor

Hello,

does the console output any error message? Are you sure the camera object is actually not a camera, but a vision sensor? Make sure that the vision sensor camera is explicitely handled.

Can you prepare a minimal test case that illustrates the bug? Best would be to have a self-contained, very simple scene.

Cheers

SkySurfer1
Posts: 10
Joined: 14 Mar 2016, 14:34

### Re: Synchronize with external ImageSensor

Attached https://mega.nz/#!5Lwz0ALa!7X-a96tG3tKX ... JcD1deV0UM there is a scene and an python script to trigger it external.
The scene is modified in the manner described above.

Load the scene and run the script.

coppelia
Posts: 7396
Joined: 14 Dec 2012, 00:25

### Re: Synchronize with external ImageSensor

Ok, now I understand.

In your case, you are calling sim.handleVisionSensor(camera): this normally does 3 things:
• clear previous computed filter data
• compute filter data on the new image
In your case however, that vision sensor is flagged as external input: because of that, sim.handleVisionSensor(camera) will simply clear previouscomputer filter data and return.

When you call sim.setVisionSensorImage, then filter data will be computed. But above erases them again.
So sim.handleVisionSensor(camera) should be removed.

Cheers

SkySurfer1
Posts: 10
Joined: 14 Mar 2016, 14:34

### Re: Synchronize with external ImageSensor

Thx that fixed the issue.
Maybe I missed that in the documentation.
Otherwise a little hint in the API of handleVisionSensor or setVisionSensorImage would be great.