Synchronize with external ImageSensor

Typically: "How do I... ", "How can I... " questions
Post Reply
SkySurfer1
Posts: 10
Joined: 14 Mar 2016, 14:34

Synchronize with external ImageSensor

Post by SkySurfer1 » 07 May 2019, 15:15

Hi,

I use the remote API to modify the output of an VisionSensor externally and pass it back to a second VisionSensor which is set to externally triggered. What I try now is to compare the blob detection rate between the original (gt) and the modified image.

The setup is like this:
-+ BlobDetectionCamera
+- BlobDetectionCamera_body
+- BlobDetectionCamera_camera (streams to external)
+- BlobDetectionCamera_camera1 (external triggered)

Both cameras show their image in a floating view. camera1 has a notable delay but also shows the blob detection frame round the parts.

I created a litle external application for testing purposes which run synchronous, fetches the image from camera, pass it back to camera1 and triggers the next step.

BlobDetectionCamera has a non-threaded child script:

Code: Select all

if (sim_call_type==sim.syscb_init) then 
    camera=sim.getObjectHandle('blobDetectionCamera_camera1')
    cameraGT=sim.getObjectHandle('blobDetectionCamera_camera')
end

if (sim_call_type==sim.syscb_actuation) then 
    t=sim.getSimulationTime()  
    imageAnalysisFrequency=sim.getScriptSimulationParameter(sim.getScriptAssociatedWithObject(baseObject),'imageAnalysisFrequency')
    if (t-lastImageAnalysisTime>1/imageAnalysisFrequency) then
        lastImageAnalysisTime=t
        
        sim.handleVisionSensor(cameraGT) 
        resultGT,t0GT,t1GT=sim.readVisionSensor(cameraGT) 

        sim.handleVisionSensor(camera) 
        result,t0,t1=sim.readVisionSensor(camera)  --> here result is -1
        
        if (result==-1) then 
            print(sim.getLastError())
        end
        
   end
end
Problem is that readVisionSensor(camera) returns -1 and getLastError returns nil.
Docs said that readVisionSensor may return -1 when handleVisionSensor is not called but it is.

So I suppose it is a timing issue? Is there a way to synchronize it?

coppelia
Site Admin
Posts: 7396
Joined: 14 Dec 2012, 00:25

Re: Synchronize with external ImageSensor

Post by coppelia » 09 May 2019, 06:34

Hello,

does the console output any error message? Are you sure the camera object is actually not a camera, but a vision sensor? Make sure that the vision sensor camera is explicitely handled.

Can you prepare a minimal test case that illustrates the bug? Best would be to have a self-contained, very simple scene.

Cheers

SkySurfer1
Posts: 10
Joined: 14 Mar 2016, 14:34

Re: Synchronize with external ImageSensor

Post by SkySurfer1 » 09 May 2019, 13:33

Attached https://mega.nz/#!5Lwz0ALa!7X-a96tG3tKX ... JcD1deV0UM there is a scene and an python script to trigger it external.
The scene is modified in the manner described above.

Load the scene and run the script.

coppelia
Site Admin
Posts: 7396
Joined: 14 Dec 2012, 00:25

Re: Synchronize with external ImageSensor

Post by coppelia » 13 May 2019, 10:28

Ok, now I understand.

In your case, you are calling sim.handleVisionSensor(camera): this normally does 3 things:
  • clear previous computed filter data
  • read an image
  • compute filter data on the new image
In your case however, that vision sensor is flagged as external input: because of that, sim.handleVisionSensor(camera) will simply clear previouscomputer filter data and return.

When you call sim.setVisionSensorImage, then filter data will be computed. But above erases them again.
So sim.handleVisionSensor(camera) should be removed.

Cheers

SkySurfer1
Posts: 10
Joined: 14 Mar 2016, 14:34

Re: Synchronize with external ImageSensor

Post by SkySurfer1 » 17 May 2019, 09:05

Thx that fixed the issue.
Maybe I missed that in the documentation.
Otherwise a little hint in the API of handleVisionSensor or setVisionSensorImage would be great.

Post Reply