Reading and displaying video sensor images using py3 remote API

[psychemedia]

Any (working) examples out there of how to use python3 and the remote API to:

1) read Baxter vision sensor feeds; (related to this, what is the format of the vision sensor data accessed fro the API?)
2)display single frames taken from the feed (ideally in a Jupyter notebook);
3) display a video feed (ideally in a Jupyter notebook).

This is just a first step on the road to doing image processing on the feed, but I seem (again) to be stumbling at the first hurdle:-(

--tony

[coppelia]

Hello Tony,

here a simple example that streams and displays an image, using openCV:

Code: Select all

# -*- coding: utf-8 -*-
"""
Created on Sun Jul 05 15:01:58 2015

@author: ACSECKIN
"""

import vrep
import time
import cv2
import numpy as np

vrep.simxFinish(-1)

clientID = vrep.simxStart('127.0.0.1', 19999, True, True, 5000, 5)

if clientID!=-1:
    print 'Connected to remote API server'
    print 'Vision Sensor object handling'
    res, v1 = vrep.simxGetObjectHandle(clientID, 'vs1', vrep.simx_opmode_oneshot_wait)
    print 'Getting first image'
    err, resolution, image = vrep.simxGetVisionSensorImage(clientID, v1, 0, vrep.simx_opmode_streaming)
    while (vrep.simxGetConnectionId(clientID) != -1):
        err, resolution, image = vrep.simxGetVisionSensorImage(clientID, v1, 0, vrep.simx_opmode_buffer)
        if err == vrep.simx_return_ok:
            print "image OK!!!"
            img = np.array(image,dtype=np.uint8)
            img.resize([resolution[1],resolution[0],3])
            cv2.imshow('image',img)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
        elif err == vrep.simx_return_novalue_flag:
            print "no image yet"
            pass
        else:
          print err
else:
  print "Failed to connect to remote API Server"
  vrep.simxFinish(clientID)

cv2.destroyAllWindows()

Cheers

[psychemedia]

Lovely - thank you:-)
--tony