Transfering camera and gyro data from V-REP to the MATLAB via RemoteAPI

Typically: "How do I... ", "How can I... " questions
Post Reply
robotico
Posts: 11
Joined: 08 Nov 2018, 15:02

Transfering camera and gyro data from V-REP to the MATLAB via RemoteAPI

Post by robotico » 09 Nov 2018, 20:05

Hello, I am new at V-REP. I want to use quadricopter with MATLAB. There are front camera, floor camera and gyro on robot. First of all I run the RemoteAPI connection without any problem. After that, I want to transfer cameras' image and gyro data to the MATLAB. I tried but I could not. How do we get camera image and gyro data from the V-REP?
Below you can find a part from my MATLAB code:

Code: Select all

%%
vrep=remApi('remoteApi'); % using the prototype file (remoteApiProto.m)
vrep.simxFinish(-1); % just in case, close all opened connections

clientID=vrep.simxStart('127.0.0.1',19999,true,true,5000,5);
N =[800 600];
if (clientID>-1)
    disp('Connected')
    %Handle
    [returnCode,targetObj]=vrep.simxGetObjectHandle(clientID,'Quadricopter_target',vrep.simx_opmode_blocking);
    [returnCode,copter]=vrep.simxGetObjectHandle(clientID,'Quadricopter_base',vrep.simx_opmode_blocking);
    [returnCode,floorCam]=vrep.simxGetObjectHandle(clientID,'Quadricopter_floorCamera',vrep.simx_opmode_blocking);
    [returnCode,frontCam]=vrep.simxGetObjectHandle(clientID,'Quadricopter_frontCamera',vrep.simx_opmode_oneshot_wait);
    [returnCode,gyroRef]=vrep.simxGetObjectHandle(clientID,'GyroSensor_reference',vrep.simx_opmode_blocking);
    [err,signal]=vrep.simxReadStringStream(clientID,'gyroData',vrep.simx_opmode_streaming);
    pause(0.1)
    [err,signal]=vrep.simxReadStringStream(clientID,'gyroData',vrep.simx_opmode_buffer);
    
    %% Initialization
    
    % Initialize robot pose stream
    [returnCode,next_robot_position] = vrep.simxGetObjectPosition (clientID,copter,-1,vrep.simx_opmode_streaming);  
    [returnCode,next_robot_orientation] = vrep.simxGetObjectOrientation(clientID,copter,-1,vrep.simx_opmode_streaming);
    [returnCode,N,image]= vrep.simxGetVisionSensorImage2(clientID,frontCam,0,vrep.simx_opmode_streaming); % Initialize streaming
    pause(0.1)
    [returnCode,N,image]= vrep.simxGetVisionSensorImage2(clientID,frontCam,0,vrep.simx_opmode_buffer); % Try to retrieve the streamed data
    pause(1)
    imshow(image);

coppelia
Site Admin
Posts: 7007
Joined: 14 Dec 2012, 00:25

Re: Transfering camera and gyro data from V-REP to the MATLAB via RemoteAPI

Post by coppelia » 12 Nov 2018, 10:58

Hello,

at first glance your code looks fine: you first initialize streaming, then try to read the image from the input buffer. However you are maybe not waiting enough (maybe it takes a bit longer than 0.1 s for the first image to arrive).
Also, first try to simple get the image and display it via a blocking operation like in this example. If you can receive and display an image, then switch to streaming operation.

Cheers

robotico
Posts: 11
Joined: 08 Nov 2018, 15:02

Re: Transfering camera and gyro data from V-REP to the MATLAB via RemoteAPI

Post by robotico » 17 Nov 2018, 21:12

Hello,
Thank you very much for your reply.
I tried to receive image from the front camera and floor camera but I could not.
When I switch to kinect and use

Code: Select all

vrep.simxGetVisionSensorImage2
command, and request image from the sensor (kinect_rgb) directly, I could get the image.
I read your comments about cameras and vision sensors but I did not understand why we cannot use camera as in real life.

coppelia
Site Admin
Posts: 7007
Joined: 14 Dec 2012, 00:25

Re: Transfering camera and gyro data from V-REP to the MATLAB via RemoteAPI

Post by coppelia » 19 Nov 2018, 08:02

A Camera in V-REP doesn't not have a specific resolution: it can display in a large sub-view, and in a small sub-view too. At the same time, you cannot perform any image processing on it, nor retrieve the image.

On the other hand, a vision sensor has a specific resolution and mode of operation. You can apply image processing and also retrieve the image as RGB buffer.

Cheers

Post Reply