Remote API calls getting slower and slower

Report crashes, strange behaviour, or apparent bugs
JoakimMagnusson

Remote API calls getting slower and slower

Post by JoakimMagnusson »

Hi,

I have a problem that my python remote API calls are getting slower and slower whenever I run a simulation.
I run in synchronous mode.

For get handles calls I'm doing like this:

Code: Select all

errorCode, handle = vrep.simxGetObjectHandle(self.clientID, string, vrep.simx_opmode_blocking)
For get variable calls I'm first initiating with this call:

Code: Select all

vrep.simxGetObjectPosition(self.clientID, self.hexapod, -1, vrep.simx_opmode_streaming)
and then during the rest of the simulation I'm doing like this:

Code: Select all

while (vrep.simxGetConnectionId(self.clientID) != -1):
			[result,position]=vrep.simxGetObjectPosition(self.clientID, self.hexapod, -1, vrep.simx_opmode_buffer)
			if (result==vrep.simx_return_ok):
 				return position
Worth mentioning is also that I resets my simulation model (of a robot) several times during the simulation. In my reset function, I'm doing like this for all get variable calls:

Code: Select all

vrep.simxGetObjectPosition(self.clientID, self.hexapod, -1, vrep.simx_opmode_discontinue)
Then I remove and reload my model (In order to move it to it's starting position). After this I initiate all get variable calls with simx_opmode_streaming again.

For set variable calls I'm using the simx_opmode_oneshot or simx_opmode_oneshot_wait operators.

I thought doing like this would prevent the simulation from slowing down but it still does. How do I know it's getting slower? I'm timing my simxSynchronousTrigger call which are getting longer and longer during simulation.

Is there something else to have in mind in order to not get any memory leaks or whatever is causing this issue?

I can show more code if necessary but I tried to extract the important parts and thought it would be more helpful than just printing all my code.

JoakimMagnusson

Re: Remote API calls getting slower and slower

Post by JoakimMagnusson »

I tried to just restart the simulation instead of removing and reloading my robot model, this made the problem disappear. Restarting the simulation is slower than reloading the model and I would like to use the fastest option. So I still wonder why this problem occur, do I reload the model in the wrong way maybe?

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Remote API calls getting slower and slower

Post by coppelia »

Hello,

I am not sure due to what reason the simulation gets slower. Is it the simulation itself, or the remote API?
My first guess was that it could be the remote API, where streaming data is not properly discontinued. But it seems you are discontinuing streaming commands, so that is not the reason. Can you notice a memory usage increase that is more than what you expect?
Maybe when remove your model, not everything is removed (e.g. a collision checking object) and you end-up with many duplicates? (normally this is automatically handled when removing the model, but specific collision objects are persistent)

Cheers

JonasMuegge

Re: Remote API calls getting slower and slower

Post by JonasMuegge »

Hi, is there a solution to this problem, cause I have got the same.
In my project I connect from a julia script via python b0 remote api to a simulation.
In a test script I only connect and do some basic movements and call some functions in the simulation

the connection is build up here and works fine:

Code: Select all

	b0RemoteApi=PyCall.pyimport("b0RemoteApi")
	client=b0RemoteApi.RemoteApiClient("b0RemoteApi_V-REP-addOn","b0RemoteApiAddOn")
	client.simxSynchronous(true)
	topicPub = client.simxDefaultPublisher()
	topicSubStart = client.simxDefaultSubscriber(callbackFunStart)
	topicSubStop = client.simxDefaultSubscriber(callbackFunDone)
Then I repeat the following code to move a human model in a circle:

Code: Select all

	client.simxStartSimulation(topicPub)
	client.simxCallScriptFunction("selectWalkingDirection@Bill","sim.scripttype_customizationscript",walkingDirection,topicPub)
	client.simxCallScriptFunction("selectUpperBodyMotion@Bill","sim.scripttype_customizationscript",bend,topicPub)
	for i=1:timesteps
		client.simxGetSimulationStepStarted(topicSubStart)
		client.simxGetSimulationStepDone(topicSubStop)
		client.simxSynchronousTrigger()
		client.simxSpinOnce()
	end
	client.simxPauseSimulation(topicPub)
The elapse time for this code section gets higher and higher. It starts at 0.5 seconds and increases. After 300 executions it takes around 2 seconds and after 500 executions around 3 seconds.

Hope there is a solution, Thanks.

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Remote API calls getting slower and slower

Post by coppelia »

Hello,

thanks for bringing this again to our attention. Would it be possible to create a very simple example that illustrates the slowdown? The simpler, the easier it would be for us to identify the source of the problem. Best of course would be if that can be reproduced with an empty scene, since this would very probably indicate that the slowdown is really caused by the B0-based remote API.

Cheers

JonasMuegge

Re: Remote API calls getting slower and slower

Post by JonasMuegge »

Hi,

I deleted all unnecessary code from my project. Now it only connects via b0remoteApi, starts simulation, stops simulation, and spins. Here the code from the script:

Code: Select all

VRPInterface=include("VREPInterface.jl")

for a = 1:100
    client,topicPub,topicSubStart,topicSubStop = VREPInterface.connectToVREP() #,collisionHandle,distanceHandle,robotHandle,humanHandle=VREPInterface.connectToVREP()
    for i = 1:1000
        @info "step number " (a-1)*100 + i
        #starttime = Dates.t
        step = mod(i,2)
        @info @elapsed VREPInterface.executeAction(step,0, client, 2, topicPub, topicSubStart, topicSubStop)
        # distCalcReturn=client.simxReadDistance(distanceHandle,client.simxServiceCall())
        # success,v_linear,v_angular=client.simxGetObjectVelocity(robotHandle,client.simxServiceCall())
        # collisionReturn = client.simxReadCollision(collisionHandle,client.simxServiceCall())
        # velocityReturn = client.simxGetObjectVelocity(robotHandle,client.simxServiceCall())
        # positionReturn = client.simxGetObjectPosition(humanHandle, -1, client.simxServiceCall())
    end
    #client.__exit__(client)
end
And the code for the VREPInterface,jl module

Code: Select all

module VREPInterface

using PyCall
export connect_to_vrep

function connectToVREP()
	@info pwd()
	
	pushfirst!(PyCall.PyVector(PyCall.pyimport("sys")."path"), "")
	b0RemoteApi=PyCall.pyimport("b0RemoteApi")
	client=b0RemoteApi.RemoteApiClient("b0RemoteApi_V-REP-addOn","b0RemoteApiAddOn")
	client.simxSynchronous(true)
	topicPub = client.simxDefaultPublisher()
	topicSubStart = client.simxDefaultSubscriber(callbackFunStart)
	topicSubStop = client.simxDefaultSubscriber(callbackFunDone)
	return client,topicPub,topicSubStart,topicSubStop #,collisionHandle,distanceHandle,robotHandle,humanHandle
end

function callbackFunStart(msg)
	print("simulation Step Started")
end

function callbackFunDone(msg)
	print("Simulation Step Done")
end

# communication to simulation VREP, setting the motion and executing it
function executeAction(walkingDirection,bend,client,timesteps,topicPub,topicSubStart,topicSubStop)
	client.simxStartSimulation(topicPub)
	#client.simxCallScriptFunction("selectWalkingDirection@Bill","sim.scripttype_customizationscript",walkingDirection,topicPub)
	#client.simxCallScriptFunction("selectUpperBodyMotion@Bill","sim.scripttype_customizationscript",bend,topicPub)
	for i=1:timesteps
		client.simxGetSimulationStepStarted(topicSubStart)
		client.simxGetSimulationStepDone(topicSubStop)
		client.simxSynchronousTrigger()
		client.simxSpinOnce()
	end
	client.simxPauseSimulation(topicPub)
end
end

Currently I am using julia 1.3.1 and https://github.com/JuliaPy/PyCall.jl to use python in it.
Is this ok for you?

Running the above code I get the following outcome.
As you can see the elapse time still increases.

Code: Select all

   
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 2.006505101
┌ Info: step number 
└   (a - 1) * 100 + i = 329
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 1.993775999
┌ Info: step number 
└   (a - 1) * 100 + i = 330
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 2.100352901
┌ Info: step number
└   (a - 1) * 100 + i = 331
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 1.9877204
┌ Info: step number
└   (a - 1) * 100 + i = 332
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 2.013947899
┌ Info: step number 
└   (a - 1) * 100 + i = 333
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 1.864915399
...

┌ Info: step number
└   (a - 1) * 100 + i = 670
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 4.5439249
┌ Info: step number
└   (a - 1) * 100 + i = 671
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 3.7425493
┌ Info: step number
└   (a - 1) * 100 + i = 672
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 4.9993546
┌ Info: step number 
└   (a - 1) * 100 + i = 673
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 4.3945773
┌ Info: step number
└   (a - 1) * 100 + i = 674
Simulation Step Donesimulation Step StartedSimulation Step Donesimulation Step Started[ Info: 4.9315689

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Remote API calls getting slower and slower

Post by coppelia »

Hello Jonas,

Thanks for the effort... I would however really appreciate if you could simplify your example to use a simple Python script as client, since otherwise things are more difficult for use to set-up and also identify the source of problem.

Cheers

JonasMuegge

Re: Remote API calls getting slower and slower

Post by JonasMuegge »

All right.

Code: Select all

import b0RemoteApi
import time

with b0RemoteApi.RemoteApiClient('b0RemoteApi_pythonClient','b0RemoteApiAddOn',60) as client:    

    def callb(msg):
        #print(msg)
        None
    
    def action():
        client.simxStartSimulation(topicPub)
        for i in range(4):
            client.simxGetSimulationStepStarted(topicSubStart)
            client.simxGetSimulationStepDone(topicSubStop)
            client.simxSynchronousTrigger()
            client.simxSpinOnce()

    client.simxAddStatusbarMessage('Hello',client.simxDefaultPublisher())
    topicPub = client.simxDefaultPublisher()
    topicSubStart = client.simxDefaultSubscriber(callb)
    topicSubStop = client.simxDefaultSubscriber(callb)
    for i in range(1000):
        print(i)
        starttime = time.time()
        action()
        endtime = time.time()
        print(endtime - starttime)

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Remote API calls getting slower and slower

Post by coppelia »

Hello Jonas,

in your example, you are actually subscribing a new message in each simulation step... why? You should call for instance:

Code: Select all

client.simxGetSimulationStepStarted(topicSubStart)
only once to register that subscriber. If you do it more than once, you'll have everytime one additional subscriber enabled.

Additionally, you start simulation but never stop it. And when stopping it, beware that it will take a few additional simulation steps until it has stopped. So slightly modifying your example to following doesn't have the delay anymore:

Code: Select all

import b0RemoteApi
import time

with b0RemoteApi.RemoteApiClient('b0RemoteApi_pythonClient','b0RemoteApiAddOn',60) as client:    

    def callb(msg):
        print(msg)
        None
    
    def action():
        client.simxStartSimulation(topicPub)
        for i in range(4):
            client.simxSynchronousTrigger()
            client.simxSpinOnce()
        client.simxStopSimulation(topicPub)
        for i in range(4):
            client.simxSynchronousTrigger()
            client.simxSpinOnce()

    client.simxAddStatusbarMessage('Hello',client.simxDefaultPublisher())
    topicPub = client.simxDefaultPublisher()
    topicSubStart = client.simxDefaultSubscriber(callb)
    topicSubStop = client.simxDefaultSubscriber(callb)
    client.simxGetSimulationStepStarted(topicSubStart)
    client.simxGetSimulationStepDone(topicSubStop)
    for i in range(1000):
        print(i)
        starttime = time.time()
        action()
        endtime = time.time()
        print(endtime - starttime)
Of course you could register a new subscriber in each new simulation run. But then you should remove it again at the end of the simulation. Taking the demo scene synchronousImageTransmissionViaRemoteApi.ttt, you can test it with following code:

Code: Select all

import b0RemoteApi
import time

with b0RemoteApi.RemoteApiClient('b0RemoteApi_pythonClient','b0RemoteApi') as client:    

    client.doNextStep=True
    client.runInSynchronousMode=True

    def simulationStepStarted(msg):
        simTime=msg[1][b'simulationTime'];
        #print('Simulation step started. Simulation time: ',simTime)
        
    def simulationStepDone(msg):
        simTime=msg[1][b'simulationTime'];
        #print('Simulation step done. Simulation time: ',simTime);
        client.doNextStep=True
        
    def imageCallback(msg):
        #print('Received image.',msg[1])
        client.simxSetVisionSensorImage(passiveVisionSensorHandle[1],False,msg[2],client.simxDefaultPublisher())
        
    def stepSimulation():
        if client.runInSynchronousMode:
            while not client.doNextStep:
                client.simxSpinOnce()
            client.doNextStep=False
            client.simxSynchronousTrigger()
        else:
            client.simxSpinOnce()
    
    visionSensorHandle=client.simxGetObjectHandle('VisionSensor',client.simxServiceCall())
    passiveVisionSensorHandle=client.simxGetObjectHandle('PassiveVisionSensor',client.simxServiceCall())

    if client.runInSynchronousMode:
        client.simxSynchronous(True)

    for y in range(100):
        starttime = time.time()
        sub1=client.simxDefaultSubscriber(imageCallback)
        client.simxGetVisionSensorImage(visionSensorHandle[1],False,sub1)
        sub2=client.simxDefaultSubscriber(simulationStepStarted)
        client.simxGetSimulationStepStarted(sub2);
        sub3=client.simxDefaultSubscriber(simulationStepDone)
        client.simxGetSimulationStepDone(sub3);
        client.simxStartSimulation(client.simxDefaultPublisher())
        startTime=time.time()
        for x in range(4):
            stepSimulation()
        client.simxStopSimulation(client.simxDefaultPublisher())
        endtime = time.time()
        print(endtime - starttime)
        client.simxRemoveSubscriber(sub3)
        client.simxRemoveSubscriber(sub2)
        client.simxRemoveSubscriber(sub1)
Cheers

JonasMuegge

Re: Remote API calls getting slower and slower

Post by JonasMuegge »

Thanks a lot for the answer, it helped me a lot. Now I don't have any issues concerning time.

Is there a definite number of timesteps for coppeliasim to stop the simulation after calling simxStopSimulation()?

Post Reply