Learning joint dynamics

Typically: "How do I... ", "How can I... " questions
Post Reply
mthor13
Posts: 61
Joined: 03 Jul 2017, 08:32
Contact:

Learning joint dynamics

Post by mthor13 »

Dear all,

I came across a study where they learn the actuator/joint dynamics instead of using physic engine calculations.

Is there any simple way of replacing the code for a joint with, e.g., a neural network that takes as input the command and outputs sensory values?

Best Regards,
Mathias Thor

RobAtLab
Posts: 92
Joined: 10 Jan 2018, 17:49

Re: Learning joint dynamics

Post by RobAtLab »

You can replace a joint object's callback script with something else, and you can then feed data to the joint object from another object by using something like a custom datablock. But I can't imagine why you would want to replace the simplicity of a deterministically commanded joint with a neural network. If you want to try getting a robot to learn to make a certain movement you're better using normally controleld joints and having the neural network have whatever data you want as it's input and then having it output an angle and/or speed for the joint to go to.

http://forum.coppeliarobotics.com/v ... f=9&t=7752 that thread talks about custom callback scripts for joints, you'd just have to put your custom code in the joint instead of the usual PID type of control code.

mthor13
Posts: 61
Joined: 03 Jul 2017, 08:32
Contact:

Re: Learning joint dynamics

Post by mthor13 »

Hi RobAtLab,

Isn't the Joint callback functions only for low-level control algorithms as the physical engine handle the joint? The reason why I want to completely replace the joint dynamics is that by learning to mimic the joint of the real-life robotic system it is easier to cross the reality gap (i.e., the gap between the simulation and real life). Said in another way, the transferability of the controller will be increased when using a neural network for approximating the dynamics of the real-life joint.

Best,
Mathias Thor

fferri
Posts: 1193
Joined: 09 Sep 2013, 19:28

Re: Learning joint dynamics

Post by fferri »

I think in this task you want to control the entire kinematic chain (i.e. the joints motors) as a whole, to perform tasks such as inverse kinematics/dynamics, gravity compensation, and so on.

This is not a use case for messing with the joint callback. The default joint callback will do just fine.

You should rather have a child script computing the motor commands to apply, given whatever (sensory?) input you want.

mthor13
Posts: 61
Joined: 03 Jul 2017, 08:32
Contact:

Re: Learning joint dynamics

Post by mthor13 »

Hmmm, I don't think that would work as that would still make use of the joint dynamics specified by the physics engine (e.g., vortex). What I want is to replace the default joint (which takes inputs and outputs sensor values) with another function (i.e., a neural network that takes the same inputs and also outputs sensor values)

I hope you get what I mean :-)

Best,
Mathias Thor

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Learning joint dynamics

Post by coppelia »

I am not sure I understand what you mean ;)

If you want to produce specific force/torque output for a given joint, based on some value calculated by a neural network for instance, then you should use a joint control callback function. The force/torque will still be applied by the underlying joint model in the physics engine. You could also imaging applying directly a force/torque to an object, and so completely replacing the joint model in the physics engine. But the force/torque will still be applied by the physics engine (and the result would be very poor).
So you can't really leave out the physics engine. Unless the whole mechanism is kinematically handled (and thus the physics engine won't touch those joints and bodies). For that you can either turn off the dynamics for the whole scene, or build a mechanism that has joints in passive mode (i.e. vs having joints in force/torque mode) and all shapes as static (i.e. vs having the shapes dynamic). And then you can compute the physics/dynamics yourself if you wanted.

Cheers

mthor13
Posts: 61
Joined: 03 Jul 2017, 08:32
Contact:

Re: Learning joint dynamics

Post by mthor13 »

Hi Coppella,

That completely answers my question, thank you very much.

If you are interested, then you can find a presentation of the research, from where I got the idea, here: https://www.youtube.com/watch?v=aTDkYFZFWug

Best Regards,
Mathias Thor

Post Reply