Train a camera to reach an object

Typically: "How do I... ", "How can I... " questions
Post Reply
raamasgh
Posts: 8
Joined: 03 Feb 2020, 12:18

Train a camera to reach an object

Post by raamasgh »

Hello!

I want to do the following, and I've been trying for so long with no luck!

1. Create a camera object
2. Create a cube in a table (in a random position)
3. find the path between the camera and the cube
4. let the camera follow the path and stop once reaches the cube.
5. record the camera image and velocity at each step in the path.

Is this possible? if yes, could you please advise on how to do it?

Thanks!

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Train a camera to reach an object

Post by coppelia »

Hello,

of course this is possible. I suppose you want to avoid obstacles, otherwise it would be as simple as interpolating between two object's positions.

But first you need to decide what kind of movement your camera will do: a simple movement to the cube on the table, without colliding with other objects is nice, but I guess you'd like to keep a constant eye on your cube, while moving towards it. That makes things more complicated. Create a cuboid that represents your camera. Attach a camera on top of that cuboid. That cuboid should be collidable, and static. Then use the OMPL functions to find a path between the cuboid and the cube on the table. You basically want to bring a point e.g. 0.5 meters offset from your cuboid onto the cube on the table, i.e. you want to reach the cuboid on the table with a 0.5 meter offset. Also, I'd use a state validation callback, in which you'd check whether the provided cuboid states collide with the environment, and whether the line of sight to the cube on the table is ok. You'll end-up with a collision-free path, where your camera always has sight onto the cube on the table. Then, you'll have to smooth that path somehow, while still guaranteeing constant collision-free and sight onto your cube.
Have a look at the simple demo scene scenes/6DoFHolonomicPathPlanning.ttt. For a state validation callback example, have a look at this demo scene.

Cheers

raamasgh
Posts: 8
Joined: 03 Feb 2020, 12:18

Re: Train a camera to reach an object

Post by raamasgh »

Thanks for your response!

I managed to get it done when the cube and vision sensor positions are fixed (i used vision sensor because I wasn't able to get the camera working with python)

However, when I choose the cube and camera positions to be random, the following is the behaviour:
https://drive.google.com/open?id=1xU-DU ... 87z_BEG3n7

I want the camera to follow the cube, and to orient towards it. I'm attaching as well a copy of the scene

Note: I'm controlling it using python, but I attached the script to the scene to illustrate the behaviour:

https://drive.google.com/open?id=10DXZ6 ... fMFe-Spjim

Much appreciated!

coppelia
Site Admin
Posts: 10339
Joined: 14 Dec 2012, 00:25

Re: Train a camera to reach an object

Post by coppelia »

If your vision sensor simply always need to look at the target, then you could programmatically adjust its orientation in each simulation step.
Or you can attach a camera to your path planning object, and on top of the camera you vision sensor. Then, for the camera, in its properties dialog, adjust item Tracked object to be your cube.

Cheers

raamasgh
Posts: 8
Joined: 03 Feb 2020, 12:18

Re: Train a camera to reach an object

Post by raamasgh »

It works!!!!
your effort is very much appreciated!

Post Reply