This presentation introduces "geometry-aware imitation learning" for robotics, which integrates known physics and geometric constraints into machine learning models to enable robots to learn and perform complex tasks more autonomously and robustly.
Mind Map
คลิกเพื่อขยาย
คลิกเพื่อสำรวจ Mind Map แบบอินเตอร์แอคทีฟฉบับเต็ม
to to
to
not included kindly turn your your
camera off
so my name is actually conti and I'm a
research associate at the art
engineering Institute here in London and
I would also like to introduce my
colleague and co-organizer David
massager who's a PhD enrichment student
also a design engineering
and for those of you who are not
familiar the island touring Institute is
UK's National Institute for data science
and AI
also for the advantage of those joining
us for the very first time I'd like to
briefly introduce the topic for the
seminar series that is physics enhanced
machine learning focusing on engineering
applications so physics and noise
machine learning is what most of you
know an upcoming subfield International
machine learning and deep learning that
aims to incorporate known physics
understanding into the machine learning framework
framework
on a side note I've also about to say
that if any of you are interested to
share your work within this agreement
please feel free to reach out to other
data myself or Andrea who's currently so
the unfortunately not joining us
um to find the slots for your for your
seminar that would be really great to
share your work with the community we
have quite a nice growing Community now um
um
and as well as format of this Center so
uh our speaker for today Mateo will
speak for some time around 45 minutes
and then we will have questions and
answers uh towards the end
but you might do so either uh in the
chat or asking in person all right so
without further Ado I lost my colleague David
David
to go to desired speaker for today and
with that as always I thank you all
yeah thanks uh Zach and hello everyone
um I would like to welcome here Mateo
and also introduce him briefly
um so Mateo received his bachelor's and
master of science degree in automatic
control and Engineering the University
of Naples Napoli in Italy in 2008 and
2011 respectively
uh then he received his PID from the
Technical University of Munich in 2017.
currently he is an assistant professor
at the department of industrial
engineering at the University of
torrento in Italy and privately he was
an assistant professor at the University
of Ginsburg and a postdoctoral
researcher at the German Aerospace
Center the DLR so uh Mateo thank you
very much for joining us today we are
very excited to see what uh we're gonna
learn today and yes uh
um we are ready okay
okay
thank you very much David and Zach for
actually giving me the opportunity to
present part of my research during this
seminar I hope you can hear me and see
do you see my mouse pointer yes yes okay
because I I use it a lot so I'm kind of
lost without it okay the title of my of
my talk is basically geometry aware
imitation learning in robotics I'll try
to give a little bit like teaching style
presentation at the beginning in order
to uh let also the people that are not
super familiar with the manual manifold
and differential geometry concepts to be
on board and then in the second part of
the talk I will move more towards
research oriented things and works that
actually we are doing together with other
other
cultures basically
okay but first of all I would like to
say a few words if needed why actually
willing we need learning in robotics and
I usually put this in my talks because
actually as as David said I studied uh
control engineering so I'm an engineer
and uh in the control Community uh I'm
in a large part of the country Community
is still a bit uh how to say reluctant
to go towards learning and data driven
solutions for you know the lack of
mathematical proofs guarantees and so on
even if I mean these things are now
coming but let's have a look at
basically what we have now and what we
have reached and more or less we are
somewhere here now and we can say that
basically here it's a mixture of control
Techniques plus some nice uh programming
skills well people that have basically
developed GUI
and you can just uh
actually compose some simple motion in a
sequence or in parallel and so on put
conditions and that's it this is of
course there and working it's robust
reliable and so on but the limitation is
that that's pretty limited autonomy and
requires still moderate or to high level
of expert domain knowledge and the goal
of Robotics the long term the Holy Grail
is basically to go in this area where
you have fully to um
um
uh almost fully autonomous robots or
really likely or weakly supervised
robots and
not much domain expert knowledge to
actually operate them
and this is basically the domain where
learning starts to be fundamental in
order to again increase the autonomy
without increasing the domain expert
knowledge and in this area I actually
place myself
in the imitation Learning Community also
called programming by demonstration in
in our field
so then why geometry aware imitation
learning well the the answer is pretty
easy so we are dealing with robots and
there are many type of data in robotics
that basically do not belong to the
euclidean space okay so the position is
the typical quantity that you can treat
as a euclidean
type of data because it is but already
if you move to the orientation then it's
not anymore no orientation leaves it's
on its own space similarly you have
other type of quantities like inertia
impedance and manipulability that belong
to symmetric and positive different matrices
matrices
what was uh in the past at least the
imitation Learning Community the way of
doing and dealing learning and actually
retrieving from this data was okay let's
just vectorize this data and then we do
some post processing in order to
basically restore the geometric
constraint or the manifold structure
that are very uh
uh row vectorization of this quantity
will of course completely destroy and to
better understand this actually we have
this this uh small video and and an
example here so let's assume that our
points live on a circle okay this blue
circle here and then we have these dots
here the gray ones right and we want to
compute something simple the mean
indicated with mu and which is you know
the basic of many uh probabilistic based
uh approaches in machine learning
then what happens is if we calculate the
mean treating these points as basically
belonging to R2 to the euclidean space
is that more or less we end up here and
then we can simply realize that this
point is too close to the center of the
circle to be a real one then what we can
do we can rescale this point you know
pushing it back on the circle where
originally it was and then we end up
here but if you calculate the mean using
basically a proper proper concept of
distance that preserves the fact that
these points are only Circle we may end
up here so every time we do basically we
treat Romanian data as euclidean
and then try to do something some
post-processing to push back on the
manifold we introduce some inaccuracies
and of course if this is used to
generate robot trajectory and maybe we
integrate we propagate this error
through our algorithm and you know we
end up with something that may easily go
you know out of distribution very quickly
quickly
so I'll show the video again here and
basically you see that the problem is
this one so two points in the euclidean
space no or two trajectories are here
and if they have to stay on the main for
basically we have to deform the space in
order to let them working on the Manifest
Manifest
so this means that if we want to
properly deal with Romanian data we
should redefine first of all the concept
of distance and use proper Romanian
distance and all the derived quantities
so the solution is then use the proper
tools that you know have been developed
by materializations uh in the framework
of Romanian geometry also called
sometimes differential geometry so a
reminder manifold is very general
concept it's a small topological space
equipped with the metric so with the
distance function the type of distance
function defines the property of the
manifold and it's not uniquely defined
but there are some
distances that you know give nicer
properties and are predominant in the
manifold here on the left you see a
sphere basically embedded in the
three-dimensional space this is the
manifold where orientation lives and for
example unit quaternions
and here on the right you see the cone
that actually contains symmetric and
positive definite matrices
okay so let's start giving some elements
of remaining geometry now in order to
then understand how we can deal with
this data so as I said manifold it's
actually locally isomorphic to the
euclidean space we can uh a little bit
you know improperly say with an abuse of
basically notation say that it's locally
euclidean even if this doesn't mean much
from a rigorous mathematical point of
view but I think you get the sense
actually in each points on each point on
the manifold you can actually Define the
so-called tangent space
but here it's pretty easy to to depict
where basically you can define a
coordinate frame and treat actually your
quantities as if they were euclidean in
order to do so of course we need some
operation that actually let us to move
from the manifold to the tangent space
and back these operations are usually
called the exponential map and the
logarithmic map so the exponential map
is the one that pushes a point from
tangent space onto the manifold
and the opposite is the logarithmic map
that actually lift a point from the
and another important concept is the one
of parallel transport why we need the
parallel transport because actually all
what we will build in a moment is
actually uh based on placing attention
space you know in a at some points and
then lift uh the data the nearby data on
the attention space and then use
euclidean tools to learn in this uh the
proximity of the tangent space and then
push back the data on the manifold and
then move on to the tangent space and
then when we have to basically compare
two vectors that are projected into two
different tangent space we cannot just
sum them up or subtract but we actually
need to move one vector a vector from
one tangent space to the other one so in
in the end we need that all the vectors
are represented in the same tangent
space in order to perform operations and
otherwise actually basically this
happens let me show once again so if we
just move and with typical euclidean
tools no from one tangent space to
another one we have that you see a
vector that stays here is kind of
completely changed while
uh uh the parallel transpose is
basically a way no to transfer the
proper way to transfer the vector by
keeping let's say the orientation that
it has so you move along a geodesic
connecting to two points where the two
tangent spaces are placed while actually
preserving the directionality so we need
this kind of operations but this
actually can be defined in any Romanian manifold
manifold
um this is uh just to summarize basic
arithmetic on manifolds so the
logarithmic map can be related to the
concept of difference in euclidean space
the exponential map is actually an addition
addition
and the distance I mean this concept of
distance of course you have to redefine
the distance to be on the manifold and
the interpolation so it's important can
be achieved through exponential mapping
you know multiplying basically by the
time T this difference of points in the
tangent space
okay so now we have basically introduced
our Machinery okay the Romanian manifold
Machinery let's now have a look at how
we can use that framework into basically
typical robotic problems and a first
example is uh the the super famous at
least in the imitation Learning
Community framework called Dynamic
movement Primitives introduced in the early
early
2000s by uh esper Stefan shall and
others so working at that time at the
University of California Los Angeles
so what was their idea basically they
wanted to reproduce some demonstrations
and generate something that was actually
feasible for the robot so to execute so
sufficiently smooth that's why they went
to second order Dynamics
and preserve the stability okay because
very often in robotics you want to reach
certain position a certain configuration
and keep it
so they had this idea let's say okay
let's start with the simplest dynamics
that we can consider a second board a
spring damper actually Mass system with
unitary Mass this is very famous
actually physical model everyone has
probably they have to deal with this in
basic course and physics
this is a second order Dynamics linear
uh stable asymptotically stable means
that from each point so I can basically
you can imagine that I can pull no this
side of this spring damper system and
sooner or later it will converge
depending on the value of the Springer
and the damping I can have oscillations
or I can have an exponentially decaying
trajectory because we usually say in
control tier you know it's a critically
done system
converts towards an equilibrium this is
pretty nice but of course it's quite
limited so the idea of this guys here
actually was okay let's add to this
simple system uh Force actually no
linear force in turn learned from
demonstration in order to basically uh
execute some arbitrary trajectories
and then finally retrieve asymptotic
convergence by actually killing okay so
this force is basically multiplied by a
banishing term so that with time
vanishes and let the system retrieves
asymptotic stability towards the desired code
code
this is basically uh the the formulation so
so
mono-dimensional but I mean it's not so
hard to extend this to higher Dimensions
so uh these two equation the Y Dot and Z
dot are basically the Dynamics of this guy
guy
so the velocity is the time derivative
of the position the acceleration is
basically again multiplied by the
position error minus the velocity plus
this non-linear force center only
unfortunate term is a radial basis
function Network so we're actually this
c i basis functions are gaussian components
components
nothing nothing complicated and I mean
this can be learned from demonstration
in a with a pretty pretty easy
formulation so this is the the standard
one and now of course we want to extend
this approach to basically consider data
that are embedded in a Romanian manifold
here for example I'm showing ellipsoids
which means symmetric and positive
definite matches so I can keep most of
the structure what I did so something
minor I extended this to to be
multi-dimensional and of course each of
these Y and that are now
vectors so we need to basically deal
with that and here the the way it's
become uh uh Matrix such that uh sorry
um yeah a vector so each each I is a
vector so all together can be seen as a
matrix such that also the forcing term
is a vector
and what I have changed here is uh not
basically dramatic so I have the error
here so subtraction I replaced it with
the logarithm
and the same here this is just a scaling
to prevent eye acceleration at the beginning
beginning
The crucial Point here is that I'm
actually placing my tangent space at y
so at the current point and this means
that I'm not working in a fixed
attention space but I have like a
tangent space that is sliding over the
manifold and actually I need just to do
one integration step this is very
important because if you place the
tangent space uh
somewhere and you have points that are
actually far from the center of the
tangent space you again start to
introduce some deformations in your
manifold and some numerical layers while
working like this you actually minimize
the the error that you introduce is just
one integration step and then you like
start again placing your tension space
in the next point that comes out of the
integration so integrating this one is
done through the exponential mapping
and actually this is basically one way
to extend the framework to Romanian manifolds
manifolds
uh the problem is now that how we
compute actually the training data so to
train that guy we need actually uh
derivatives so why not
and Y double dot and uh if you are in
Cartesian space pretty easy we just take
the difference between consecutive
sample and divide by the sampling time
and do again for the second derivative
on manifolds it's a little bit more
different so what we do is
um we go uh basically on the data we do
a this is called piecewise geodesic
interpolation actually so we place let's
assume that for example the two
consecutive points are this one and this
one a bit exaggerated so we have pretty
closer points so actually what we do we
then place the tension space here on the
left take this point project on this
tangent space assuming that they are
closed so I do not defer match the
trajectory then this gives me
the difference basically or what is the
equivalent of difference on a Romanian manifold
manifold
and then I divided this one by the
sampling time and this gives me actually
my derivative in any point
uh there is of course a bit of
technicality here so if I have to once I
have projected my data in the tangent
space usually I basically do not have to
consider the geometric constraints
anymore just to give you an example the
tangent space of the S3 sphere The
Continuous sphere is just a
three-dimensional euclidean space while
the tangent space of symmetric and
positive definite matrices is just the
space of symmetric matrices so asymmetic
Matrix is just uh you know equal to its
transpose and I can simply vectorize it
by taking the independent components
then you can calculate the velocity and
once you have these velocities you can
just apply euclidean tools basically to
calculate the second derivative and with this
this
your DMP
one of the interesting features of DMP
is the possibility to change the goal on
the fly now imagine that you have a
camera it's tracking an object you want
to catch it and you have learned some
stereotypical motion some template
trajectory reaching uh technology but
this goal is changing on the fly so what
you can do with DMP you can just Define
a linear system that actually
with the time depending on this gain
uh changes the goal from G to the new one
one
smoothly the equivalent of this on my
default is again A dynamical system that
instead of this difference considers uh
the logarithmic map this case placed at G
G
of jinu and then calculate g dot
you can also change actually the the
trick here is that basically you can
usually interchange these two
for for many manifolds smooth manifolds
and basically
um if you change this one then you have
to do the parallel transpose minus the
parallel transpose field okay in order
to Interchange these two so
this is where actually the parallel
transpose comes into the game this is
just a simulation so you see that uh the
DMP was basically trained to reach a
certain goal here and then we start
switching to another one and actually
our ellipsoid here that represents a
symmetric and positive definition
matrices actually changes so and instead
of reaching the Shaded goal here which
is just another one smoothly
okay it's time now to see a little bit
of of results so we have this paper and
the review
now uh where actually we describe what I
have told you so far this geometry where
moment primitive so here you see um some
Cartesian letters and then we have
transformed this pretty simply projected
on on manifolds so here on the left is
quaternion here is rotational Matrix and
on the right you see SPD
matrices so how you can reproduce that
and then some experiments so now the
robot here for example has to fill this
uh this watering can
and then it can control basically
position orientation and and stiffness
and in the other case uh the robot has
to pick from a box so we have instructed
robot to actually enter this box so we
control in this case the position and
the manipulability that is actually an
SPD Matrix in order to as a secondary
task and when we have to enter another
box we command the position and to
prevent that robot you know deform the
trajectory too much we have this
secondary task so the robot tries to
keep actually its posture
okay so the posture that has been
demonstrated during the motion now in
order to do this this basically type of
motion so you see that here this is the demonstration
demonstration
and actually when we try a little bit to generalize
