The research activities that I carry out study various aspects of machine learning and computer vision, with a focus on how they can be integrated within real-world applications. In particular, I am interested in the design and integration of perception components based on machine learning for cyber-physical systems, and in particular robotic systems. Since the beginning of my PhD in 2015, I have been involved in many projects focusing on different parts of the perception-decision-action paradigm, which is essential for autonomous robotic systems. In particular, I am interested about
- how a robotic system can learn to achieve actions in the real-world (reinforcement learning),
- how it can use previous knowledge to understand the real-world (transfer learning),
- how it can act to improve its understanding of the real-world (active vision),
- how our knowledge of the physics of the system can be leveraged to improve trust in the model (predictions correction).
January 2021: I submitted my application for the CNRS 2021 CRCN competition! Thanks a lot to the members of the RAP (LAAS-CNRS),
ICAR (LIRMM) and ComSee (Institut Pascal) teams for their support.
January 2021: Our paper Combining pretrained CNN feature extractors to enhance clustering of complex natural images
was published in Neurocomputing!
November 2020: I am joining the Artificial and Natural Intelligence Toulouse Institute (ANITI), where I will work with
Jérémie Guiochet and Kévin Delmas,
on problems related to runtime monitoring of learning-based systems. A huge thanks to Luiz Marcos Gonçalves
for the welcome I received in the NatalNet lab. Now looking forward to start this new exciting project.
September 2020: Our paper Robust Detection of Objects under Periodic Motion with Gaussian Process Filtering was
accepted for oral presentation at ICMLA 2020!
August 2020: Our paper Towards practical implementations of person re-identification from full video frames
was published in Pattern Recognition Letters!
July 2020: Our paper Forecasting Covid-19 Dynamics in Brazil: A Data Driven Approach
has been published in the International Journal of Environmental Research and Public Health. Check it out to understand how clustering of regions can constitute a valuable preprocessing to improve prediction results!
Combining pretrained CNN feature extractors to enhance clustering of complex natural images
Joris Guérin, Stéphane Thiery,
Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features,
extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However,
in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims
at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted
and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering.
These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue,
we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different
architectures as different “views” of the same data. This approach is based on the assumption that information contained in the
different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture
that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets,
and produces state-of-the-art results for IC.
Robust Detection of Objects under Periodic Motion with Gaussian Process Filtering
Anne Magaly de Paula Canuto,
Luiz Marcos Garcia Gonçalves
International Conference on Machine Learning and Applications (ICMLA), 2020
Object Detection (OD) is an important task in Computer Vision with many practical applications.
For some use cases, OD must be done on videos, where the object of interest has a periodic motion.
In this paper, we formalize the problem of periodic OD, which consists in improving the performance
of an OD model in the specific case where the object of interest is repeating similar spatio-temporal
trajectories with respect to the video frames. The proposed approach is based on training a
Gaussian Process to model the periodic motion, and use it to filter out the erroneous predictions
of the OD model. By simulating various OD models and periodic trajectories, we demonstrate that
this filtering approach, which is entirely data-driven, improves the detection performance by a large margin.
Semantically Meaningful View Selection
International Conference on Intelligent Robots and Systems (IROS), 2018
An understanding of the nature of objects could help robots to solve both
high-level abstract tasks and improve performance at lower-level concrete tasks.
Although deep learning has facilitated progress in image understanding, a robot's
performance in problems like object recognition often depends on the angle
from which the object is observed. Traditionally, robot sorting tasks rely on
fixed top-down views of the objects. By changing its viewing angle, a robot
can select a more semantically informative view leading to better performance
for object recognition. In this paper, we introduce the problem of semantic
view selection, which consists in finding good camera poses to gain semantic
knowledge about observed objects. We propose a conceptual generic formulation
of the problem, together with a relaxation based on clustering, to make it
solvable. We then present a new image dataset consisting of around 10k images
representing various views of 144 objects under different poses. Finally we use
this dataset to propose a first solution to the problem by training a neural
network to predict a "semantic score" from a top view image and camera pose.
The views predicted to have higher scores are then showed to provide better
clustering results than fixed top-down views.
Learning local trajectories for high precision robotic tasks :
application to KUKA LBR iiwa Cartesian positioning
To ease the development of robot learning in industry, two conditions need
to be fulfilled. Manipulators must be able to learn high accuracy and precision
tasks while being safe for workers in the factory. In this paper, we extend
our previous paper,
which consist in rapid learning of local high accuracy behaviors. By exploration and regression,
linear and quadratic models are learnt for respectively the dynamics and cost function.
Iterative Linear Quadratic Gaussian Regulator combined with cost quadratic regression
can converge rapidly in the final stages towards high accuracy behavior as the cost
function is modelled quite precisely. In this paper, both a different cost function
and a second order improvement method are implemented within this framework. We also
propose an analysis of the algorithm parameters through simulation for a positioning task.
Finally, an experimental validation on a KUKA LBR iiwa robot is carried out. This collaborative
robot manipulator can be easily programmed into safety mode, which makes it qualified for the
second industry constraint stated above.
Machine learning improvements for robotic applications in industrial context:
Case study of autonomous sorting
Ph.D. dissertation (2018)
Thanks to their flexible mechanical design, modern industrial robots can be programmed
for different tasks without physical modification. In addition, they are highly
instrumented and should be able to be responsive to their environment. However,
the use of robots in industry is still restricted to repeatable tasks with low level
of adaptability. In an industrial context, it is essential to program robots that can
autonomously adapt to different applications and are robust to changes in working
conditions. The machine learning framework for robot programming is well suited to
design such kinds of adaptive and robust applications. Hence, in this thesis, several
machine learning contributions are presented, aiming at designing smarter robotic
applications, with a broader operational range. The methods developed are centered on
autonomous sorting, but may be useful to address problems in many other subfields
of robotics. Throughout this thesis, we propose new approaches to image clustering,
optimal view selection, trajectory learning and stereo localization, with the objective
of designing more universal robotic sorting applications.