My main interest is Reinforcement Learning, with particular focus on Distributional RL, Goal-Conditioned RL, and Hierarchical RL. My goal is to become a researcher in the field, and I am currently looking for PhD opportunities. My life in brief:

• 2019 - Today: Machine Learning Master student at KTH Royal Institute of Technology. Research-oriented Master program where I first met and fell in love with Reinforcement Learning. Particular focus of the program is on Deep Learning, and I developed several implementations of deep architectures such as CNNs, RNNs, and HGNs), as well as Deep RL algorithms like ACER and Categorical DQN.
• 2019 - Today: Driverless Engineer at KTH Formula Student. The goal of KTH Formula Student is to develop an autonomous electric racing car. I joined at the beginning of my Master studies in the SLAM (Simultaneous Localisation and Mapping) subgroup. After an initial study of SLAM algorithms, my first contribution was a core implementation of GraphSLAM. Having gained a good knowledge of the SLAM software and of the overall driverless system, I had a substantial part in the recruiting process of new members for the driverless team in early 2020, interviewing > 15 candidates with responsibilities in the decisional process, and oversighted the integration of new members in the SLAM group. From February 2020 to February 2021 I covered the role of Technical Integrator of the Driverless team, supervising integration of software modules developed by subgroups in the racing car. Currently, I am working on improving the legacy implementation of FastSLAM, and I developed a Python library to compute evaluation metrics of the SLAM performances in order to track the level of improvement of new features.
• 2018 - 2019: Embedded Software Developer Intern at Profitap. During the internship I gained a lot of valuable experience in C and C++ developing, as well as best practices of development such as documentation, testing, version control, and continuous integration. I developed a PAM authentication module in C and a SNMP protocol implementation in C++ for network devices. I developed a Bash library for Continuous Integration to efficiently handle dependencies and versioning in several projects.
• 2015 - 2018: Bachelor Degree in Computer Engineering at Università degli Studi di Padova.

2021, [Re] Hamiltonian Generative Networks, Carles Balsells Rodas, Oleguer Canal, Federico Taschin, Re Science C 7, 2, #18

• 2021 [Re] Hamiltonian Generative Networks: Implementation of the paper "Hamiltonian Generative Networks". Learning Hamiltonian dynamics from videos of simulated physical systems. We provided the first open-source implementation of the algorithm, reproduced the original experiments, performed additional experiments on new environments, and proposed a baseline comparison.
• 2020 Self-learned vehicle control using PPO: Learn to control a car and a drone in simulation using Reinforcement Learning (PPO). We perform path planning on the visibility graph and make the vehicles learn to follow the waypoints as fast as possible. Our code won the competition against other 15 groups.
• 2020 Neural Network Surgery in Deep Reinforcement Learning: Extensive analysis and evaluation of performing weight transfer in Deep RL models when the model changes, for example, when actions, observations, or hidden layers are added. This avoids re-training a model from scratch when the we apply modifications to it, resulting in better performances and less training time.
• 2020 Reinforcement Learning with Hardcoded Policies: In this work I investigate how Reinforcement Learning can exploit hardcoded policies to learn faster, and how hardcoded policies can be improved with Renforcement Learning.
• 2019 [Re] Gaussian Process Latent Variable Model: Implementation of the paper Gaussian process models for visualisation of high dimensional data. We implement the GPLVM model and evaluate it on the paper's datasets plus some additional ones. We perform an extensive comparison of the model against other dimensionality reduction techniques such as PCA and Kernel PCA.
• 2019 Cooperative Multi-Agent AI: We deal with multi-agent tasks such as search, vehicle routing, and formation sweeping. We exploit a combination of classic spanning tree, routing, and formation algorithms and improve on them with efficient evolutionary algorithms. Overall, our group performed best on the 4 benchmark tasks among the 12 participant groups.
• 2018 Analysis of a Dermatoscopic Dataset, Bachelor Thesis: Application of Image Segmentation techniques to images of melanocytic lesions, extrcting their shape and size and analyzing the data.

CampusAI: Created with Oleguer Canal, we publish content related to Machine Learning and Reinforcement Learning.