Your Title

Policy Learning

I've been always interested the ultimate goal of learning in agents: learning to behave toachieve goals. That's why I have done some research (including my PhD) in policy learning (mainly in reinforcement learning).

Simplifying Deep Temporal Difference Learning (Gallici, M.; Fellows, M.; Ellis, B.; Pou, B.; Masmitja, I.; Foerster, J. N.; Martin, M.), In proceedings of ICLR 2025, 2025.
Morphological symmetries in robotics (Apraez, D. O.; Turrisi, G.; Kostic, V.; Martin, M.; Agudo, A.; Moreno-Noguer, F.; Pontil, M.; Semini, C.; Mastalli, C.), The International Journal of Robotics Research, 2025. DOI: 10.1177/02783649241282422.
Integrating supervised and reinforcement learning for predictive control with an unmodulated pyramid wavefront sensor for adaptive optics (Pou, B.; Pou, B.; Pou, B.; Smith, J.; Smith, J.; Quinones, E.; Martin, M.; Gratadour, D.), Optics Express, Vol. 32, Issue 21, pp. 37011-37035, vol. 32, no. 21, pp. 37011--37035, 2024. DOI: 10.1364/OE.530254.
Integrating deep neural networks with COSMIC for real-time control (Pou Mulet, B.; Ferreira, F.; Quinones, E.; Martin, M.; Gratadour, D.), In proceedings of Software and Cyberinfrastructure for Astronomy VIII, vol. 13101, pp. 24, 2024. DOI: 10.1117/12.3019710.
Dynamic robotic tracking of underwater targets using reinforcement learning (Masmitja, I.; Martin, M.; O'Reilly, T.; Kieft, B.; Palomeras, N.; Navarro, J.; Katija, K.), Science Robotics, vol. 8, no. 80, 2023. DOI: 10.1126/scirobotics.ade7811.
Morphological symmetries in robot learning (Ordonez-Apraez, D.; Agudo, A.; Martin, M.; Moreno-Noguer, F.), In proceedings of RSS'23 Workshop on Symmetries in Robot Learning, 2023.
On discrete symmetries of robotics systems: A group-theoretic and data-driven analysis (Ordonez-Apraez, D.; Martin, M.; Agudo, A.; Moreno-Noguer, F.), In proceedings of Robotics: Science and Systems (RSS'23), 2023.
Platform-portable reinforcement learning method to localize underwater targets (Masmitja, I.; Martin, M.; Reilly, T. O.), In proceedings of OCEANS23, 2023.
TransfQMix: Transformers for Leveraging the Graph Structure of Multi-Agent Reinforcement Learning Problems (Gallici, M.; Martin, M.; Masmitja, I.), In proceedings of 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023), 2023. DOI: 10.5555/3545946.3598825.
A Non Linear Control Method with Reinforcement Learning for Adaptive Optics with Pyramid Sensors (Pou, B.; Smith, J.; Quiñones, E.; Martin, M.; Gratadour, D.), In proceedings of SciOps 2022: Artificial Intelligence for Science and Operations in Astronomy, 2022. DOI: 10.5281/ZENODO.6567127.
A reinforcement learning path planning approach for range-only underwater target localization with autonomous vehicles (Masmitja, I.; Martin, M.; Katija, K.; Gomariz, S.; Navarro, J.), In proceedings of IEEE 18th International Conference on Automation Science and Engineering (CASE), pp. 675--682, 2022. DOI: 10.1109/CASE49997.2022.9926499.
Model-free reinforcement learning with a non-linear reconstructor for closed-loop adaptive optics control with a pyramid wavefront sensor (Pou Mulet, B.; Smith, J.; Quiñones, E.; Martin, M.; Gratadour, D.), In proceedings of Adaptive Optics Systems VIII, vol. 15, pp. 101, 2022. DOI: 10.1117/12.2627849.
Adaptive optics control with multi-agent model-free reinforcement learning (Pou, B.; Pou, B.; Ferreira, F.; Quinones, E.; Gratadour, D.; Gratadour, D.; Martin, M.), Optics Express, Vol. 30, Issue 2, pp. 2991-3015, vol. 30, no. 2, pp. 2991--3015, 2022. DOI: 10.1364/OE.444099.
An Adaptable Approach to Learn Realistic Legged Locomotion without Examples (Ordonez-Apraez, D.; Agudo, A.; Moreno-Noguer, F.; Martin, M.), In proceedings of 2022 International Conference on Robotics and Automation (ICRA), pp. 4671--4678, 2022. DOI: 10.1109/ICRA46639.2022.9812441.
Adaptive optics control with multi-agent model-free reinforcement learning (B. Pou, F. Ferreira, E. Quinones, D. Gratadour, and M. Martin ), Optics Express, Vol. 30, Issue 2, pp. 2991-3015, vol. 30, no. 2, pp. 2991–3015, Jan. 2022. DOI 10.1364/OE.444099 -
Denoising wavefront sensor image with deep neural networks (Pou, B.; Quiñones, E.; Gratadour, D.; Martín, M.), In proceedings of Adaptive Optics Systems VII, pp. 327, 2020. DOI: 10.1117/12.2576242.
Bellman residuals minimization using online support vector machines (Esposito, G.; Martin, M.), Applied Intelligence, vol. 47, no. 3, pp. 670--704, 2017. DOI: 10.1007/s10489-017-0910-7.
Approximate policy iteration using regularised Bellman residuals minimisation (Esposito, G.; Martin, M.), Journal of Experimental and Theoretical Artificial Intelligence, vol. 28, no. 1-2, pp. 351--367, 2016. DOI: 10.1080/0952813X.2015.1024494.
Approximate policy iteration with Bellman residuals minimization (Espósito, G.; Martin, M.), 17th International Conference of the Catalan Association for Artificial Intelligence, 2014. DOI 10.3233/978-1-61499-452-7-3
Learning Generalized Policies from Planning Examples Using Concept Languages (Mario Martin and Hector Geffner), Applied Intelligence 20 (1) , 9-19, 2004
Guest Editorial: Machine Learning Policies (Cortés, U.; Martín, M.), Applied Intelligence, vol. 20, no. 1, pp. 7--8, 2004. DOI: 10.1023/B:APIN.0000011235.06050.f6.
Learning generalized policies in planning using concept languages. (Mario Martin and Hector Geffner). In Proceedings of the Seventh Internatinal Conference on Knowledge Representation and Reasoning (KR 2000), Morgan Kaufmann. [Colorado (EUUU), 2000].
Learning generalized policies from planning examples using concept languages (Martin, M.; Geffner, H.), In proceedings of 3rd International Conference of the Catalan Association for Artificial Intelligence, 2000.
Reinforcement Learning for Embedded Agents facing Complex Tasks (main part) and appendixes and bibliography Ph.D. Thesis. (pdf version full thesis) Approved in June 1998.
Learning to Solve Complex Tasks by reinforcement: A New Algorithm. (Mario Martin and Ulises Cortés). Technical Report LSI-95-14-R, 1995. Also available from the LSI Technical Report Archive.
Animats Adaptation to Complex Environments as Learning Guided by Evolution. (Mario Martin, Marius Garcia and Ulises Cortés). Technical Report LSI-95-13-R, 1995. Also available as Technical Report.
Learning to Solve Complex Tasks for Reactive Systems (Extended Abstract) (Mario Martin and Ulises Cortés). Proceedings of ECML-95. LNCS, Springer-Verlag. [Heraklion (Greece), 1995].
Learning to solve complex tasks for reactive systems (Extended abstract) (Martin, M.; Cortés, U.), Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer Verlag, pp. 315--318, 1995. DOI: 10.1007/3-540-59286-5_76.
A Developmental Approach to Integrated Architectures. (Mario Martin and Ulises Cortés). In Proceedings of the Workshop on Integrated Learning Architectures in ECML-93 pp. 57 . [Wien (Austria), 1993].
Desarrollo de arquitecturas basadas en sociedades de agentes. (Mario Martin and Ulises Cortés). Proceedings of the "III Congreso Iberoamericano de Inteligencia Artificial" IBERAMIA 92, pags 349-361. [La Habana (Cuba), 1992.]

ML and Neurophysiology

In last years, I've had the opportunity to work with real data from neurological origin. That's especially interesting and somewhat ironic. Artificial Intelligence to know more about intelligence!

Supraspinal Shaping of Adaptive Transitions in the State of Functional Connectivity Between Segmentally Distributed Dorsal Horn Neuronal Populations in Response to Nociception and Antinociception (Martín, M.; Béjar, J.; Chávez, D.; Ramírez-Morales, A.; Hernández, E.; Moreno, L.; Contreras-Hernández, E.; Glusman, S.; Cortés, U.; Rudomin, P.), Sensory Adaptation, Frontiers Media SA, 2022. DOI: 10.3389/978-2-88974-179-3.
DIALCAT: Diabetes as an accelerator of cognitive impairment and Alzheimer's disease, comprehensive approach and adherence to treatment (Barrué, C.; Vila, R.; Martinez, A. B.; Martín, M.; Vázquez Salceda, J.; Simó, R.; Barroso, M.; Cortés, U.), Computación y Sistemas, vol. 23, no. 2, pp. 351--363, 2019. DOI: 10.13053/cys-23-2-3198.
Supraspinal Shaping of Adaptive Transitions in the State of Functional Connectivity Between Segmentally Distributed Dorsal Horn Neuronal Populations in Response to Nociception and Antinociception (Martín, M.; Béjar, J.; Chávez, D.; Ramírez-Morales, A.; Hernández, E.; Moreno, L.; Contreras-Hernández, E.; Glusman, S.; Cortés, U.; Rudomin, P.) In Frontiers of System Neuroscience, vol. 13, no. September, 2019.DOI 10.3389/fnsys.2019.00047 -
Supraspinal modulation of neuronal synchronization by nociceptive stimulation induces an enduring reorganization of dorsal horn neuronal connectivity (Contreras-Hernández, E.; Chávez, D.; Hernández, E.; Velázquez, E.; Reyes, P.; Béjar, J.; Martin, M.; Cortés, U.; Glusman, S.; Rudomin, P.) In The Journal of Physiology, vol. 596, no. 9, pp. 1747–1776, May 2018.DOI 10.1113/JP275228 -
Deep learning for freezing of gait detection in Parkinson’s disease patients in their homes using a waist-worn inertial measurement unit. (Camps, J.; Sama M.; Martin, M; et.alt.). In Knowledge-Based Systems 2018. -
Markovian analysis of the sequential behavior of the spontaneous spinal cord dorsum potentials induced by acute nociceptive stimulation in the anesthetized cat. (Martin, M.; Bejar, J.; Espósito, G.; Chávez , D.; Contreras-Hernández, E.; Glusman, S.; Cortes, C.; Rudomín, P.). In Frontiers in computational neuroscience vol. 11, pp. 1-32 2017. -
Deep Learning for Detecting Freezing of Gait Episodes in Parkinson’s Disease Based on Accelerometers. (Camps, J.; Sama M.; Martin, M; et.alt.). In 14th International Work-Conference on Artificial Neural Networks (IWANN) 2017. -
Kernel alignment for identifying objective criteria from brain MEG recordings in schizophrenia. (Martin, M.; Bejar, J.; Espósito, G.; Català Roig, N.; Cortes, C.; Viñas, F.; Tarragó, J.; Rojo, E.; Nowak, R.). In Pattern recognition letters 2016.
A machine learning methodology for the selection and classification of spontaneous spinal cord dorsum potentials allows disclosure of structured (non-random) changes in neuronal connectivity induced by nociceptive stimulation. (Martin, M.; Contreras-Hernández, E.; Bejar, J.; Espósito, G.; Chávez , D.; Glusman, S.; Cortes, C.; Rudomín, P.). In Frontiers in Neuroinformatics, vol. 9, no. 21, pp. 1-16 2015.
Using kernel alignment for feature selection in schizophrenia diagnostic (Martin, M.; Bejar, J.; Espósito, G.; Català Roig, N.; Cortes, C.; Viñas, F.; Tarragó, J.; Rojo, E.; Nowak, R.), 18th International Conference of the Catalan Association for Artificial Intelligence, 2015. DOI 10.3233/978-1-61499-578-4-143
A data mining methodology for event analysis in neurophysiological signals (Bejar J.;, Martin, M.; Contreras-Hernández, E.; Espósito, G.; Chávez , D.; Glusman, S.; Cortes, C.; Rudomín, P.), 18th International Conference of the Catalan Association for Artificial Intelligence, 2015. DOI 10.3233/978-1-61499-578-4-80
Intersegmental synchronization of spontaneous cord dorsum potentials as a clinical parameter to evaluate changes in neuronal connectivity produced by peripheral nerve and spinal cord damage ( Martin M.; Chávez D.; Bejar J.; Espósito, G. ; Rodríguez E.; Cortés U., Rudomín P.)International Conference on NeuroRehabilitation, 2012. DOI 10.1007/978-3-642-34546-3_91

Machine Learning

I've done some basic research in Machine learning, especially in the topics of support vector machines, transductive learning and unsupervised learning

A randomized algorithm for the exact solution of transductive support vector machines (Esposito, G.; Martin, M.). In Applied artificial intelligence, vol. 29, no. 5, pp. 459-471, 2015.
A randomized algorithm for exact transduction (Espósito, G.; Martin, M.), 17th International Conference of the Catalan Association for Artificial Intelligence, 2014. DOI 10.3233/978-1-61499-452-7-13
Feature Selection for Support Vector Machines by Alignment with Ideal Kernel (Neus Català and Mario Martin). Technical Report LSI-07-48-R. 2007.
On-Line Support Vector Machine Regression (Martin, M.), Proceedings of the 13th European Conference on Machine Learning (ECML 2002), Springer Berlin Heidelberg, pp. 282--294, 2002. DOI: 10.1007/3-540-36755-1_24..
On-line support vector machines for function approximation (Martin, M.), Technical Report LSI-02-11-R 2002 Universitat Politècnica de Catalunya, Departament de Llengatges i Sistemes Informàtics, pp. 1--11, 2002.
Development of an Intelligent Data Analysis System for Knowledge Management in Environmental Data Bases (Sànchez-marrè, M.; Gibert, K.; Rodríguez-roda, I.; Bueno, E.; Mozo, L.; Clavelle, A.; Martín, M.; Rougé, P.), In proceedings of International Environmental Modelling and Software Society, pp. 420--425, 2002.
Biasing Induction with Previous Knowledge for Knowledge Acquisition in Imprecise Domains. (Mario Martin, Ramón Sanguesa and Ulises Cortés). In Proceedings of the Eleventh International Workshopon Expert Systems & Their Applications (Avignon'91). [Avignon (Francia), 1991].
Knowledge Acquisition Combining Analytical and Empirical Techniques (Martin, M.; Sangüesa, R.; Cortes, U.), In Proceedings of the Eight International Workshop on Machine Learning (ML91). Editors: L.A. Birbaum, G.C. Collins. Published by Morgan-Kauffman. [Illinois (USA), 1991] pp. 657--661, 1991. DOI: 10.1016/B978-1-55860-200-7.50133-1.
A new approach to knowledge acquisiton: integrating analytical and empirical techniques (Mario Martin, Ramón Sanguesa). In proceedings of ICYCS'91: Proceedings of the first international conference on Young computer scientists, pp. 355 [Beijing (China), 1991]., 1991.
LINNEO: Eina per l'ajut en la construcció de bases de coneixements en dominis poc estructurats. Master Thesis, presented at the "Universitat Politecnica de Catalunya", 1991.
Metodología de adquisición de conocimientos basada en clasificaciones y caracterizaciones difusas. (Mario Martin and Ramón Sanguesa). In the book Nuevas Tendencias en Inteligencia Artificial. Editor: Anselmo del Moral, 1991.
LINNEO: Herramientas para la adquisicion de conocimientos y generacion de reglas en dominios de clasificacion y diagnosis. (Mario Martin, Ramón Sanguesa and Ulises Cortés). Proceedings of the "II Congreso Iberoamericano de Inteligencia Artificial" IBERAMIA-90. [Morelia (Mexico), 1990].
LINNEO: A Knowledge Acquisition Tool (Angels Hernández, Mario Martin, Ramón Sanguesa and Ulises Cortés). Proceedings of the 2nd Paneuropean Simposium on Applied Informatic. [Salerno (Italia), 1990].
Artificial Intelligence Versus Classic Criteria (Martin, M.), In proceedings of III Intern. Workshop on Systematics of Atlantic Mediterranean Sponges, 1989.

Learning for Information Extraction.

If have done also some research on application of semi-supervised techniques of Machine Learning to automatically extract patterns for Information Extraction in NLP

Coin_flipper at eHealth-KD challenge 2019: voting LSTMs for key phrases and semantic relation identification applied to spanish eHealth texts (Català, N.; Martín, M.), In proceedings of Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019): co-located with 35th Conference of the Spanish Society for Natural Language Processing (SEPLN 2019): Bilbao, Spain, September 24th, 2019, pp. 17--25, 2019.
A portable method for acquiring information extraction patterns without annotated corpora. (Neus Català, Nuria Castell and Mario Martin). Natural Language Engineering 9(2), 151-179, 2003.
Essence: A Portable Methodology for Acquiring Information Extraction Patterns. (Neus Català, Nuria Castell and Mario Martin). In Proceedings of the 14th European Conference on Artificial Intelligence (ECAI 2000). pp. 411--415, 2000. [Berlin, 2000].
Using ESSENCE for Acquiring Information Extraction Patterns. (Neus Català, Nuria Castell and Mario Martin). Technical Report LSI-00-65-R. 2000.

Fuzzy Sets

My introduction to research in Artificial Intelligence was through Fuzzy sets in collaboration with CNRS in Toulouse

Conceptual Connectivity by means of fuzzy partitions. (Josep Aguilar-Martin, Mario Martin and Nuria Piera). In Uncertainty Management. Editors: M. Bouchon-Mounier, R. Yager and L. Zadeh. Edited by Springer-Verlag (LNCS), 1992.
Conceptual conectivity analysis by means of fuzzy partitions (Aguilar-Martin, J.; Martin, M.; Piera, N.), Uncertainty in Knowledge Bases: 3rd International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU '90) [Lecture Notes in Computer Science 521], Springer-Verlag, pp. 165--172, 1990. DOI: 10.1007/BFb0028100.
Conceptual connectivity analysis by means of fuzzy partitions. (Josep Aguilar-Martin, Mario Martin and Nuria Piera). Proceeding of the "3rd International Procesing Management of Uncertainty", IPMU-90. [Paris (France), 1990].

Book chapters

I have also written some book chapters about AI and ML

Aprendizaje automático (Moreno, A.; Armengol, E.; Bejar, J.; Belanche, L.; Cortes, U.; Gavaldà, R.; Gimeno, J.; Martin, M.; Sànchez-Marrè, M.; López, B.), Edicions UPC, 1994.
Inteligencia artificial (Cortes, U.; Bejar, J.; Moreno, A.; Gimeno, J.; Rodriguez, H.; Belanche, L.; Martin, M.; Sànchez-Marrè, M.; Sangüesa, R.), Edicions UPC, 1993.

Mario Martin's home page