A scientific review published on 28 July 2026 analyses 130 studies on reinforcement learning for robots performing long tasks. It shows strong growth in research, but also a concentration of work on certain difficulties.
Stacking blocks, assembling several parts or tidying a kitchen all require a series of actions carried out in a precise order. For robots, learning these manipulations remains complex. Researchers Matthew Acs and Xiangnan Zhong examined the scientific work on this problem in a review published in the journal Discover Robotics, following the PRISMA-ScR methodological guidelines.
What makes long, multi-step manipulation tasks hard for robots
Long-horizon robotic manipulation refers to tasks whose success depends on a sequence of interdependent actions, skills or physical contacts. It covers in particular the rearrangement of multiple objects, assembly, tool use and some household activities.
Part of the difficulty lies in how robots learn. With reinforcement learning, a system adjusts its behaviour based on the rewards it receives from its interactions, rather than only following instructions programmed in advance.
When a reward comes only at the end of a task, the system has to work out which earlier decisions contributed to its success. The longer the sequence of actions, the harder this attribution becomes.
Exploring new strategies also becomes costly when rewards are rare. At the same time, the robot has to coordinate perception of its environment, action planning and execution, across different time scales.
Robot reinforcement learning research grew from 5 to 41 papers a year between 2020 and 2025
The researchers identified 130 studies published between 2020 and 2026 on reinforcement learning applied to long-horizon robotic manipulation.
Their search covered five scientific databases: IEEE Xplore, ACM Digital Library, Scopus, Web of Science and PubMed. An initial search, covering January 2020 to October 2025, identified 1,316 references. After duplicates were removed and eligibility criteria applied, 104 studies were retained. An update carried out through June 2026 added 26 more.
According to Matthew Acs and Xiangnan Zhong, the annual number of publications rose from 5 studies in 2020 to 41 in 2025, more than an eightfold increase. Growth accelerated between 2020 and 2023, before levelling off in 2024.
The work is concentrated mainly in specialist robotics conferences and publications:
- International Conference on Robotics and Automation (ICRA): 22 publications.
- IEEE Robotics and Automation Letters: 16 publications.
- Conference on Robot Learning (CoRL): 10 publications.
- International Conference on Intelligent Robots and Systems (IROS): 9 publications.
The authors also note that journal publications, initially few in number, had almost caught up with conference papers by 2024. They interpret this shift as a sign that the field is maturing.
Research focuses on only part of the robot manipulation problem
Beyond the growth in publications, the authors find a concentration of scientific effort on certain aspects of the problem, while other difficulties linked to long manipulation sequences remain relatively little studied.
This finding is directly relevant to robots intended for assembly work or household tasks. Their learning involves handling intermediate goals, dependent actions and results that sometimes can only be assessed after several manipulations.
To define the scope of their analysis precisely, the researchers excluded work focused mainly on navigation, locomotion, aerial robots and coordination between multiple agents. Research centred on large language models, foundation models or vision-language-action systems was also excluded when reinforcement learning was not its main contribution. Isolated grasping tasks and short reactive control were treated the same way.
Limitations of the review, according to its authors
Matthew Acs and Xiangnan Zhong state that their review was not pre-registered. They also specify that the selection and classification of studies did not involve a formal independent double assessment, nor statistics measuring agreement between reviewers.
Some classification decisions therefore rest on the individual interpretation of the researchers. This methodological feature should be taken into account when assessing the distribution of the work identified and the research areas described as less explored.