Skip to content

fix(taskprocessing): claim tasks atomically so parallel workers don't duplicate #61053

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bygadd wants to merge 2 commits into
base: master
Choose a base branch
from
+25 −11
Open

fix(taskprocessing): claim tasks atomically so parallel workers don't duplicate #61053

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ee7f9c2
fix(taskprocessing): claim tasks atomically in worker via lockTask()
bygadd Jun 6, 2026
df826bb
fix(taskprocessing): tighten lockTask guard to status = scheduled
bygadd Jun 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 24 additions & 10 deletions core/Command/TaskProcessing/WorkerCommand.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,17 +165,31 @@ private function processNextTask(OutputInterface $output, array $taskTypes = [])
// Fetch the oldest scheduled task across all eligible task types in one query.
// This naturally prevents starvation: regardless of how many tasks one provider
// has queued, another provider's older tasks will be picked up first.
try {
$task = $this->taskProcessingManager->getNextScheduledTask(array_keys($eligibleProviders));
} catch (NotFoundException) {
return false;
} catch (Exception $e) {
$this->logger->error('Unknown error while retrieving scheduled TaskProcessing tasks', ['exception' => $e]);
return false;
}
$taskIdsToIgnore = [];
while (true) {
try {
$task = $this->taskProcessingManager->getNextScheduledTask(array_keys($eligibleProviders), $taskIdsToIgnore);
} catch (NotFoundException) {
return false;
} catch (Exception $e) {
$this->logger->error('Unknown error while retrieving scheduled TaskProcessing tasks', ['exception' => $e]);
return false;
}

$taskTypeId = $task->getTaskTypeId();
$provider = $eligibleProviders[$taskTypeId];
$taskTypeId = $task->getTaskTypeId();
if (!isset($eligibleProviders[$taskTypeId])) {
$taskIdsToIgnore[] = (int)$task->getId();
continue;
}
$provider = $eligibleProviders[$taskTypeId];

// Atomically claim the task; if another worker grabbed it between
// the SELECT above and now, lockTask() returns false -> skip it.
if ($this->taskProcessingManager->lockTask($task)) {
break;
}
$taskIdsToIgnore[] = (int)$task->getId();
}

$output->writeln(
'Processing task ' . $task->getId() . ' of type ' . $taskTypeId . ' with provider ' . $provider->getId(),
Expand Down
2 changes: 1 addition & 1 deletion lib/private/TaskProcessing/Db/TaskMapper.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ public function lockTask(Entity $entity): int {
$qb->update($this->tableName)
->set('status', $qb->createPositionalParameter(\OCP\TaskProcessing\Task::STATUS_RUNNING, IQueryBuilder::PARAM_INT))
->where($qb->expr()->eq('id', $qb->createPositionalParameter($entity->getId(), IQueryBuilder::PARAM_INT)))
->andWhere($qb->expr()->neq('status', $qb->createPositionalParameter(2, IQueryBuilder::PARAM_INT)));
->andWhere($qb->expr()->eq('status', $qb->createPositionalParameter(\OCP\TaskProcessing\Task::STATUS_SCHEDULED, IQueryBuilder::PARAM_INT)));
try {
return $qb->executeStatement();
} catch (Exception) {
Expand Down