[RFC] queue_job: enqueue dependent job in jobrunner

[sbidoul]

@guewen I'm considering to move _enqueue_dependent_jobs to the jobrunner, for two reasons

• I feel it logically belong there as it is where we decide when job must start so changing for waiting dependency to pending there too feels logical
• but more importantly it should help with concurrency errors in that area such as queue_job: concurrent update exception on job state update #855

Do you have any thoughts about that?

[guewen]

It makes sense for the reasons you mention.
How would the jobrunner identify waiting jobs ready to process, is there any other way than polling?
I'm wondering if we could leverage NOTIFY payloads from the worker that ends a job, so the jobrunner could check only a subset of the jobs.

[sbidoul]

The notification payload already has the job uuid.

But thinking about it, it means if the jobrunner misses a done notification dependencies will stay waiting. Hm...