Skip to content

Feature request: Job — one class with multiple ordered tasks, one live object per instance #414

Description

@pacofina

Goal

Let a single class hold several publishing tasks (validate, extract, integrate, …)
as decorated methods, with one object of that class kept alive for the whole
processing of a given instance — so shared state lives in self.<field> instead
of the untyped instance.data dict.

Motivation

An InstancePlugin runs a single process() per instance, and because a fresh
plug-in is instantiated on every call, it cannot carry state between stages. The
only channel to pass state from validation to extraction today is instance.data.

Two practical costs:

  1. A real "job" usually bundles several operations that currently force one
    plug-in class each, scattering what is conceptually one unit of work.
  2. Shared state lives in an untyped dict (instance.data["..."]) instead of
    typed fields on a class, which is both less clear and less efficient.

I think this is worth having because it makes multi-step, stateful publishing
read naturally as one class, while still compiling down to ordinary pyblish
plug-ins so nothing else in the pipeline has to change.

Suggested implementation

from pyblish.api import Job, validation, processor

class MyJob(Job):
    families = ["mesh"]

    def __init__(self, instance):
        self.instance = instance
        self.vertices = None

    @validation
    def validate(self):
        self.vertices = self.instance.data["vertices"]
        assert self.vertices

    @processor
    def process(self):
        export(self.vertices)   # reuses state from validate, via self

Sketch of how it works:

  • @task(order=...) tags a method with its order; @validation / @processor /
    @integration are sugar mapping to the CVEI bands, with an optional relative
    offset. A Job.order is added to every task as a per-Job tie-breaker; a task
    leaving its CVEI band, or landing in the collector band, raises at discovery.
  • Job.discover() expands each decorated method into a real synthetic
    InstancePlugin whose order comes from the decorator, so the whole existing
    pipeline (Iterator, matching, results, GUI) works unchanged.
  • All synthetic plug-ins of a Job share one live object per instance, kept on the
    instance itself (instance._jobs), created once and discarded with the
    instance — no registry on the class, no weakref bookkeeping.
  • Discovery is the only place the core learns about Jobs, purely by duck-typing:
    plugins_from_module expands any class exposing a discover() classmethod.
    The core does not import the job module.

This rides on a small, backward-compatible refactor of plug-in matching: two
overridable classmethods (is_host_compatible / is_instance_compatible) whose
defaults reproduce the current family/host behaviour exactly, so a Job (or any
plug-in) can resolve its own compatibility.

I already have a working implementation with tests (all green). Happy to open a
PR if this is something you'd consider.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions