User Tools

Site Tools


beewm:devel:workflow_dataset_combination

Workflow Dataset Combination

:!: This documentation needs to be updated to describe the current implementation

Motivation

Assigning Datasets to a Workflow.

When starting a workflow, assign:

  • choose one CP1 pipeline (from storage)
  • choose a Shading Correction Model
  • choose an Object Classification Model
  • choose N plates (from storage)

Possible solution

From the specification of the workflow we can extract the required datasets and their types. Then we can display a data set selector for each dataset type. The provided dataset selector can be configurable by the administrators or users (look at the example data-selector from iPortal), but in the end it would provide one or more dataset codes for a given input dataset of the workflow.

Example:

If the workflow definition contains three input datasets. Then the portal after parsing this definition and identifying the inputs and their types can generate three data-selectors (it can be on a large page or it can generate a wizard like layout and show one data-selector on one page). Each data-selector would show only dataset with the appropriate type, eg.: if the workflow needs a raw image dataset a shading correction model and a cp pipeline, then the portal would first show raw image dataset for selection then shading correction models then finally the cp pipelines from the storage.

See also:

NOTE

For selecting the raw image datasets we need multiple selection, because usually users would run the same processing on several plates, but for selecting the “parameter” like datasets as models we need single selection. This can be a configuration parameter of the gui (in this way the multiple selection would be enabled per dataset type). With the multiple selection option we don't necessary put interface requirements to the workflow manager. The portal code can still submit several request per raw image, but of course if the workflow manager has option submit a list as an input dataste like “plate = ds1, ds2, ds3” to start a parameter sweep like processing on them with the same workflow that's also possible. The gui could support different data-selectors, like a big table with all the data set properties or a space and project chooser which then display the datasets only under a given project or a three structure like chooser. If we implement them according some abstraction then later we could easily provide several selectors, or replace the old ones.

beewm/devel/workflow_dataset_combination.txt · Last modified: 2016/05/17 16:17 by 127.0.0.1