Advanced Workflows (Visual Workflow Designer)

Description

An Advanced workflow runs on a visual canvas — the Visual Workflow Designer — where steps are nodes in a directed graph (a DAG). The lines between nodes show how data flows, and the runtime executes them in dependency order rather than top-to-bottom. Branches that don’t depend on each other run in parallel automatically.

Advanced is one of PlaidCloud’s workflow types; the others are Standard. The capabilities in this guide — the visual canvas, breakpoints, containers, Run From Here, Simulate, and the docked Inspector — are all Advanced-only. The canvas also supports real-time collaboration: several people can open the same Advanced workflow and edit it together, seeing each other’s presence and changes live.

Standard vs Advanced

Every workflow has a type, chosen when you create it:

Type	Steps are arranged in…	…and execute
Standard Serial	the Steps list	top to bottom, one at a time
Standard Parallel	the Steps list	from the list, in parallel where dependencies allow
Advanced (DAG canvas)	a visual graph of nodes and connections	in dependency order — independent branches run in parallel automatically

Standard and Advanced run on different engines: a Standard workflow executes from its Steps list, while an Advanced workflow executes from the graph you draw, following the explicit producer/consumer connections between steps. That’s why the Visual Workflow Designer opens only for Advanced workflows — arrows drawn on a Standard workflow wouldn’t change how it runs.

Choose the Workflow Type

You can set the type when creating a workflow, or convert an existing Standard workflow:

At creation — the workflow type selector offers Standard Serial, Standard Parallel, Advanced (DAG canvas), and Macro (coming soon). New workflows default to Standard Serial.
Convert an existing workflow — select a Standard workflow in the Workflows list and choose Convert to Advanced…, then confirm. Your steps, their configuration, and their dependencies are preserved — only how the workflow is displayed and executed changes.

The Canvas

Each step is a node, and the connections between nodes define the order steps run in. The Designer lays steps out automatically, and you can rearrange them freely.

Who’s Here

When more than one person has the same Advanced workflow open, a row of avatars appears in the canvas toolbar showing who else is viewing or editing it — the same live-presence idea as a shared document. Each person is shown by their gravatar (or their initials if they have none), and hovering an avatar shows their name. A person appears once no matter how many browser tabs they have the workflow open in, and when there are more viewers than fit, the extras collapse into a +N marker.

Step Tiles

Each tile summarizes its step at a glance, top to bottom:

Header — the step type’s icon and name, plus a status icon showing the step’s last run result.
Inputs — an icon for each table, dimension, or file the step reads. Click a table or dimension icon to open it, or a file icon to download it. A step with many inputs or outputs collapses the extras into a +N marker.
Step name — centered in the body of the tile.
Outputs — icons for the step’s outputs, with the same click-to-open behavior. Once the output table exists, this row also shows its row and column counts (for example, 12,345 rows · 8 cols) — the same Rows and Cols shown in the step’s Inputs/Outputs tables — so you can watch data volumes move through the workflow without opening each table. Until then (or when the step outputs a file or dimension instead), the row shows a plain Outputs label. Hover any table icon for its name and counts.
Footer — the step’s last and average run durations, tinted when the latest run was unusually slow compared to its history.

Navigate

Zoom in / Zoom out, Reset zoom to 100%, and Fit all nodes to view frame the workflow at any size.
Pan tool — when active, dragging pans the canvas instead of selecting. Middle-click drag and Space+drag pan regardless of the toggle.
Snap to grid rounds node positions to a fixed grid when you drop them, for tidy alignment.

Minimap

For large workflows that don’t fit on screen, toggle the Minimap to dock an overview panel in the bottom-right corner. It shows the whole workflow’s steps and connections in miniature, with a rectangle marking the part of the canvas you’re currently viewing.

The viewport rectangle moves live as you pan and zoom, so you always know where you are in the larger diagram.
Click or drag anywhere in the panel to jump the canvas to that part of the workflow.
While a workflow runs, each step in the minimap is color-coded by its run state (running, done, error, and so on), so the panel doubles as an at-a-glance health overview.

Lay Out

Tidy Layout (auto-arrange) re-runs the automatic left-to-right layout. It overwrites the current positions but preserves connections, notes, and highlights.
Undo and Redo step backward and forward through layout changes (Cmd/Ctrl+Z and Cmd/Ctrl+Shift+Z), and History opens the panel of changes.

Annotate

Annotations are visual only — they document the diagram and never affect execution.

Add Note — drop a sticky note anywhere on the canvas, then Edit Text… to write in it (or Delete Note to remove it).
Add Highlight — draw a translucent box around a group of related steps and give it a label with Edit Label… (or Delete Highlight to remove it).
Color — color-code a step (or reset it to Default) to group work visually.

Lock

A workflow can be locked to prevent accidental edits. Click the lock toggle to switch between Workflow editing — click to lock and Workflow is locked — click to unlock for editing. While locked, the canvas is read-only until you unlock it.

Collapse and Resize the Palette and Inspector Panes

The step palette on the left and the Inspector on the right can each be collapsed out of the way when you want the whole window for the diagram.

Collapse a pane — click the chevron in the palette heading or in the Inspector header. The pane shrinks to a narrow rail with its title running down the side. The collapsed Inspector rail shows the name of the step you have selected, so a full-width canvas doesn’t lose track of what you were inspecting; hover it for the full name if it’s clipped.
Bring it back — click anywhere on the rail, not just the chevron.
Keyboard — press [ to collapse or expand the palette and ] to collapse or expand the Inspector. Both are ignored while you’re typing in a field, so they never interrupt the filter box or an inline rename.
Double-click a divider — double-clicking the divider beside a pane collapses or expands it, the same as the chevron.
Resize — drag either divider. Both panes have a minimum width; below it the Inspector’s Inputs and Outputs cards scroll sideways rather than clipping their names and row counts.

Export the Diagram

The toolbar’s export button saves a picture of the canvas for documentation, review, or sharing. Clicking it exports a PDF; its dropdown offers Export as PDF and Export as PNG.

Dark Theme

The canvas follows the PlaidCloud theme you have chosen. In the Dark theme the canvas background, step tiles, connectors, notes, highlights, containers, the minimap, and the step palette, Inspector, and Data Preview drawer alongside it are all drawn for a dark background, with run-status colors and label text chosen to stay legible against it. Switching themes reloads the interface, and the canvas picks up the new colors when it reopens.

Add, Connect, and Edit Steps

Click a step type in the palette to drop it on the canvas, or drag it where you want it. To connect steps, drag from one step to another to draw a connector — this is what tells the runtime that one step’s output feeds the next.

Right-click any step for:

Edit Step Configuration… — the step’s settings form.
Edit Step Details… — name, memo, error handling, retry, and conditions.
Convert Step Type… — change the step to a different operation type.
Duplicate Step… — open the new-step form pre-populated from this step.
Enable Step / Disable Step — a disabled step is skipped at run time.
View Step Inputs / View Step Outputs — open the step’s data in the Inspector.
Preview Output Data — open the step’s output in the docked Data Preview drawer. Hovering a step also fades in a browse glyph that does the same thing.
Color — apply or clear a step color.
Delete Step… — removes it from the workflow structure. Downstream steps that depended on its output will need to be reconfigured.

Sub-Workflow Concurrency

When a step invokes another workflow — a Run Workflow, Run Workflow Conditional, or Run Workflow Loop step pointing at a workflow that is not a Macro — only one invocation of that sub-workflow can run at a time. A non-Macro sub-workflow shares a single run state, so two invocations at once would collide.

This matters on the Advanced canvas because steps that aren’t ordered relative to each other run in parallel. If two such steps both invoke the same non-Macro sub-workflow, they race.

The designer checks for this when you save the canvas and shows an advisory toast titled Workflow Advisory that names the offending steps and the shared sub-workflow — for example:

Steps ‘Build’, ‘Refresh’ all invoke non-macro sub-workflow ‘Load Dims’ but are not ordered relative to each other; they may run concurrently and collide (a non-macro sub-workflow can’t run twice at once). Order the steps so they don’t overlap, or make ‘Load Dims’ a macro (macros run with per-invocation isolation).

To clear it:

Order the steps so they don’t overlap — draw a connector so one runs after the other.
Make the sub-workflow a Macro — each Macro invocation runs in its own isolated scratch schema, so the same Macro is safe to call concurrently. See Create a Macro and the Macro Run reference.

The Step Palette

The Step Palette on the left of the canvas holds every step type you can add, filed in ten folders: Import; Export & Archive; Tables; Dimensions; Allocations; AI, ML & Text; Spatial; Documents & Reports; Workflow & Variables; and External Systems. Folders start closed. Click one to open it; a few hold sub-folders, such as Sage under Import and Notifications under Workflow & Variables. The palette remembers which folders you left open.

Folder Shading and Indent Guides

The palette shows where you are in the hierarchy without you having to remember it:

Folders are shaded by depth, so a top-level folder, a sub-folder, and the steps inside them are distinguishable at a glance. The shading is a secondary cue only — the folder icon and bold caption already mark a folder.
Each open folder draws a vertical indent guide down the left of everything inside it, and clicking a guide closes the folder it belongs to. That’s the quick way out of a long open folder whose own heading has already scrolled past. Hovering a guide shows Collapse this folder.
Folders open and close with the same +/− box used on the dimension screens, so the gesture reads the same way across the product.
The heading of the folder you are reading stays pinned to the top of the list while you scroll, so the steps under your pointer are never anonymous. Inside a sub-folder, its heading and its parent’s are both pinned.
Opening or closing a folder keeps your scroll position instead of throwing you back to the top of the list.
Expand all folders and Collapse all folders are the two small buttons in the palette heading, next to the chevron that collapses the palette itself.

There are two ways to add a step:

Click it. The row flashes to confirm the add, and the step appears near the top-left of the visible canvas, already selected, so the Inspector shows its details. Drag it to where you want it. Clicked the wrong one? Press Delete and confirm — it’s still selected, so there’s nothing to hunt for.
Drag it from the palette onto the canvas, and it lands where you drop it.

Two more things worth knowing:

Filter… searches the whole catalog by step name, whatever is open or closed. Each match is listed under the heading of the folder it came from, so you can see where a step belongs without opening anything. When nothing matches, the palette says No steps match.
The star beside a step pins it to Favorites, a section that sits above the folders and is never filed inside one. Right-click a step for the same Add to Favorites / Remove from Favorites, plus Add to Canvas.

The palette can be driven entirely from the keyboard, and it is navigable with a screen reader: it is exposed as a tree, and each row reports whether it is a folder, whether that folder is open, and how deep it sits. With a filter typed, the results are exposed as the flat list of matches they are.

Tab moves from the Filter… field into the list — one stop for the whole list, not one per step — and Tab again leaves for the canvas. Inside the list:

Key	What it does
Down / Up	Move to the next or previous row
Right	Open the folder you are on — or, on a step, move down
Left	Close the folder you are on — or, on a step, jump up to the folder holding it
Enter or Space	Add the step you are on to the canvas, or open and close the folder
Esc	Return to the Filter… field

Run Controls

The canvas runs the whole workflow or any part of it. Because execution follows the graph, “from here” and “selected” honor dependencies rather than list position.

Action	What it runs
Run Workflow	The entire workflow from its starting steps.
Run This Step	Only the selected step.
Run From Here	The selected step and every downstream step that would normally run after it.
Run Selected	Only the selected steps — they fire in parallel and the runtime sequences them by their dependencies.
Run Section	The selected steps plus every step the graph places between the topologically earliest and latest of your selection.

While a workflow is running you can Pause (in-flight steps finish; new steps wait until you Resume), Resume a paused or stopped workflow from where it left off, or Stop (in-flight steps finish; queued steps are cancelled). Resume re-runs the node that failed and everything still waiting on it, leaving completed nodes alone — see Pause, Stop, and Resume.

Simulate

Simulate walks the workflow’s graph without running the actual transforms. Steps paint as they would during a real run, so you can visualize the order and branches before committing to compute. Nothing executes and no data changes.

Breakpoints

A breakpoint pauses a run when it reaches a step, so you can inspect upstream output before the rest of the workflow continues.

Right-click a step and choose Set Breakpoint (or Clear Breakpoint to remove it).
Run the workflow. When execution reaches a step with a breakpoint, the run pauses there; everything downstream waits.
Inspect the step’s inputs and outputs in the Inspector, then Resume to continue.

Breakpoints are saved with the workflow, so a breakpoint you set persists across sessions until you clear it.

Containers

A container groups related steps into a labeled box you can collapse or disable as a unit — useful for organizing large workflows or toggling a whole sub-process on and off.

Group into Container — select the steps, then group them and give the container a name.
Collapse Container / Expand Container — fold the container down to a single tile to declutter the canvas, or open it back up.
Disable Container / Enable Container — disabling a container skips all of its member steps in one action. Enabling restores them.
Rename Container… — change its label.
Ungroup Container — remove the container; its member steps stay on the canvas.

Execution Containers

A plain container is a visual grouping — it tidies the canvas but doesn’t change how the steps run. Turn a container into an execution container and it also becomes a unit of execution: the whole group runs as one sealed sub-workflow, behaving as a single node in the parent graph.

Turn it on — right-click the container and toggle Execution on. The container is redrawn with a distinct border and a labeled entry port and exit port, so you can tell an execution container from an ordinary group at a glance.

One edge into the entry port feeds the sealed group; the exit port releases to downstream work only once every member has finished.

How it runs — a sealed sub-DAG. Once a container is marked for execution, its boundary is sealed in both directions:

Nothing inside starts until the container’s inputs have arrived. Every member step waits for whatever feeds the group, even a member that has no upstream of its own.
Nothing downstream of the container starts until every member has finished. The group’s exit releases only when the last member is done, so the parent workflow treats the whole container as one node.

Independent branches inside the container still run in parallel, exactly as they would on the open canvas — the seal governs only the group’s boundary, not the order of work within it.

Wire an upstream step to the entry port. Drag a connector from an upstream step onto the container and drop it on the entry port. That one edge feeds the whole group — every member that needs the input picks it up once the container’s inputs arrive.

To connect the container’s output, draw a connector from a member step to the downstream step that consumes it, the way you connect any two steps. The runtime routes that connection out through the container’s exit port for you.

Failure behavior. A member that fails follows the same error handling as any other step:

Under the default stop on error, a member failure halts the run, and everything past the container is held.
Under continue on error, the failed member is treated as complete; the container’s exit still releases once its remaining branches finish, and downstream work proceeds.

Disabling an execution container. Disabling an execution container skips it as a unit — its members don’t run, and because a skipped container produces no output, everything downstream of the container is skipped too rather than running against absent input. This is different from disabling a plain (non-execution) container, which only skips the member steps; downstream steps that don’t depend on those members still run. Re-enable the container to restore the whole group.

Running and Monitoring Execution Containers

An execution container is also something you run and watch as a unit. You can start just its work, follow a rolled-up status while it runs, and jump straight to a failure inside it — on the canvas and in the Run Monitor alike.

Run Group / Run Group + Downstream. Right-click an execution container to run it without touching the rest of the workflow:

Run Group runs the container’s member steps.
Run Group + Downstream runs the members and everything downstream of the container.

Neither re-runs the upstream steps that feed the container — they run the group as it stands, using the input already in place. Both use the same Run Selected mechanism as individual steps, so members fire in parallel and the runtime sequences them by their dependencies.

Live status on the group. While a workflow runs, the container shows a rolled-up status of its members: a status dot, an N of M done count, and a colored border. The status reflects the members as a whole — error if any member failed, running while any member is running, done once all members finished, skipped when the group was skipped, and so on. A collapsed container shows the same aggregate as its single tile, so folding a group away doesn’t hide how its work is going.

Jump to the failure. When a group has a failed member, click its status to select and center that member on the canvas — no hunting through the group to find what stopped the run.

Retry from Failure. Right-click a failed execution container and choose Retry from Failure to re-run only the failed member (or members) and whatever is downstream of them inside the group; members that already succeeded are skipped. This is the difference from Run Group, which re-runs every member regardless. Retry from Failure is the quicker path back to green after you’ve fixed what a member tripped on.

Last-run time. The container header shows the summed last-run duration of its members, so you can read the whole group’s cost at a glance rather than adding up the members yourself.

Port tooltips. Hover an execution container’s entry or exit port to see the tables crossing that boundary — the inputs the group draws in, or the outputs it hands downstream — listed in the tooltip.

Iterating a Container

An execution container can run its sealed sub-DAG once per iteration of a set of values — so a group that loads and transforms one region’s data can process every region in turn without you copying the steps once per region. Right-click an execution container and choose Container Settings…, then the Iterate Over tab, and tick Iterate this execution container to turn iteration on (the settings appear only on execution containers, since iteration drives a group that runs as a unit). Iteration is optional — leave the box unticked and the container runs once, so a container can carry conditions on their own without any iterate settings.

Each pass binds the loop variable to the next value, then runs the whole sealed group — in order, one pass at a time.

Configure the loop. The Container Settings… dialog’s Iterate Over tab uses the same iteration model as the Workflow Loop step, on tabbed sub-pages. Pick how the iteration set is built:

Specify Variable Values to Iterate Over — define the values yourself on the Manual Value Iteration tab. Add one or more loop variables, and for each give either an explicit list of values or a range:

Range type Iterates once per…

Integer number from a start, end, and step

Upper alpha / Lower alpha letter in a range (A–Z or a–z)

A range can carry a list of excluded values to skip. When you define more than one variable, the container runs once per combination of their values (the cross product).
Use Combination of Unique Values in Table — pick a source table on the Iterate Over tab, then on the Data to Variable Mapping tab map one or more of its columns to loop-variable names. The container runs once per unique combination of those columns’ values. Use the Table Data Filters tab to restrict which rows are considered.

Range type	Iterates once per…
Integer	number from a start, end, and step
Upper alpha / Lower alpha	letter in a range (`A`–`Z` or `a`–`z`)

Steps inside the container reference each loop variable like any workflow variable — {region} — so a member’s SQL, expressions, table names, and file paths can all vary by iteration. Optionally turn on Combine all iterations’ output to accumulate each pass’s results instead of keeping only the last — see the note below on where that stands.

How it runs. Iterations run serially, one after the next — the members share the loop variables, so two passes at once would read each other’s values. Each iteration binds the loop variables to the next value (or combination) and then runs the whole group as an ordinary execution-container run; the next iteration starts only when the previous one finishes. A container is capped at 1,000 iterations.

What downstream reads. Because each pass overwrites the group’s output tables, a step downstream of the container reads the last iteration’s output. If you need every iteration’s rows, turn on Combine all iterations’ output so the passes accumulate rather than replace.

For example, to build one filtered extract per region, add a loop variable region with a list of values East, West, and give a member step a filter that reads the variable:

SELECT *
  FROM sales
 WHERE region = '{region}'

The container runs twice — region = 'East', then region = 'West' — running the whole sealed group each time.

Container Conditions

An execution container can also carry conditions that gate whether the whole container runs — the same variable, table, document, and time checks a step’s conditions use, so the two work the same way. Right-click the container, choose Container Settings…, and open the Conditions tab — the first tab, where a step’s conditions sit: it’s the same Conditions UI a step uses, unchanged. Conditions need no iteration, so a container can carry them on their own with Iterate this execution container left off.

Evaluated once, at entry. A container’s conditions are checked a single time, when the container is reached in the graph — not once per member, and for an iterating container, not once per iteration either. If the conditions pass, every member step runs as usual. If they fail, no member runs; the container produces no output, so everything downstream of it is skipped too, the same as a disabled container.

Skip-only — there’s no “go to.” A step’s failed condition can send the run to a named step instead of just falling through; a container’s can’t. The “go to step” picker a step offers is hidden for a container, so a failed container condition always means skip the container and everything downstream of it — there is no alternate landing point to choose.

How it composes:

Disable wins. A container you’ve disabled is skipped regardless of its conditions — disabling is a stronger, unconditional skip that doesn’t evaluate them at all.
Iteration is gated, not looped. A container that both iterates and has conditions checks the conditions once, before the first iteration, to decide whether the loop runs at all. A failed condition skips every iteration, not just the first; a passing one is not re-checked pass to pass.
Resume keeps the decision. Resuming a run that skipped a container this way doesn’t re-evaluate the condition — the container stays skipped, like any other completed decision from before the pause.
A broken condition fails the run. An error while evaluating a condition — a misconfigured check, for example — fails the run with that error. It is not treated the same as a failed condition, so it doesn’t silently skip the container.

The Inspector

The docked Step Inspector shows everything about the step you select:

Inputs and Outputs — the data flowing into and out of the step. The eye icon on an output card opens the Data Preview drawer for a quick look at the table’s first rows.
Run Stats — the step’s Last Run result, Last Duration, run count, and timing summaries including the Average and p95 durations, plus the Last Run Error or Last Run Warning when there is one.
Edit Step Configuration… and Edit Step Details… — jump straight to the step’s forms, and rename the step inline by clicking its name.
Rollback Step Config (Flashback) — restore the step’s configuration to an earlier saved version.
View Run History — open this step’s full run history.

Select a single step to inspect it; select several and the Inspector points you to the bulk actions in the canvas toolbar.

The Inspector is resizable — drag the divider between it and the canvas — and collapsible: the chevron beside the step name (or the ] key) folds it down to a narrow rail that keeps showing the selected step’s name down its side, so you can hand the width back to the canvas without losing track of what you had selected. Click anywhere on the rail to open it again. See Collapse and Resize the Palette and Inspector Panes.

Run History

Choose View this workflow’s run history to see past runs with summary statistics. The canvas also paints a heatmap from recent run records, so frequently failing or slow steps stand out at a glance.

View Logs

Click View Logs in the toolbar to jump straight to the project’s Log view, already filtered to this workflow — the full run log you would otherwise reach from the Project area. See Viewing Workflow Log.

Next Steps

Preview Step Data — browse any step’s output in the docked Data Preview drawer
Viewing Workflow Log — read a workflow’s full execution log
Run a workflow — running a workflow end to end
Managing step errors — debugging failures
Upcoming runs calendar — see when scheduled workflows will run