Projects

• 1: Viewing Projects
• 2: Managing Projects
• 3: Managing Tables and Views
• 4: Managing Hierarchies
• 5: Managing Data Editors
• 6: Archive a Project
• 7: Viewing the Project Log

1 - Viewing Projects

Description

Within Analyze, the Projects function provides a level of compartmentalization that makes controlling access and modifying privileges much easier. Projects are what provide the primary segregation of data within a workspace tab.

While Projects fall under Analyze, workflows fall under Projects, meaning that Projects contain workflows. Workflows, simply put, perform a wide range of tasks including data transformation pipelines, data analysis, and even ETL processes. More information on workflows can be found under the “Workflows” section.

Accessing Projects

To access Projects:

1. Open Analyze
2. Select “Projects” from the top menu bar

This displays the Projects Hierarchy. From here, you will see a hierarchy of projects for which you have access. There may be additional projects within the workspace, but, if you are not an owner or assigned to the project, they will not be visible to you.

2 - Managing Projects

Searching

Searching for projects is accomplished by using the filter box in the lower left of the hierarchy. The search filter will search project names and labels for matches and show the results in the hierarchy above.

Creating New Projects

To create a new project:

1. Open Analyze
2. Select “Projects” from the top menu bar
3. Click the “New Project” button
4. Complete the form information including the “Access Control” section
5. Click “Create”

The project is now ready for updating access permissions, adding owners, and creating workflows.

Automatic Change Tracking

All changes to a project, including workflows, data editors, hierarchies, table structures, and UDFs are tracked and allow point-in-time recovery of the state. This allows for easy recovery from user introduced problems or simply copying a different point-in-time to another project for comparison.

In addition to overall tracking, projects and their elements also allow for versioning. Not only is creating a version easy, you can also merge changes from one version to another. This provides a simple way to keep track of snapshots or to create a version for development and then be able to merge those changes into the non-development version when you want.

Managing Project Access

Types of Access

Project security has been simplified into three types of access:

• All Workspace Members
• Specific Members Only
• Specific Security Groups Only

Setting the project security is easy to do:

1. Open Analyze
2. Select “Projects”
3. Click the edit icon of the project you want to restrict
4. Choose desired restriction under “Access Control”
5. Click “Update”

All Workspace Members

“All Workspace Members” access is the most simple option since it provides access to all members of the workspace and does not require any additional assignment of members.

Specific Members Only

“The Specific Members Only” access setting requires assignment of each member to the project.To assign members to a project:

1. Open Analyze
2. Select “Projects” from the top menu bar
3. Click the members icon
4. Grant access to members by selecting the check box next to their name in the “Access” column
5. Click “Update”

For clouds with large numbers of members, this approach can often require more effort than desired, which is where security groups become useful.

Specific Security Groups Only

The “Specific Security Groups Only” option enables assigning specific security groups permission to access the account. With access restrictions relying on association with a security group or groups, the administration of account access for larger groups is much simpler. This is particularly useful when combined with single sign-on automatic group association. By using single sign-on to set member group assignments, these groups can also enable and disable access to projects implicitly.

To edit assigned groups:

1. Open Analyze
2. Select “Projects” from the top menu bar
3. Click the security groups icon
4. Grant access to security groups by selecting the check box next to their name in the “Access” column
5. Click “Update”

Setting Different Viewing Roles

Many times a project may require several transformations and tables to complete intermediate steps while the end result may end up only consisting of a few tables. Members do not always require viewing of all the elements of the project, sometimes just the final product. PlaidCloud offers you the ability to set different viewing roles to easily declutter and control the visibility of each member.

There are three built-in viewing roles: Architect, Manager, and Explorer

The Architect role is the most simple because it allows full visibility and control of projects, workflows, tables, variables, data editors, hierarchies, and user defined functions.

The Manager and Explorer roles have no specific access privileges but can be custom-defined. In other words, you can choose which items are visible to each group.

You can make everyone an Architect if you feel visibility of everything within the project is needed; otherwise, you can designate members as Manager and/or Explorer project members and control visibility that way.

To set the different role:

1. Open Analyze
2. Select “Projects”
3. Click the members icon
4. Select the member you whose role you would like to change
5. Double click their current role in the “Role” column
6. Select the desired role
7. Click “Update”

Managing Project Variables

When running a project or workflow it may be useful to set variables for recurring tasks in order to decrease clutter and save time. These variables operate just like a normal algebraic variable by allowing you to set what the variable represents and what operation should follow it. PlaidCloud allows you to set these variables at the project level, which will effect all the workflows within that project, or at the workflow level, which will only effect that specific workflow.

To set a project level variable:

1. Open Analyze
2. Select “Projects”
3. Click the Manage Project Variables icon

From the Variables Table you can view the variables and view/edit the current values. You can also add new or delete existing variables by clicking the “New Project Variable” button.

Cloning a Project

When a project is cloned, there may be project related references, such as workflow steps, that run within the project. PlaidCloud offers two options for performing a full duplication:

• Duplicate with updating project references
• Duplicate without updating project references

Duplicating with updating project references means all the related references point to the newly duplicated project.

To duplicate with updating project references:

1. Open Analyze
2. Select “Projects”
3. Select the project you would like to duplicate
4. Click the “Actions” button
5. Select the “Duplicate with project reference updates” option

To duplicate without updating project references means to have all of the related references continue pointing to the original project.

To duplicate without updating project references:

1. Open Analyze
2. Select “Projects”
3. Select the project you would like to duplicate
4. Click the “Actions” button
5. Select the “Duplicate without project reference updates” option

Viewing the Project Report

When a project or workflow is dynamic, maintaining detailed documentation becomes a challenge. To help solve this problem, PlaidCloud provides the ability to generate a project-level report that gives detailed documentation of workflows, workflow steps, user defined transforms, variables, and tables. This report is generated on-demand and reflects the current state of the project.

To download the report:

1. Open Analyze
2. Select “Projects”
3. Click the report icon

3 - Managing Tables and Views

PlaidCloud offers the ability to organize and manage tables, including labels. Tables are available to all workflows within a project and have many tools and options.

In addition to tables, PlaidCloud also offers Views based on table data. Using Views allows for instant updates when underlying table changes occur, as well as saving data storage space.

Options include:

• The same table can exist on multiple paths in the hierarchy (alternate hierarchies)
• Tables are taggable for easier search and inclusion in PlaidCloud processes
• Tables can be versioned
• Tables can be published so they are available for Dashboard Visualizations

PlaidCloud uses a path-based system to organize tables, like you would use to navigate a series of folders, allowing for a more flexible and logical organization of tables. Using this system, tables can be moved within a hierarchy, or multiple references to one table from different locations in the hierarchy (alternate hierarchies), can be created. The ability to manage tables using this method allows the structure to reflect operational needs, reporting, and control.

Searching

Searching for tables is accomplished by using the filter box in the lower left of hierarchy. The search filter will search table names and labels for matches and show the results in the hierarchy above.

Move

To move a table:

1. Drag it into the folder where you wish it to be located

Rename

To rename a table:

1. Right click on the table
2. Select the rename option
3. Type in the new name and save it
4. The table is now renamed, but it retains its original unique identifier.

Clear

To clear a table:

1. Select the tables in the hierarchy ‘
2. Click the clear button on the top toolbar.

Note: You can clear a single table or multiple tables

Delete

To delete a table:

1. Select the tables in the hierarchy
2. Click the delete button on the top toolbar
3. The deleted operation will check to see if the table is in use by workflow steps or Views. If so, you will be asked to remove those associations before deletion can occur.

Note: You can also force delete the table(s). Force deletion of the table(s) will leave references broken, so this should be used sparingly.

Create New Directory Structure

To add a new folder:

1. Click the New Folder button on the toolbar

To add a folder to an existing folder:

1. Right-click on the folder
2. Select New Folder

View Data (Table Explorer)

Table data is viewed using the Data Explorer. The Data Explorer provides a grid view of the data as well as a column by column summary of values and statistics. Point-and-click filtering and exporting to familiar file formats are both available. The filter selections can also be saved as an Extract step usable in a workflow.

Publish Table for Reporting

Dashboard Visualizations are purposely limited to tables that have been published. When publishing a table, you can provide a unique name that may distinguish the data. This may be useful when the table has a more obscure name on part of the workflow that generated it, but it needs a clearer name for those building dashboards.

Published tables do not have paths associated with them. They will appear as a list of tables for use in the dashboards area.

Mark Table for Viewing Roles

The viewing of tables by various roles can be controlled by clicking the Explorer or Manager checkboxes. If multiple tables need to be updated, select the tables in the hierarchy and select the desired viewing role from the Actions menu on the top toolbar.

Memos to Describe Table Contents

Add a memo to a table to help understand the data.

View Table Shape, Size, and Last Updated Time

The number of rows, columns, and the data size for each table is shown in the table hierarchy. For very large tables (multi-million rows) the row count may be estimated and an indicator for approximate row count will be shown.

View Additional Table Attributes

To view and edit other table attributes:

1. Select a table
2. Click the view the table context form on the right.

Duplicate a Table

To duplicate a table:

1. Selecting the table
2. Click on the duplicate button on the top toolbar.

4 - Managing Hierarchies

PlaidCloud offers the ability to organize and manage hierarchies, including labels. Hierarchies are available to all workflows within a project.

PlaidCloud uses a path-based system to organize hierarchies, like you would use to navigate a series of folders, allowing for a more flexible and logical organization (control hierarchy) of the hierarchies. Using this system, hierarchies can be moved within a control hierarchy, or multiple references to one hierarchy, from different locations in the control hierarchy (alternate hierarchies) can be created. The ability to manage hierarchies using this method allows the structure to reflect operational needs, reporting, and control.

Searching

To search for hierarchies:

1. Use the filter box in the lower left of the control hierarchy
2. The search filter will search hierarchy names and labels for matches and show the results in the control hierarchy above

Move

To move a hierarchy within the control hierarchy:

1. Drag it into the folder where you wish to place it

Rename

To Rename a Hierarchy:

1. Right click on the hierarchy
2. Select the rename option
3. Type in the new name and save it
4. The hierarchy is now renamed, but it will retain its original unique identifier

Clear

You can clear a single hierarchy or multiple hierarchies.

To clear a hierarchy:

1. Select the hierarchies in the control hierarchy
2. Click the clear button on the top toolbar

Delete

You can delete a single hierarchy or multiple hierarchies.

To delete a hierarchy:

1. Select the hierarchies in the control hierarchy
2. Click the delete button on the top toolbar

The delete operation will check to see if the hierarchy is in use by workflow steps, tables, or views. If so, you will be asked to remove those associations.

Create New Directory Structure

To create a new folder:

1. Clicking the New Folder button on the toolbar

To add a folder to an existing folder:

1. Right-click on the folder
2. Select New Folder.

Mark Hierarchy for Viewing Roles

To view hierarchies by roles:

1. Click in the Explorer or Manager checkboxes

To view hierarchies that need to be updated:

1. Select the hierarchies in the control hierarchy
2. Select the desired viewing role from the Actions menu on the top toolbar

Memos to Describe Table Contents

To add a memo to a hierarchy:

1. Select the hierarchy
2. Update the memo in the right context form

View Additional Hierarchy Attributes

To view and edit additional hierarchy attributes:

1. Select a hierarchy
2. View the hierarchy context form on the right

Duplicate a Hierarchy

To duplicate a hierarchy:

1. Select the hieracrhy

2. Click the duplicate button on the top toolbar

5 - Managing Data Editors

PlaidCloud offers the ability to organize and manage data editors, including labels. Data Editors allow editing table data or creating data by user interaction.

PlaidCloud uses a path-based system to organize data editors, like you would use to navigate a series of folders, allowing for a more flexible and logical organization (control hierarchy) of the data editors. Using this system, data editors can move within a control hierarchy. Multiple references to one data editor from different locations in the control hierarchy (alternate hierarchies) can be created. The ability to manage data editors using this method allows the structure to reflect operational needs, reporting, and control.

Searching

To search for data editors:

1. Use the filter box in the lower left of the control hierarchy

The search filter will search data editors’ names and labels for matches and show the results in the control hierarchy above.

Move

To move a data editor within the control hierarchy:

1. Drag it into the folder where you wish to place it

Rename

To rename a data editor:

1. Right click on the data editor
2. Select the rename option
3. Type in the new name and save it

The data editor will now be renamed but retain its original unique identifier.

Delete

You can delete a single data editor or multiple data editors.

To delete a data editor:

1. Select the data editors in the control hierarchy
2. Click the delete button on the top toolbar

Create New Directory Structure

To add a new folder to the control hierarchy:

1. Click the New Folder button on the toolbar

To add a folder to an existing folder:

1. Right-click on the folder
2. Select New Folder

Mark Hierarchy for Viewing Roles

The viewing of data editors by various roles:

1. Click in the Explorer or Manager checkboxes

To update multiple data editors:

1. Select the data editors in the control hierarchy
2. Select the desired viewing role from the Actions menu on the top toolbar

Memos to Describe Table Contents

To add a memo to a data editor:

1. Select the data editor
2. Update the memo in the right context form

View Additional Hierarchy Attributes

To view and edit additional data editor attributes:

1. Select the data editor and view the data editor context form on the right

Duplicate a Data Editor

To duplicate a data editor:

1. Select the data editor
2. Click on the Duplicate button on the top toolbar

6 - Archive a Project

Creating an Archive

Projects normally contain critical processes and logic, which are important to archive. If you ever need to restore the project to a specific state, having archives is essential.

PlaidCloud allows you to archive projects at any point in time. Creation of archives complements the built-in point-in-time tracking of PlaidCloud by allowing for specific points in time to be captured. This might be particularly useful before a major change or to capture the exact state of a production environment for posterity.

Full backup: This includes all the data tables included in a project. The archive may be quite large, depending on the volume of data in the project.

Partial backup: This can be used if all of the project data can be derived from other sources. If this is the case, it is not necessary to archive the data in the project and have it remain elsewhere. Partial archives save time and storage space when creating the archive.

To archive a project:

1. Open Analyze
2. Select the “Projects” tab

Restoring an Archive

Once you have an archive, you may want to restore it. You can restore an archive into a new project or into an existing project.

To restore an archive:

1. Open Analyze
2. Select the “Projects” tab

Archiving Schedule

Archives can also serve as a periodic backup of your project. PlaidCloud allows you to manage the backup schedule and set the retention period of the backup archives to whatever is most convenient or desired.

Since all changes to a project are automatically tracked, archiving is not necessary for rollback purposes. However, it does provide specific snapshots of the project state, which is often useful for control purposes and/or having the ability to recover to a known point.

To set an archiving schedule:

1. Open Analyze
2. Select the “Projects” tab
3. Click the backup icon
4. Choose a directory destination in a Document account
5. Choose the backup frequency and retention
6. Choose which items to backup
7. Click “Update”

7 - Viewing the Project Log

Viewing and Sorting the Project Log

As actions occur within a project, such as assigning new members or running workflows, the Project Log stores the events. The Project Log consolidates the view of all individual workflow logs in order to provide a more comprehensive view of project activities. PlaidCloud also enables the viewer to sort and filter a Project Log and view details of a particular log entry.

To view the Project Log:

1. Open Analyze
2. Select “Projects”
3. Click the log icon

To sort and filter the Project Log:

1. Click the small icon to the right of the log and to the left of the “log message”
2. Select desired guidelines

To view details of a particular log entry:

1. Right click on the desired log entry
2. View the “Log Message” box for details

Clearing the Project Log

Clearing the Project Log may be desirable from time to time

To clear the Project Log:

1. Open Analyze
2. Select “Projects”
3. Click the log icon
4. Click the “Clear Log” button