Setting up tasks for import, publish and clean up using Task Manager

This article describes how to set up tasks that start on schedule for import, publish and clean up.

This function is available in version 5.0 and later.

Overview

The Task Manager is used to schedule import and publish tasks on a daily or weekly basis. The function can trigger an action when a trigger occur. The actions can be import, publish or cleanup.

The Task Manager uses a config file to store the different tasks, and uses Windows Task Scheduler to trigger the actions. Tasks which are applied, are added to the Windows Task Scheduler and will be executed.

Tasks and actions

Trigger

Usage

Trigger

Usage

Once

Trigger an action at a defined date and time.

Weekly

Trigger an action at a defined time at selected days of the week, every week.

Montly

Trigger an action at a defined time at selected days of the month, every month.

Disabled

Disable triggers. Used when a first task triggers start of a second task., then second task has trigger: Disabled. An alternative is to set second task trigget to Once and the data to past time.

 

Action

Usage

Action

Usage

Import

Start an import in import storage. All import folders according to the selection and not previously imported will get imported.

Publish

Run publish. Only publications which has a label will be possible to run in the Task Manager.

Cleanup

Clean up content in import folders to save disk space.

General

Version 5.1 and later allows to start sql scripts using task manager.

Usage

The function is started using the Task menu option under Tools menu.

Create tasks

The Task Manager will display a list of tasks that are currently configured. Each task will have a name and a site. Enable flag allows to ensure a task is not applied in the Windows Task Scheduler. Locked flag allows to protect a task from accidental edits. The linked column shows that a task will start a next task when completed.

In the Task schedule list it is possible to create new tasks, edit and delete tasks. To save a task it has to have a unique name, and the name has to be unique to all sites.

Applying a task will reult in the task being sent to Windows Task Schedule. Before a task is applied, it cannot be active. Closing the window will apply all tasks.

Advanced option will bring up Windows Task Scheduler. Changing a task in Windows Task Scheduler will result in changes not being saved.

It is only possible to edit tasks with the same site as selected site.

Trigger

A task will have the trigger which is selected in the tab view, e.g. Weekly in the screen below.

Trigger action can be defined by setting day and time, and selecting repeat value. Repeat 1 in weekly option results in every week, while 2 results in every 2:nd week.

Tasks that are intended to be started after other tasks finish shall not have any trigger. In this case choose Disabled as trigger.

Action

Actions are used to run publish, import or general tasks. Many publish or many imports can be run in one task.

Import

Import tasks are defined in two steps.

  1. Select source folders to scan for imports that have not yet been imported. By default only one folter exist, but import.config may be used to define several source folders. This can be useful when several different systems export the same kind of data.

  2. Select what to import from these folders. The options will correspond to all installed transformers as defined in import.config.
    It is possible to select multiple transforms and the order of import will be from top to bottom. Change order by using the Up and Down buttons.

Publish

Publish tasks are defined by checking a list. All checked publications will be processed in the order top to bottom. Change order by using the Up and Down buttons.

Create LogicalIdentity (Label) in the publication repository by editing directly in the cell, the property panel do not show this attribute

General tasks

In version 5.1 and later it is possible to start General tasks. Select Action tab and then General tab and select an action. By default undefined will be selected as an option in action dropdown. General task with undefined action can not be saved and it will result a validation error. That will be reminder for the user to select a proper action.

In version 5.1 and later user can start task such as running SQL scripts at server. Scripts needs to be available at server directory. It can be configured in server.config file available at path C:\ProgramData\Signifikant\Assert . We can add directory path in configuration e.g <SQLScriptsPath>C:\ProgramData\Signifikant\Assert</SQLScriptsPath>.

After configuring the SQL scripts path in server.config we can select Action tab and then General tab and select action SqlScript. All SQL script filenames will be shown in the dropdown where user can make a selection. There is some more information shown to the user e.g script path, Api method. Api method shows what Api method will be executed in background against server to complete this task. selected SQL script will be executed at server according to how the task is set up.

 

A task may start another task when it completes. The following task will start only if current task completes successfully.

In order to start a task after successful completion, name the task in the task tab and enable the linking by checking “Link”.

Miscellaneous

When Send checkbox is checked, the Task Manager will send an email once the task is completed. Notification settings from server.config configuration are displayed to remind what settings are currently made. Edit mail settings for notifications with a Type = Task, mail settings are not possible to change here.

Cleanup

Cleanup action is used to remove import files after some time. A cleanup task will remove all import files except status information when data is older than selected number of months. The definition holds only one clean up task, and is stored under a fixed name in the Task Scheduler.

The option allows to choose if both export and import areas are to be cleaned or only import area.

For error search purposes it is good to keep export file area.

Import configuration

In import.config the different sources are defined. There are two options to differentiate between different systems exporting the same data.

  1. Each system export to a separate folder. In this case define <Sources> to set the paths to these locations.

  2. Each system export to the same folder, but uses different folder name. In this use the Source attribute in the TransformerSettings tag.

 

<XmlConfig> <!-- Default path to look for imports --> <ExportBasePath>C:\Signifikant\Export</ExportBasePath> <ImportBasePath>C:\Signifikant\Import</ImportBasePath> <StyleSheetsBasePath>C:\ProgramData\Signifikant\Assert</StyleSheetsBasePath> <Sources> <!-- Define a path where system1 exports its files --> <!-- These options will appear in the "Select sources" area of import screen --> <ImportSource name="System1"> <ExportBasePath>C:\Signifikant\SourceSiegen\Export</ExportBasePath> <ImportBasePath>C:\Signifikant\SourceSiegen\Import</ImportBasePath> </ImportSource> <!-- Define a path where system2 exports its files --> <ImportSource name="System2"> <ExportBasePath>C:\Signifikant\SourceBPCS\Export</ExportBasePath> <ImportBasePath>C:\Signifikant\SourceBPCS\Import</ImportBasePath> </ImportSource> </Sources> <InitialItemStatusList/> <ImportSetting/> <TransformerSettingList> <TransformerSetting Name="Documents" TransformType="SignifikantSimplifiedXml"> <StylesheetPath>C:\ProgramData\Signifikant\Assert\Customize</StylesheetPath> <FolderRegex>System1_Document_</FolderRegex> <SequenceNumber>1</SequenceNumber> <BatchSizeMB>2</BatchSizeMB> <BatchCounter>0</BatchCounter> <EnableEditExistingSpecificationTypes p5:nil="true" xmlns:p5="http://www.w3.org/2001/XMLSchema-instance" /> <Options/> </TransformerSetting> <!-- Source for Documents SSX import is System3. The option will appear in quick selection dialogue to make it easy to import data from system 3. --> <TransformerSetting Name="Documents" TransformType="SignifikantSimplifiedXml" Source="System3"> <StylesheetPath>C:\ProgramData\Signifikant\Assert</StylesheetPath> <FolderRegex>Documents_System3|Documents_SYSTEM3</FolderRegex> <SequenceNumber>2</SequenceNumber> <BatchSizeMB>10</BatchSizeMB> <BatchCounter>10</BatchCounter> <EnableEditExistingSpecificationTypes p5:nil="true" xmlns:p5="http://www.w3.org/2001/XMLSchema-instance" /> <Options/> </TransformerSetting> <!-- Source for Documents SSX import is System4. The option will appear in quick selection dialogue to make it easy to import data from system 4. --> <TransformerSetting Name="Documents" TransformType="SignifikantSimplifiedXml" Source="Siegen"> <StylesheetPath>C:\ProgramData\Signifikant\Assert</StylesheetPath> <FolderRegex>Documents_System4|Documents_SYSTEM4</FolderRegex> <SequenceNumber>3</SequenceNumber> <BatchSizeMB>10</BatchSizeMB> <BatchCounter>10</BatchCounter> <EnableEditExistingSpecificationTypes p5:nil="true" xmlns:p5="http://www.w3.org/2001/XMLSchema-instance" /> <Options/> </TransformerSetting> </TransformerSettingList> </XmlConfig>

 

Logging

Task Scheduler will start task actions where each action write log files to a folder defined for each task; it is a folder name on the server computer that runs Task Scheduler. In case the folder name is left empty it defaults to the folder: C:\ProgramData\Signifikant\Assert on the server computer.

Look for files named as the acting task with a date mark, “task-name_YYYYMMDD.log”. All tasks started on the same day will log to the same log file by appending text.

Adding other tasks manually to the Task Scheduler

The task Scheduler can be used for many causes, but be cautious that the Assert folder is cleared and replaced when tasks are saved from the Manager program. Create a separate folder for manually entered tasks.