Import Service is an automatic function to import Xml file data into Signifikant databases. It is run as a Windows Service and is always active on a server. The same server may be available in the Signifikant Server and in the Signifikant Web Viewer, but with separate configuration.

The service acts on new files, Files and handles them in a pipe line - files are transformed into internal data format - data is uploaded to a remote server or stored locally.

Xml file transformations are configurable through scripts, in case no further programming is needed to convert a new custom Xml file format into the internal format accepted for upload and stored into a database. The remote server Api is called using REST calls, at HTTP port 80, to reduce the need to setup a network to access remote server.

The service do not import documents, images, illustrations. video or other media data. It is intended to update Signifikant databases, either the editor database or the server database.

This service is available in version 5.1 and later

Overview

The import process require a configuration of folders, filenames, transformers, handlers, logging etc,. Once this is setup the service is working automatically to handle new data.

Data flow and process:

A new file is generated on a predefined disk area by an external system
The service supervises the disk area and detects the new file, the file name is compared to a name pattern to determine if the file is handled and how it is handled
The import handler is called to transform the file to an internal format suitable for upload
The items loaded to the importer is passed to the uploader
The uploader handler calls a remote server Api, passing a chunk of items
The remote Api will handle the data and store it to the server, the REST call indicates what interface is called and type of the data to store

Installation

Run the installer.
Possibly register the service, see Windows Service installer finishes correctly but service is missing.
Ensure Signifikant Service is run under “Local System” or an account which has access to SQL Server.

Configuration

Imported data

Currently we handle part prices, part availability, organisations and customers - all is imported to the Server database on a remote server. It is also possible to import part information into a Manager site.

The service configuration is in: C:\ProgramData\Signifikant\Assert\importer-service.config. Each configuration is defined in a CustomFolderSettings tag and, if it is to upload to a web server, refers an UploaderId depending on upload action.

UploaderId	Usage

UploaderId	Usage
PartsPrice	Import prices for parts per price list into Server database.
PartsBalance	Import availability for parts per warehouse into Server database.
Parts	Import ReplacementCode and AvailabilityCode on parts.
Organizations	Import organizations and addresses for organizations into Server database.
Users	Import users into Server database.
AssetUpload	Import assets into Asset database.
Invoice	Import invoices into Server database.

Sample configuration

<?xml version="1.0" encoding="utf-8"?>
<CustomFolderConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
	<Enabled>true</Enabled>
	<!-- In case SaveItem is true, importer service will save imports directly to the database. In this case importer service will 
	need to know where to find the server.config file since it may be either in AppData or in Program data depending on if importer 
	service is installed on Manager Server or on Web Server. -->
	<ServerConfigPath>C:\ProgramData\Signifikant\Assert\server.config</ServerConfigPath>
	<!-- Log files placed here. -->
	<LogFolder>C:\Signifikant\Log\ImportLog</LogFolder>
	<!-- File watcher will be recreated every x hrs. Default is 0 which means no recreation. Setting available from version 5.1.7 and later. -->
	<SuperviseHours>48</SuperviseHours>
	
	<Settings>
		<CustomFolderSettings ImporterId="Parts">
			<DumpUploadItems>true</DumpUploadItems>
			<FolderDescription>Part status</FolderDescription>
			<FolderEnabled>true</FolderEnabled>
			<FolderFilter>Parts*.xml</FolderFilter>
			<FolderPath>C:\Signifikant\Import_service</FolderPath>
			<BackupFolder>C:\Signifikant\Import_service_backup</BackupFolder>
			<FolderIncludeSub>false</FolderIncludeSub>
			<MoveToBackup>true</MoveToBackup>
			<SaveItems>true</SaveItems>
			<XlsSheet/>
			<SsxSheet>signifikant_parts.xsl</SsxSheet>
			<UploadItems>true</UploadItems>
			<UploadServer>http://localhost/AssertWeb/</UploadServer>
			<UploaderId>Parts</UploaderId>
			<UploadSiteId>Alo</UploadSiteId>
			<UploadLogin>false</UploadLogin>
			<UploadUsername>uploader</UploadUsername>
			<UploadPassword>uploader</UploadPassword>
			<Options>
				<Option key="PurgeAfter" value="true"/>
			</Options>
		</CustomFolderSettings>
		<CustomFolderSettings ImporterId="PartsPrice">
			<DumpUploadItems>true</DumpUploadItems>
			<FolderDescription>Part price</FolderDescription>
			<FolderEnabled>true</FolderEnabled>
			<FolderFilter>PartsPrice*.xml</FolderFilter>
			<FolderPath>C:\Signifikant\Import_service</FolderPath>
			<BackupFolder>C:\Signifikant\Import_service_backup</BackupFolder>
			<FolderIncludeSub>false</FolderIncludeSub>
			<MoveToBackup>true</MoveToBackup>
			<SaveItems>true</SaveItems>
			<XlsSheet/>
			<SsxSheet>signifikant_price.xsl</SsxSheet>
			<UploadItems>true</UploadItems>
			<UploadServer>http://localhost/AssertWeb/</UploadServer>
			<UploaderId>PartsPrice</UploaderId>
			<UploadSiteId>Alo</UploadSiteId>
			<UploadLogin>false</UploadLogin>
			<UploadUsername>uploader</UploadUsername>
			<UploadPassword>uploader</UploadPassword>
			<Options>
				<Option key="PurgeAfter" value="true"/>
			</Options>
		</CustomFolderSettings>
		<CustomFolderSettings ImporterId="PartsBalance">
			<FolderEnabled>true</FolderEnabled>
			<FolderDescription>Part availability</FolderDescription>
			<FolderFilter>PartsBalance*.xml</FolderFilter>
			<FolderPath>C:\Signifikant\Import_service</FolderPath>
			<BackupFolder>C:\Signifikant\Import_service_backup</BackupFolder>
			<FolderIncludeSub>false</FolderIncludeSub>
			<MoveToBackup>true</MoveToBackup>
			<SaveItems>true</SaveItems>
			<SsxSheet>signifikant_avaibility.xsl</SsxSheet>
			<UploadItems>true</UploadItems>
			<UploadServer>http://localhost/AssertWeb/</UploadServer>
			<UploaderId>PartsBalance</UploaderId>
			<UploadSiteId>Alo</UploadSiteId>
			<UploadLogin>false</UploadLogin>
			<UploadUsername>uploader</UploadUsername>
			<UploadPassword>uploader</UploadPassword>
			<Options>
				<Option key="PurgeAfter" value="true"/>
			</Options>
		</CustomFolderSettings>
		<CustomFolderSettings ImporterId="Organizations">
			<FolderEnabled>true</FolderEnabled>
			<FolderDescription>Organization import</FolderDescription>
			<FolderFilter>Organisations*.xml</FolderFilter>
			<FolderPath>C:\Signifikant\Import_service</FolderPath>
			<BackupFolder>C:\Signifikant\Import_service_backup</BackupFolder>
			<FolderIncludeSub>false</FolderIncludeSub>
			<MoveToBackup>true</MoveToBackup>
			<SaveItems>true</SaveItems>
			<SsxSheet>signifikant_organization.xsl</SsxSheet>
			<UploadItems>true</UploadItems>
			<UploadServer>http://localhost/AssertWeb/</UploadServer>
			<UploaderId>Organizations</UploaderId>
			<UploadSiteId>Alo</UploadSiteId>
			<UploadLogin>false</UploadLogin>
			<UploadUsername>uploader</UploadUsername>
			<UploadPassword>uploader</UploadPassword>
			<Options>
				<Option key="PurgeAfter" value="true"/>
			</Options>
		</CustomFolderSettings>
		<CustomFolderSettings ImporterId="Users">
			<FolderEnabled>true</FolderEnabled>
			<FolderDescription>User Import</FolderDescription>
			<FolderFilter>Users*.xml</FolderFilter>
			<FolderPath>C:\Signifikant\Import_service</FolderPath>
			<BackupFolder>C:\Signifikant\Import_service_backup</BackupFolder>
			<FolderIncludeSub>false</FolderIncludeSub>
			<MoveToBackup>true</MoveToBackup>
			<SaveItems>true</SaveItems>
			<SsxSheet>signifikant_user.xsl</SsxSheet>
			<UploadItems>true</UploadItems>
			<UploadServer>http://localhost/AssertWeb/</UploadServer>
			<UploaderId>Users</UploaderId>
			<UploadSiteId>Alo</UploadSiteId>
			<UploadLogin>false</UploadLogin>
			<UploadUsername>uploader</UploadUsername>
			<UploadPassword>uploader</UploadPassword>
			<Options>
				<Option key="PurgeAfter" value="false"/>
			</Options>
		</CustomFolderSettings>
		<CustomFolderSettings ImporterId="Invoice">
          <FolderEnabled>true</FolderEnabled>
          <FolderDescription>Initial data place holder and template (Invoices for Dometic)</FolderDescription>
          <FolderFilter>Invoice*.xml</FolderFilter>
          <FolderPath>C:\Signifikant\Import_service</FolderPath>
          <DeleteAfterImport>true</DeleteAfterImport>
          <FolderIncludeSub>false</FolderIncludeSub>
          <SaveItems>true</SaveItems>
          <XlsSheet>signifikant_invoice.xsl</XlsSheet>
          <SsxSheet />
          <UploadItems>true</UploadItems>
          <UploaderId>Invoice</UploaderId>
          <UploadServer>http://localhost/AssertWeb/</UploadServer>
          <UploadUsername>uploader</UploadUsername>
          <UploadPassword>uploader</UploadPassword>
          <UploadLogin>false</UploadLogin>
          <Options>
            <Option key="PurgeAfter" value="false"/>
          </Options>
        </CustomFolderSettings>
	</Settings>
</CustomFolderConfiguration>

Desrciption of tags

Tags	Description	Options

Tags	Description	Options
ServerConfigPath	Path to the server.config file where configuration is kept for the editor- or web-server. Important is the Notification settings, and the integration to Monitoring dashboard.	For an editor server: C:\ProgramData\Signifikant\Assert\server.config. For a web server: C:\inetpub\wwwroot\AssertWeb\App_Data
FolderEnabled	If configuration is active or not.	true/false
FolderDescription	Description.	Text
FolderFilter	Regex used on the files that appear in the folder. Only files matching the regex will be handled, e.g. Part_price*.xml.	Text
FolderPath	Folder path to monitor for new files.
FolderIncludeSub	If sub folders are to be scanned for imports.	Text
MaxIdleDays	Time limit when it is considered an error if no new import files arrive, see below.	Integer
MaxIdleHours	Time limit when it is considered an error if no new import files arrive, see below.	Integer
DeleteAfterImport	If file should be deleted after a successful import. (NB: This takes precedence over backup settings) Note. This option is available in version 5.2 and later.	true/false
MoveToBackup	If file is to be moved to backup folder.	true/false
BackupFolder	After an import or upload is completed, move file to this folder.
SaveItems	If file content is to be saved on database on same server as importer service runs on.	true/false
SiteId	Site name when importing into Manager site.	Site name
UploadItem	If file content is to be uploaded to web server site.	true/false
UploadSiteId	Site name when uploading into web viewer site.	Site name
UploadServer	Url for the web server e.g https://www.domain.com/AssertWeb/
UploaderId	Type of upload, see table above.	See table above
UploadLogin	If login required to upload items to webserver.	true/false
UploadUsername	User name and password for calling the upload API on the web server. Api permissions are setup in: C:\ProgramData\Signifikant\Assert\api-permission.config, see Technical documentation - Signifikant REST API authentication .
UploadPassword
DumpUploadItems	If upload dump is to be generated in log folder.	true/false
XlsSheet	Custom Xls for the importer. Custom Xls shall be placed in C:\ProgramData\Signifikant\Assert\Customize
SsxSheet	Signifikant predefined SSX format for the import type as signifikant_organization.xsl
Options	Used to define alternatives to control the import	See separate table below.

Options tag has a set of keys.

Key	Value	Description	Options

Key

Value

Description

Options

PurgeAfter

bool

After last call with all uploaded part numbers, a valid number list with all valid part numbers are uploaded. All parts in the price list or warehouse lists which not in the valid numbers list removed from that price list or warehouse list. That way removed prices/warehouse stock will get deleted.

Use this method if full price lists or warehouse lists are sent.

Customization

Custom style sheet may be used to transform input format. Use tag <SsxSheet> to define a custom xsl.

Handlers for import or upload to remote server may be added in a customize module. By referring the custom handler in service configuration, any customer use case regarding automated file import, can be implemented. Custom modules require programming.

Permissions

The remote server may require permissions to access the upload Api interface. In that case we use basic authentication with a username/password to secure the function.

In this case the service need to declare login parameters for an upload handler, in the service configuration.

Logging

Configuration declares folders for logging, where each handler appends text to one file per day. Example of a log file name: UploadService_20200528_PartsPrice.log, indicating price information was uploaded.

The service also logs to an application log file, normally found at: C:\ProgramData\Signifikant\Assert\ImportService.log

Mail and event notification

Setting is available in 5.1.1 and later. Notification is made by mail, by monitoring event or both.

Notification by mail

Time limit when it is considered an error if no new import files arrive. If no import files arrives within the defined number of hours and days, a notification is created. The notification is defined in server.config.

<Notification>
	<Type>Service</Type>
	<MailEnabled>true</MailEnabled>
	<MailRecipient>support@company.com</MailRecipient>
	<MailSender>noreply@company.com</MailSender>
	<MailAtSuccess>false</MailAtSuccess>
	<MailAtFailure>true</MailAtFailure>
</Notification>

If none of MaxIdleHours and MaxIdleDays are defined, no monitoring will be turned on.

Ensure to use the server.config defined in tag ServerConfigPath in importer-service.config.

Notification by event generation to monitoring dashboard

To send import events to the monitoring dashboard, it uses a configuration file monitoring-service-client.json. The file is located in the same folder as the file defined by the ServerConfigPath element. Ensure that the file is in the same folder as the active server.config file. Should the file be missing. an empty file is generated, as a template for configuration. It has the enabled attribute defined to false to disable event generation initially.

Example of file monitoring-service-client.json, apiKey is received from Signifikant, externalId and customer may be used to identify events later.

{
    "enabled": true,
    "url": "https://development.signifikant.se/monitoringserviceapi/",
    "apiKey": "*",
    "defaults": {
        "externalId": "id",
        "customer": "name"
    },
    "apiKeyHeaderName": "api-key",
    "sendUrl": "api/event"
}

Custom Xsl for uploads

Overview

Xsl files transform customer exported files in one Xml format to a format that the import/upload engine handles. File names are entered in the configuration file and the file is located in common or custom program data folders.

Prices

Input format

Standard input format:

Output format

The price conversion Xsl file generates output records, one for each parts price for a price list.

If a part has a price list which does not exist, that price list will be created and made available. It will not be a default price list.

Availability

Input format

Standard input format:

Output format

The availability conversion Xsl file generates output records, one for each parts balance for a warehouse. The total of balance + supply-balance is loaded to the database - 137 in the example below.

If a part has a warehouse which does not exist, that warehouse will be created and made available. It will not be a default warehouse.

Part status

Input format

The part status import is used to set the AvailabilityCode and ReplacementCode on parts. Standard input format:

Organizations

Input format

For the file format, refer to definition of the Signifikant import format for organization. Standard input format.

Users

Input format

Here is an example of users xml.

Assets

Currently there is no transformation for the asset data. To import assets it is likely that a customize module is required to maintain all business data in exports. One simple example of an asset looks like.

Settings for testing

Version 5.1 and later

Importer service has a few extra settings intended for testing purposes, which may be used for error search.

Page:

Service Desk - Customer guide
Page:

OCR Application - How to use
Page:

Technical documentation - Import using Excel
Page:

Using modes in the Manager and at xml import
Page:

Setting up tasks for import, publish and clean up using Task Manager

Signifikant Knowledge Base

Technical documentation - Importer Service

Overview

Installation

Configuration

Imported data

Sample configuration

Desrciption of tags

Customization

Permissions

Logging

Mail and event notification

Notification by mail

Notification by event generation to monitoring dashboard

Custom Xsl for uploads

Overview

Prices

Input format

Output format

Availability

Input format

Output format

Part status

Input format

Organizations

Input format

Users

Input format

Assets

Settings for testing

Overview

Installation

Configuration

Imported data

Sample configuration

Desrciption of tags

Customization

Permissions

Logging

Mail and event notification

Notification by mail

Notification by event generation to monitoring dashboard

Custom Xsl for uploads

Overview

Prices

Input format

Output format

Availability

Input format

Output format

Part status

Input format

Organizations

Input format

Users

Input format

Assets

Settings for testing

Related articles