Building a Norce [Storm] Channel Feed receiver

You can use Norce's channel functionality to set up specific criterias for fetching and passing a specific assortment of products to target systems. This enables you to manage product data feeds and updates in a scalable and reliable manner. The target systems could be for example search services, marketplaces, and other similar platforms.

Examples and common practices

Related guides:

• Using an external search

Open-api and end-point documentation:

• API References
• Schemas

Feed examples:

• Product feed example
• ProductStatus feed example

Responsibilities of the Channel feed receiver

You are responsible for implementing the receiver for Norce [Storm]'s channel feed. The receiver will handle feeds created by the channel service and export them to an http endpoint. It will also download and pass the data to the target systems, after mapping to the target's format and applying any necessary business logic.

Your receiver is responsible for managing, controlling, and checking the feeds using Norce [Storm]'s Channel Service API. This includes:

• Triggering the creation of new feeds, from an external call
• Fetching feeds from the URL when they're ready (once the file URL is no longer locked)
• Handling feed process failures (by changing deltafromdate when needed and monitoring and handling download and target system push problems)

Configurations

You can configure the channel export in the Admin UI under the Channel Page. The important configuration fields are:

• Key A GUID generated by Norce, and required in the API calls
• Culture Specifies the language for product texts that are passed to the feed, multiple cultures are allowed
• Price lists Determines the assortment passed to the feed and which price lists are part of the best price validation; if left empty, all public price lists for the application will be used
• Parametrics Add parametrics to product export, default is false
• Relations Add product relations to product exports, default is false
• Popularities Add popularities to product exports, default is false
• Include Inactivated Variant Day Count If a number above zero, a variant/product that has been removed from the price list will still be included but without a price, this makes it possible to actively disable variants that are removed from the assortment, number of days (24 hours) from inactivation the variant can be fetched, max value is 5
• Use fallback culture values If true, the export will return default culture on active products that do not have text for the specified culture

Configure Delta changes

Enable tracking of delta changes

To track changes of your products, you must enable "delta tables" under Settings → Client → Settings (last checkbox under “Products”).

Unclick the arrow to use the checkbox.

Delta configuration

Lastly, you must decide what changes in your products you want to track.

Under Products → Settings → Delta Configuration you can modify the settings to determine which changes will be tracked.

Delta configuration can be complex, so here is an example with screengrabs and explanations.

Name and entity	Description	Snapshots
Product and SKU information (tClientProductSku, tProductCulture)	In this example, we are not interested in new products, thats why “on insert” is disabled (they can wait on the full feed) But when an existing product is changed on the swedish name or description. Or when the product status (statusId) or EAN code (eanCode) is changed, we want the delta feed to notice it. But only for the standard products	(Right click and open image in new tab)
Price information (tClientProductSkuPriceList)	If the product gets new prices or if the price is changed we need to get it in the feeds as soon as possible. But not all Price lists are tracked, only the Standard SE.	(Right click and open image in new tab)
Onhand information (tClientProductSkuOnhand)	When the onhand value (onHandValue) is changed or added to the European Central warehouse, it should be tracked.	(Right click and open image in new tab)
Product flag information (tProductFlag)	If the flag "Discount 10%" or "Discount 20%" is added (On Insert) or changed (On Update), it should be tracked.	(Right click and open image in new tab)

Read more about what all the delta configuration settings means here.

Feed types

The channel supports four types of feeds:

• Product (Full), full data feed.
• Product (Delta), only changes.
• ProductStatus (Full).
• ProductStatus (Delta), only changes.

Product Full

The Product feeds contain the largest amount of product data and are used to populate search services with content, images, parametrics, prices, and other information. This feed returns all products from the product catalog defined in the channel settings, such as all active and expiring products for a specific price list in a particular language.

Product Delta

Just as Product Full this contains most of the product data, but delta requests returns only changed products from a specified time. This is specified under Products -> Settings -> Delta Configuration in the admin tool, here.

The delta is triggered with a request body containing :

• channelKey : This is mandatory and specifies which channel's feed is executed.
• deltaFromDate : This is a timestamp (UTC) that specifies from when all changed products should be included. If this is not specified, a full product feed will be returned instead.

ProductStatus (Full and Delta)

The ProductStatus feed contains a smaller set of information, mainly price, availability, flags, and supplier information. This data usually changes more often than other information and can be fetched and updated in the target system.

The request body for this feed contains the same information as Product.

Fetching data from an http(s) endpoint

Norce sends the data to an endpoint that is reachable by you over http(s).

The process

1. The feed is triggered by the Receiver.
   1. The response includes a JobKey and a status code.
   2. If status code is:
      1. Success: the feed is created.
      2. AlreadyInProgress: a feed of the same type is already being created, this request is ignored.
      3. NoData: a feed could not be created because no products or changes were found.
      4. Failed: an internal server error occurred in the Norce [Storm] channel service when creating the feed. See logs in admin for more information.
2. The feed is created in the Azure storage. The feed can check the api for the status on the job, calling ChannelService/api/v1/jobs/job wait and poll until the status is changed to CompletedInExportFeed
3. The Receiver downloads the new Product data from an HTTP(s) endpoint in Azure. More information about the Azure file service API can be found here .
4. The Receiver applies mappings and business logic (if needed) to the data and creates or updates the target system(s).
5. (Optional) The receiver can store the JobKey in their own logs if needed. The data is accessible for some time.

Full feed

The Receiver triggers the feed creation by calling ChannelService/api/v1/exports/product or ChannelService/api/v1/exports/productstatus, excluding DeltaFromDate, or having it set to null.

{
  "channelId": "[the channel key]"
}

The result is the full catalog according to Norce at the specific time of export. This may require that you should deactivate all other products in the target system and populate it with the new catalog.

Delta feeds

The Receiver triggers the feed creation by calling ChannelService/api/v1/exports/product or ChannelService/api/v1/exports/productstatus, with a DeltaFromDate, that defines from when the changed products should be included.

{
  "channelId": "[the channel key]",
  "deltaFromDate": "2020-10-01T14:43:19.210Z"
}

The result is the list of modified products in the catalog. This feed is intended to update only these specific products in the target system, leaving the other products unchanged.

Old feeds

Feeds that are older than 90 days are removed, but until then you can access them at https://stormfeedsstage.blob.core.windows.net/exports/[JobKey] for the lab environment or https://stormfeeds.blob.core.windows.net/exports/[JobKey] for the production environment.

This can be useful for reviewing past data and checking how it looked at a specific time.

The Product contracts

• Product
• ProductStatus

Additionally, an example feeds are provided at

• Product feed example
• ProductStatus feed example

Developing and testing the feeds

During the development phase, use the public feed URLs for quick access to some sample data (see above).

If you have real data from the client in Norce, set up a channel and create a couple of feeds. In the channel log, you will find JobKeys that you need to access and download data.