# System integration patterns ## Import data to Norce Commerce When you have data you want to update into Norce Commerce, you should use Norce Commerce Connect. For most data endpoints this is done asynchronously, like `Products`, `Prices` or `Customers`. For more transactional information, like `Orders`, `Invoices` etc. this is done using the synchronous calls described here: [Implement an Order Receiver Service](/developer-portal/system-integration/implement-an-order-receiver-service). Read more about calling Norce Commerce Connect [here](/developer-portal/system-integration/calling-norce-commerce-connect). ### Full and delta updates There are two types of updates that should be done. **A delta update**, has a payload of items that has been changed since last time an update was done. It is the responsibility of the integration to implement a delta process, where you can *fetch changed data only* from the source (ERP) in a specified time interval. How often is depended on the client’s situation, for `Products` this might be 1-4 times a day but for `Customers` this may be more often. **A full update**, has a payload of all your current items from the source (ERP). When a full update is done, *Norce Commerce will deactivate all items* that are not in the payload. In this way, you can make sure that your integration is “self fixing” itself, that if some delta updates has failed, the full update will make sure the products are correct. A full update for the `ImportProducts` is usually schedules 1-7 times a week. Depending on how much your product data changes. But for `Customers` once every week might be enough. #### **Full import** ```mermaid %%{init: {'theme':'default'}}%% sequenceDiagram autonumber participant Q as Query participant C as Connect participant I as Integration participant S as Source system participant T as Timer T-->>I: The timer triggers at a pre-set schedule I->>S: Lookup all current item data I-->>Q: Lookup item identifiers (optional) I->>C: Import all items (fullfile=true) ``` The pattern for a full asynchronous import starts in the integration, where a schedule trigger an action to fetch all current items from the source and map and transform the source data to the Norce Commerce Connect contracts. This might require lookups of key and codes from Norce Commerce Query. Then an asynchronous call is done to one of the Import method calls in Norce Commerce Connect with the header flag `Fullfile` set to `true`. You get a job id back that you should track. Either by polling the [Job endpoint](/api-reference/connect/jobservice/openapi/job/get) in Norce Commerce Connect or by subscribing to the [JobCompletedEvent](/api-reference/event/webhooks/openapi/system/paths/jobcompletedevent/post) in Norce Commerce Event. Other things to think about are: 1. All earlier imports that has failed of the same type (delta and full) should be discarded, since the full import is the new state to start from. 2. Use the `X-Stormconnect-Header` object to control exactly what is allowed to be updated from the source. 3. If possible, try to separate different sources to different imports, this makes it easier to see and control which data is updated when. We suggest that you use [different system accounts](/developer-portal/system-integration/calling-norce-commerce-connect#using-system-accounts) so that Norce Commerce will also help to track which system has updated what and when. #### **Delta import** ```mermaid %%{init: {'theme':'default'}}%% sequenceDiagram autonumber participant Q as Query participant C as Connect participant I as Integration participant S as Source system participant T as Timer T-->>I: The timer triggers at a pre-set schedule I->>S: Lookup only changed item data I-->>Q: Lookup item identifiers (optional) I->>C: Import only changed items (fullfile=false) ``` The pattern for delta asynchronous imports looks similar to the full import. It starts in the integration, where a schedule trigger an action to fetch all changed items from the source and map and transform the data to the Norce Commerce Connect contracts. This might require lookups of key and codes from Norce Commerce Query. Then an asynchronous call is done to one of the Import method calls in Norce Commerce Connect with the header flag `Fullfile` set to `false`. You get a job id back that you should track. Either by polling the [Job endpoint](/api-reference/connect/jobservice/openapi/job/get) in Norce Commerce Connect or by subscribing to the [JobCompletedEvent](/api-reference/event/webhooks/openapi/system/paths/jobcompletedevent/post) in Norce Commerce Event. Norce [Storm] uses the old Norce [Storm] Event, that you can read about [here](/legacy/using-stormevent). Other things of note are: 1. Keep track of other jobs and make sure that your import is not overwriting newer data. 2. Use the `X-Stormconnect-Header` object to control exactly what is allowed to be updated from the source. 3. If possible, try to separate different sources to different imports, this makes it easier to see and control which data is updated when. We suggest that you use [different system accounts](/developer-portal/system-integration/calling-norce-commerce-connect#using-system-accounts) so that Norce Commerce will also help to track which system has updated what and when. A common bad design is to handle delta imports as real-time updates. This is not how you should design your integration. Use a scheduler and fetch all changed data from the last time. See warning below. Note Because of Norce Commerce Connect’s design, it is **not advisable to handle your updates one at a time**, This might cause Connect’s queue to quickly grow and might result in much longer lead times than it would be if you had fewer calls with many changed items. Read more [here](/developer-portal/system-integration/calling-norce-commerce-connect). #### **Other responsibilities of your integration** It's the responsibility of the integration to receive the `JobId` from Norce and handle monitoring and unexpected errors. For example: - Errors in mapping and business logic in the integration layer - Handle and monitor errors from Norce Commerce - Monitor for long lead times - Handle and monitor the current running jobs, like restarting a finished job. ## Get data from Norce Commerce The integration pattern for events has changes with the New Norce Commerce Event webhook feature. See the old Norce [Storm] Event pattern [here](/legacy/integration-patterns-storm-event). ```mermaid %%{init: {'theme':'default'}}%% sequenceDiagram autonumber participant E as Event participant Q as Query participant I as Integration participant T as Target system E-->>I: Trigger an event (item changed) I->>Q: Lookup item information I->>T: Create or update item in other system ``` When you want to create or update data in other target systems based on data in Norce Commerce, Norce Commerce Query and Event is used. The integration is triggered from Norce Commerce Event, that your integration gets called from as a webhook notification. Subscribe for the event types you are interested in. What you get is a small message containing only identifiable information for the item that is created or changed, and you must then look up the data in Norce Commerce Query. Read about calling Norce Commerce Query [here](/developer-portal/system-integration/calling-norce-commerce-query) and using Norce Commerce Event [here](/developer-portal/system-integration/using-norceevent). ### Responsibilities of your integration The integration is responsible for: - Receive events - Map and pass data to the target system - Look up relevant data from Norce using Norce Commerce Query, in a way that makes the solution perform well ### Subscribe and listen to Norce Commerce Event's In the Admin UI you can configure the event types you are interested in, see here. The message you receive has the event metadata described in the event detail tab in admin. This is commonly the `clientId` and `applicationId` (where it is applicable). The specific identifiers for the item you are interested in, for example `CustomerId` or product `PartNo`. These lets you look up the information you need from Norce Commerce query. Lastly you also get an `EntityChanged` property, that describes a bit more what has happened that makes the event trigger. Note - Read about [using Norce Commerce Event](/developer-portal/system-integration/using-norceevent). - Read about the Norce Commerce Event Setting and EntityChanged property [here](/developer-portal/system-integration/norce-event-settings). ### Lookup item data in Norce Commerce Query Norce Commerce Event will give out the item id’s (whereas it is `Customer`, `Product` or anything else) for you to use when fetching the data from Norce Commerce Query. With Norce Commerce query, you are able to only fetch the data your target system needs instead of handling large complex contracts of information. - Use local cache when appropriate to have a high performing solution. - Use the examples in the Postman collection to get you started setting up the integration. Note - Read about [Calling Norce Commerce Query](/developer-portal/system-integration/calling-norce-commerce-query). - Examples on Norce Commerce Query for different use cases are [here](https://documenter.getpostman.com/view/2973406/2sA35MzK11). ### Map and send data to target system After fetching the data from Norce Commerce Query, the next step is to pass it to the target system(s). How this is done is left to the integration developer and is dependent on how the target system needs the data. ## Suggested further reading - [Calling Norce Commerce Connect](/developer-portal/system-integration/calling-norce-commerce-connect) - [Calling Norce Commerce Query](/developer-portal/system-integration/calling-norce-commerce-query) - [Using Norce Commerce Event](/developer-portal/system-integration/using-norceevent)