Kylie Bot Part 14 - Populating an Azure Index
Now that we have an Azure Search index created, we will need to populate it with some content to make it meaningful. The first thing to decide is how to populate the index. For me, ease is always top of mind and in this sense, the Azure suite will make the most sense. But which service you ask? Well there are only really two main contenders that I considered, Flow and Logic Apps. There is a simple comparison here.
In my case, I chose Logic Apps primarily because of its extensibility and aim towards ‘mission-critical’ scenarios. After all, empowered knowledge is mission-critical to most organisations. With that decision made, it’s now time to provision a Logic App. First, log into the Azure Portal and you should be presented with the following dashboard:
Next, click on the ‘+New’ option in the top left of the screen and you should be presented with:
Next, type in ‘Logic App’ and select the Logic App pop-up item as follows:
You should then be presented with the following:
Click on the ‘Logic App’ item and you should then see this:
Click on the ‘Create’ button in the bottom right hand side of the screen to provision the Logic App. Enter in the required details in the provisioning screen that looks like this:
Click the ‘Create’ button in the bottom left hand side of the screen to finalise the Logic App creation. You should notice two notifications informing you that the app has begun deploying and then another to confirm that the deployment was successful. Next, navigate to and click on the ‘Resource groups’ menu item and you should see the following:
Click on the Resource Group where you just created the Logic App and then you should be presented with a list of resources. Next, click on the Logic App (mine is called ‘PopulateKylieBotIndex’) and you should see the following:
We are going to provision a blank Logic App, so click on the white square that says, ‘Blank Logic App’. Next, you should be presented with the following screen:
Type in Schedule which should result in the following:
Next, click on the blue ‘Schedule’ menu option and you should see this:
Click on the blue ‘Schedule Recurrence’ menu option and you should then see this:
For our purposes, I am going to get the index to reload once a day, so I will set the Interval field to 1 and the Frequency field to Day. Click on the ‘Show advanced options, you will also be able to specify additional parameters like this:
Once the items have been completed, click on the ‘Save’ menu item in the ribbon to save the changes. A notification should pop up in the right-hand side of the screen to confirm the save was successful. Next, we need to click on the ‘+ New step’ button to add in some additional steps. Your screen should look like this:
Click on ‘Add an action’ and you should observe the following:
Type in ‘Dynamics 365’ and you should see the following:
Go ahead and select the ‘Dynamics 365 – List Records’ option like this:
You will be asked to sign into your Dynamics 365 Customer Engagement instance. Remember this is the one that has the Knowledge Articles in it! Once Signed in, you should see the following screen:
Select the Organization Name from the drop down and then select the ‘Knowledge Articles’ option from the Entity Name drop down like this:
We will also need to ensure that only published articles are pulled into the index. To do this, we will modify the ‘List records’ operation by clicking on the blue ‘Show advanced options’ drop down and then specifying a filter as follows:
*the statuscode of 7 is the value for ‘Published’ in Dynamics 365 Customer Engagement.
Next, we will need to add a step to ensure that we only load records that have been updated or created in Dynamics 365 that aren’t already in or update in our index. To do this, add a new step but this time select the ‘Add a condition’ option and you should see this:
Click on the green ‘HTTP’ option like this:
Next, select the green ‘HTTP – HTTP’ option:
You can then go ahead and select ‘POST’ for the Method field. For the Uri field, enter the following: https://<your search service>.search.windows.net/indexes/generalcontent/docs/index?api-version=2016-09-01.
For the Headers section, make sure you add in the ‘api-key’ option with a value of the key that you copied earlier from when we created the Azure Search Service. You will also need to specify the content type of the header and the easiest way to do this is to click on the icon at the end of the line and update the field as follows:
Next, match up the attributes in the index we just created to the elements of our Knowledge Article entity like this:
*Note that the entity fields will be available from the pop up box under the ‘Dynamic content’ tab while the ‘Expression’ tab will hold formulas like the current date one we need for our LoadDate field.
Following the specification of the HTTP Body, we will need to add a condition to check that the HTTP request was successful. To do so, click on the ‘Add a condition’ button at the bottom of the dialogue. You should then see this:
In the Value field select ‘Status code’ like this:
Following this, set the value field to ‘200’ which represents an ‘OK’ status like this:
*For a list of HTTP status codes, see here.
There is no need to specify the ‘If true’ section of the statement as we are only worried when the HTTP action is unsuccessful. So let’s populate the ‘If false’ section. To do this, click on the ‘Add action’ button under the red ‘If false’ section. You should then be presented with:
Type ‘Dynamics 365’ into the box provided and then select the ‘Dynamics 365 – Create a new record’ option like this:
You should then see this:
Select your Organisation Name from the drop down list and then chose the ‘Index Populations’ option from the Entity Name drop down list.
*If the Index Populations item isn’t there, make sure you followed the steps from the previous post to create it
Next, click on the ‘Show advanced options’ drop down which is highlighted in blue, and complete the form as follows:
We can now click on the ‘Save’ option in the top left-hand side of the screen to save all of our changes. Once this is completed, we can click on the ‘Run’ menu item to test that everything is working correctly. Just in case you need it, the actual code for the logic app is here. You should see this result:
*Note that the default action of the Logic App is to ‘upload’ the documents, so this means that: “An upload action is similar to an "upsert" where the document will be inserted if it is new and updated/replaced if it exists. Note that all fields are replaced in the update case.”
From the screenshot, you can see that 104 actions were processes with no errors. Just to be on the safe side, we can go and check the actual index to make sure everything loaded correctly. To do this, navigate to the Azure Search service we created previously like this:
Here, click on the ‘generalcontent’ index and you should see this:
Next, click on the ‘Search explorer’ button on the top right-hand side of the form. You should then see this:
Click on the blue ‘Search’ button to execute a blank search and you should get a result similar to the following:
Awesome job. Now that we have populated our index with the necessary content, the next blog post will focus on updating the Kylie Bot to look at this new index.
*Don't forget that you can rerun this process with a number of different sources to populate your index with additional content!