In the previous post, we looked at how we could incorporate Google Nest devices into the Connected Field Service (CFS) solution. This post will look at extending some of these scenarios particularly around the video stream available from the Nest camera. To get started, the first scenario we are going to achieve is to record the video stream from the device to a SharePoint folder that is associated with the Dynamics 365 Customer Asset record.
In terms of prerequisites, you will need to ensure that your Dynamics 365 and SharePoint integration is setup and configured for Customer Assets. If you haven't done this already, here is a great guide from Microsoft. Architecturally, this is the pattern we will be implementing:
*The above is available in Visio format here.
Next, we are just going to visualise the video feed on a Dynamics 365 form through a custom Web Resource which connects to the camera's publicly available stream. It looks as follows:
You will notice that at the bottom of the video feed is the ability to start and stop a recording. There are some simple notifications on pushing each of these and the result is that the stream is stored into the SharePoint location associated with the Customer Asset as follows:
The magic that is happening in the background is that when you click on the 'Start' button, an Azure function is called which in turns runs a custom console app as part of the function and this then connects to the Nest Api and records the video stream before processing it and uploading it to the Document Location associated with the Customer Asset record. The result is then an .mp4 file in the SharePoint location which we can then run through the Video Indexing Cognitive Service via a Logic App. The point of this Logic App will be to process the video feed and provide appropriate IoT Alerts back into Dynamics 365 which are actionable and can be processed by the system. Let's have a look at what the Logic Apps look like:
*The link to the Video Indexer Portal is available here.
The reason that we have two Logic Apps is that the first Logic App does the actual uploading of the file and the second Logic App is the called by the first Logic App once the processing of the video has completed. This follows a typical asynchronous upload pattern and we use the 'Callback URL' property of the 'Upload video and index (Preview)' step to call the second Logic App from the first. As the Video Indexing service is still in preview, you should note that you will need to remove the following line from the Logic App's code in order to get the video to upload correctly:
Ultimately we have a result in Dynamics 365 indicating that the Nest Camera has processed the video and created some 'insights' which we have uploaded to the 'Alert Data' field on the IoT Alert entity. We can now action something based on these 'insights' if we wish. The record in Dynamics 365 looks like this:
To see this in action, watch the below video:
Lookout for the next post where we will dive into using a PowerApp to cross-validate this IoT Alert with the physical location of a technician and then enable them to execute certain actions via CFS!