Using a Power Apps app with AI Builder models to Capture a Vendor Invoice

In my previous posts, I used AI Builder in Power Apps to set up models for form processing and category classification respectively. In this post I will show the app, I have built with Power Apps that brings those two capabilities together. In the next post, I will dissect the app to show how to leverage the AI Builder models inside an app.

In the last post of this series, we will take a quick look at how I use Power Automate to submit the invoice to Dynamics 365 Finance.

As you can see from the screenshots, my use case has a slightly agricultural flavour.

When I start the app, I am presented with a landing page. As you can see, the app is quite simple, but obviously the landing page allows me to add more items to the menu, if required.

From the landing page, I select Process Invoice.

In my example, I am asking the user to select the cost centre where the cost on the invoice needs to go. In this case I select John Deere Tractor 5100R and click on the <NEXT> icon on the line.

From this screen the user is able to see the selected cost centre and the captured invoice image before submitting it to Dynamics 365 Finance. However, first, the user must capture the invoice by clicking on the camera icon.

I now come into the screen where I can capture the invoice. This is done by clicking on the Capture Invoice button. Since I am doing this from an iPhone, I am asked to take a photo or select an existing photo from the library as shown below.

In this case I take the following photo of the invoice.

When I click on Choose, the image is submitted to the AI Builder form processor for interpretation. The result is shown below.

As you can see, the image is brought back into the app and the form processor has interpreted the invoice. The image has labels applied that show the captured fields and their accuracy. In the lower section of the screen, I am using this information to fill out the variables I need to submit the invoice to Dynamics 365 Finance.

Since I am using an English form processing mode, the description has been interpreted as “110mm ror” instead of “110mm rør”. After the invoice has been interpreted, I take the description and process is through the category classification model in built earlier. This identifies the description to belong to the category “ror” and this category is tagged to the main account “2020”.

Since the invoice has been interpreted correctly, I can click on the accept icon to go back.

Coming back to the main processing screen, the captured image is now displayed. Also, the save icon becomes available and when I click on it, the invoice is submitted to Dynamics 365 Finance.

Once the Power Automate flow has finished processing, the user is told the invoice has been processed.

Obviously, in the real world I should have trained the model to read multiple lines into a table. However, I hope this simple example has shown the power of using AI Builder in apps.

In the next post, I will take a look at the mechanics of this app. Stay tuned…