Introduction
AI Builder is a Power Platform capability that provides you the AI models to take leverage of artificial intelligence to optimize your business as well as automate processes. We can either use the already existing prebuilt models that suit our needs to use the Power of AI in Power Apps or Power Automate or create our own custom AI models to suit some specific business needs. You can read more about AI Models in the Official documentation here. The out of the box pre-built available models in AI Builder are:
- Business card reader
- Category classification
- Entity extraction
- ID reader
- Invoice Processing
- Key phrase extraction
- Language detection
- Receipt processing
- Sentiment analysis
- Text recognition
- Text translation
In this article we will see how to use the AI Builder Model – Text recognition to extract text from an image and process it further
Business Use Case
We have a requirement where Field Police Officers would capture the Number plate image of speeding cars and send it as an attachment in the mail for further processing their fine dues in the back end. We will be using the Text Recognition AI model to extract the car number from the image and create a SharePoint list item automatically via flow so that any manual effort for text extraction can be avoided.
Implementation
We will be using the ‘when a new email arrives’ trigger to kickstart the flow when a speeding ticket email arrives. Followed by that we will add an Apply to each condition so that we can iterate through multiple image attachments to process the text extraction.
Within the loop, we will use the Get Attachment action to extract the images which have come as email attachments. We will then add the ‘AI Builder action Recognize text in an image or a PDF document’ action to the flow and feed the ‘Content Bytes’ Dynamic output of the Get attachment action as the AI Builder Model Input.
Now we will use a compose action to extract the Vehicle number from the extracted text output which is a JSON object
We will use the below expression to fetch the Results Array and then index it into the lines attribute which contains the vehicle number in the key ‘text’.
| outputs(‘Recognize_text_in_an_image_or_a_PDF_document’)?[‘body/responsev2/predictionOutput/results’][0]?[‘lines’][0]?[‘text’] |
Finally we will add the create item action to save the extracted text(obtained from the Compose Output action) as a list item along with the Image Attachment Name(corresponds to the vehicle owner)
The loop will execute as many times as the number of attachments and extract the number plate information in the multiple images and write back the records to SharePoint List.
Thus the overall flow will look like this:
Test the flow
Let’s test the flow by attaching multiple vehicle number images as attachments to the mail and sending them to the destination mailbox which will trigger the flow.
It will loop through all 4 attachments and extract the information as shown below
The extracted data will be written back to the SharePoint List.
Summary
Thus, we saw how to use the Text recognition AI Model to extract text from an image that was sent as a mail attachment and write it back to the SharePoint List for further processing.
