Windows Phone 8: Optical Character Recognition (OCR) library

Its been quite a while, time for blogging again. Today, I shall demonstrate how to use Optical Character Recognition (OCR) library on windows phone 8. Although, OCR library has been released with Windows Phone 8.1, but, it is compatible with Windows Phone 8 as well. This blog post is probably not the first one about OCR library, but, I want to make an easily working post out of it. Following are some prerequisites before you proceed any further in this tutorial. 

Prerequisites: 

1) Knowledge about Windows Phone platform.

Now, what OCR library is mainly about, well the library allows to extract text base information from an image. However, there are many limitations to it, you can see the details about OCR library limitations at MSDN. This library extracts digital text only.

You can download the complete source code for this tutorial or you can follow step by step discussion below. The sample code is in Microsoft Visual Studio 2012.

Download Now!

If you have downloaded the source code directly then make sure to install WriteableBitmapEx library & Microsoft OCR library either from provided links or from Nuget Manager from the Visual Studio.

Now, lets begin our coding.

1) Create a windows phone application with target of SDK 8 in your VS 2012 and name it "OCRSampleApp".
2) Then install WriteableBitmapEx library & Microsoft OCR library either from provided links or from Nuget Manager from the Visual Studio into your project.  
3) In your "MainPage.xaml" file replace existing code with following code:

<!--LayoutRoot is the root grid where all page content is placed-->  
  <Grid x:Name="LayoutRoot" Background="Transparent">  
     <Grid.RowDefinitions>  
       <RowDefinition Height="Auto"/>  
       <RowDefinition Height="*"/>  
     </Grid.RowDefinitions>  
     <!--TitlePanel contains the name of the application and page title-->  
     <StackPanel x:Name="TitlePanel" Grid.Row="0" Margin="12,17,0,28">  
       <TextBlock Text="Optical Character Recognition (OCR)" Style="{StaticResource PhoneTextNormalStyle}" Margin="12,0"/>  
       <TextBlock Text="Main Page" Margin="9,-7,0,0" Style="{StaticResource PhoneTextTitle1Style}"/>  
     </StackPanel>  
     <!--ContentPanel - place additional content here-->  
     <Grid x:Name="ContentPanel" Grid.Row="1" Margin="12,0,12,0">  
       <StackPanel Orientation="Vertical" HorizontalAlignment="Center">  
         <Image x:Name="imgObj"   
             Source="/Assets/img.jpg" />  
         <Button x:Name="btnExtractText"   
             Content="Extract Text"   
             Click="ExtractText_Click"/>  
         <TextBlock Text="Extracted Text:"  
               FontSize="28" />  
         <ScrollViewer HorizontalScrollBarVisibility="Visible">  
           <TextBlock x:Name="txtExtracted"   
                 FontSize="20" />  
         </ScrollViewer>  
       </StackPanel>  
     </Grid>  
   </Grid>  

In the code above we have simply load our sample text base image, create a place holder where our text will be extracted and create a button to extract the text. The sample image is taken from this blog.

4) Now, in your "MainPage.xaml.cs" file create two methods "GetBgraBitmapFormat()" & "ExtractText()" as follow:

    #region Get BGRA image bitmap format method.  
     /// <summary>  
     /// Get BGRA image bitmap format method.  
     /// </summary>  
     /// <param name="img">Image file parameter</param>  
     /// <returns>BGRA image bitmap format</returns>  
     private WriteableBitmap GetBgraBitmapFormat(Image img)  
     {  
       // Initialization.  
       WriteableBitmap bgraBitmap = null;  
       BitmapImage bitmap = img.Source as BitmapImage;  
       try  
       {          
         // Converting to BGRA bytes pixels  
         bgraBitmap = new WriteableBitmap(bitmap);  
       }  
       catch (Exception ex)  
       {  
         // information.  
         MessageBox.Show("Error", "Something goes wrong, please try again later", MessageBoxButton.OK);  
         Console.WriteLine(ex);  
       }  
       // Info  
       return bgraBitmap;  
     }  
     #endregion 
     
     #region Extract Text method.  
     /// <summary>  
     /// Extract Text method.  
     /// </summary>  
     /// <param name="bytePixels">Byte pixels parameter</param>  
     /// <param name="pixelsWidth">Pixels width parameter</param>  
     /// <param name="pixelsHeight">Pixels height parameter</param>  
     private async void ExtractText(byte[] bytePixels, int pixelsWidth, int pixelsHeight)  
     {  
       try   
       {  
         // Initialization.  
         OcrEngine ocrEngine = new OcrEngine(OcrLanguage.English);   
         // Verify whether image dimensions are supported for processing or not. Supported image dimensions must be between 40 and 2600 pixels.   
         if (pixelsHeight < 40 ||  
           pixelsHeight > 2600 ||  
           pixelsWidth < 40 ||  
           pixelsWidth > 2600)  
         {  
           string msg = "Image size is not supported." +  
                     Environment.NewLine +  
                     "Loaded image size is " + pixelsWidth + "x" + pixelsHeight + "." +  
                     Environment.NewLine +  
                     "Supported image dimensions must be between 40 and 2600 pixels.";  
           // information.  
           MessageBox.Show("Message", msg, MessageBoxButton.OK);  
           // Info.  
           return;  
         }  
         // Extract text.  
         var ocrResult = await ocrEngine.RecognizeAsync((uint)pixelsHeight, (uint)pixelsWidth, bytePixels);  
         // Verification  
         if (ocrResult.Equals(null))  
         {  
           // information.  
           MessageBox.Show("Message", "Image does not contain any text", MessageBoxButton.OK);  
           // Info.  
           return;  
         }  
         // Initialization.  
         string extractedText = string.Empty;  
         // Extracting Lines.  
         foreach (var line in ocrResult.Lines)  
         {  
           // Extracting Wordss.  
           foreach (var word in line.Words)  
           {  
             // Setting.  
             extractedText += word.Text + " ";  
           }  
           // Setting.  
           extractedText += Environment.NewLine;  
         }  
         // Setting.  
         this.txtExtracted.Text = extractedText;  
       }  
       catch (Exception ex)  
       {  
         // information.  
         MessageBox.Show("Error", "Something goes wrong, please try again later", MessageBoxButton.OK);  
         Console.WriteLine(ex);  
       }  
     }  
     #endregion

In the above code "GetBgraBitmapFormat()" method simply convert your image source into BGRA format which is acceptable by the OCR library. We have made this conversion possible using WriteableBitmapEx library. Similarly, we have used the Microsoft OCR library in "ExtractText()" method to extract the text from the image.

5) In next step, we will use our above created methods in the text extraction event as follow:

    #region Extract text button event method.  
     /// <summary>  
     /// Extract text button event method.  
     /// </summary>  
     /// <param name="sender">Sender parameter</param>  
     /// <param name="e">Event parameter</param>  
     private void ExtractText_Click(object sender, RoutedEventArgs e)  
     {  
       try  
       {  
         // Get BGRA image bitmap format.  
         WriteableBitmap bgraBitmap = this.GetBgraBitmapFormat(this.imgObj);  

         // Verification.  
         if (bgraBitmap.Equals(null))  
         {  
           // Info.  
           return;  
         }  
         // Extract text.  
         this.ExtractText(bgraBitmap.ToByteArray(), bgraBitmap.PixelWidth, bgraBitmap.PixelHeight);  
       }  
       catch (Exception ex)  
       {  
         // information.  
         MessageBox.Show("Error", "Something goes wrong, please try again later", MessageBoxButton.OK);  
         Console.WriteLine(ex);  
       }  
     }  
     #endregion 

In the above code, we have simply call "GetBgraBitmapFormat()" method first then "ExtractText()" method.

6) Now, "build & execute" the project and you will see following:

Before text extraction:

After text extraction:

That's about it!!

Enjoy coding!!!