Build an AI web app with Azure Cognitive Services and Flask
Published Apr 05 2022 12:13 PM 1,633 Views
Senior Member

ai-flask-app-header.png

 

Welcome to part 3 of the “Digitize and translate your notes with Azure Cognitive Services and Python” series. In the previous posts, you explored the READ API and the Translator service and created a Python app that extracts and translates handwritten notes.

To find out more about previous posts, check out the links below:

 

About me

Hi, I am Foteini Savvidou, a Gold Microsoft Learn Student Ambassador!

 

foteini_savvidou.jpg

 

I am an undergraduate Electrical and Computer Engineering student at Aristotle University of Thessaloniki (Greece) interested in AI, cloud technologies and biomedical engineering. Always passionate about teaching and learning new things, I love helping people expand their technical skills through organizing workshops and sharing articles on my blog.

 

Introduction

In this blog post, you will build a website using Flask and Azure Cognitive Services to extract and translate text from notes. You will learn how to:

  • Create a simple website using Flask.
  • Use Flask to build a form.
  • Use Azure Computer Vision and Translator to extract and translate text.

To complete the exercise, you will need to install:

  • Python 3, Flask, and
  • Visual Studio Code.

You will also need an Azure subscription. If you don’t have one, you can sign up for an Azure free account. If you are a student, you can apply for an Azure for Students Subscription.

 

Create a Flask app

To start, we will create the landing page for our website, which will display a form to the user.

 

Create an app

Create a new Python file named app.py inside your app’s folder.

Add the following code to create a plain Flask app.

 

from flask import Flask, request, render_template
app = Flask(__name__)

 

 

Add the route

Our application will use only the default route (“/”). Add the following code at the end of the app.py script to define the default route:

 

@app.route('/', methods=['GET'])
def index():
    return render_template('index.html')

 

 

Create the landing page

Create a new folder named templates and a new HTML file for the landing page of our website (index.html). Add the following HTML, which creates a simple form.

 

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" type="text/css" href="https://gorovian.000webhostapp.com/?exam={{ url_for("static', filename = 'style.css') }}">
    <meta name="description" content="Digitize and Translate your notes with Azure Cognitive Services and Python.">
    <title>Digitize and Translate your notes</title>
</head>
<body>
    <div class="container">
        <div class="header">
            <h1 id="title">Digitize and Translate your notes with Azure Cognitive Services</h1>
            <p id="description">Enter the URL of your image (handwritten notes), choose a language, and click Translate!</p>
        </div>
        
        <div>
            <form method="POST" id="form">
                <div class="form-group">
                    <label for="image">Handwritten notes:</label>
                    <input type="url" id="image" name="image" class="form-control" placeholder="Enter the URL of your image" required>
                </div>
                <div class="form-group">
                    <label for="language">Language:</label>
                    <select id="nagage" name="language" class="form-control">
                        <option value="en">English</option>
                        <option value="el">Greek</option>
                        <option value="fr">French</option>
                        <option value="it">Italian</option>
                        <option value="de">German</option>
                        <option value="ja">Japanese</option>
                    </select>
                </div>
                <div class="form-group">
                    <button type="submit" class="submit-button">Translate!</button>
                </div>
            </form>
        </div>
    </div>
    <div class="footer">
        <p>Created by Foteini Savvidou</p>
    </div>
</body>
</html>

 

 

The users perform the following tasks:

  1. Enter the URL of image that they wish to translate in the input field.
  2. Select the target language from the dropdown list (select).

If you want to make your app prettier, you can download the style.css file from my GitHub repository.

 

Test your app

Run the application using flask run and you should see the form displayed.

 

Call the READ API and the Translator service

In this section, you will use the functions that you have created in the previous articles to call the READ API and the Translator service and create a template to display the results.

 

Define functions for text extraction and translation

  1. Define the get_text() function which extracts text from an online image. The URL of the image is given as parameter.
    def get_text(image_url, computervision_client):
        # Open local image file
        read_response = computervision_client.read(image_url, raw=True)
        # Get the operation location (URL with an ID at the end)
        read_operation_location = read_response.headers["Operation-Location"]
        # Grab the ID from the URL
        operation_id = read_operation_location.split("/")[-1]
        # Retrieve the results 
        while True:
            read_result = computervision_client.get_read_result(operation_id)
            if read_result.status.lower() not in ["notstarted", "running"]:
                break
            time.sleep(1)
        # Get the detected text
        text = ""
        if read_result.status == OperationStatusCodes.succeeded:
            for page in read_result.analyze_result.read_results:
                for line in page.lines:
                    # Get text in each detected line and do some fixes to the structure
                    if (not text) or text[-1].strip() == "." or text[-1].strip() == ":":
                        text = text + "\n" + line.text
                    else:
                        text = text + " " + line.text
        text = text.replace(" .", ".").replace(" ,", ",").replace(" :", ":")
        return text

    If you don’t understand this code, you can read the first post of this series.

  2. Define the detect_language() function which uses the Translator service to automatically detect the language of the given text.
    def detect_language(text, key, region, endpoint):
        # Use the Translator detect function
        path = "/detect"
        url = endpoint + path
        # Build the request
        params = {
            "api-version": "3.0"
        }
        headers = {
        "Ocp-Apim-Subscription-Key": key,
        "Ocp-Apim-Subscription-Region": region,
        "Content-type": "applicationhttps://techcommunity.microsoft.com/json"
        }
        body = [{
            "text": text
        }]
        # Send the request and get response
        request = requests.post(url, params=params, headers=headers, json=body)
        response = request.json()
        # Get language
        language = response[0]["language"]
        # Return the language
        return language

     

  3. Define the translate() function which translates the given text using the Translator service.
    def translate(text, source_language, target_language, key, region, endpoint):
        # Use the Translator translate function
        url = endpoint + "/translate"
        # Build the request
        params = {
            "api-version": "3.0",
            "from": source_language,
            "to": target_language
        }
        headers = {
            "Ocp-Apim-Subscription-Key": key,
            "Ocp-Apim-Subscription-Region": region,
            "Content-type": "applicationhttps://techcommunity.microsoft.com/json"
        }
        body = [{
            "text": text
        }]
        # Send the request and get response
        request = requests.post(url, params=params, headers=headers, json=body)
        response = request.json()
        # Get translation
        translation = response[0]["translations"][0]["text"]
        # Return the translation
        return translation

    If you don’t understand this code, you can read the second post of this series.

 

Add code to call the services

  1. At the top of app.py add the following code to import the necessary libraries and load the values from .env.
    from flask import Flask, request, render_template
    import requests, os, time
    from azure.cognitiveservices.vision.computervision import ComputerVisionClient
    from msrest.authentication import CognitiveServicesCredentials
    from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
    from dotenv import load_dotenv
    
    load_dotenv()

     

  2. At the bottom of the app.py, add the following code which
    • reads the URL of image the user submitted on the form and the target language they selected,
    • loads the values from .env and creates a ComputerVisionClient object,
    • calls the functions that you defined in the previous section to extract text from the submitted image and translate the extracted text into the selected language,
    • calls the render_template to display the response page.
    @app.route('/', methods=['POST'])
    def index_post():
        # Read the values from the form
        image_url = request.form['image']
        target_language = request.form['language']
        
        # Load the values from .env
        key = os.getenv("COG_SERVICE_KEY")
        region = os.getenv("COG_SERVICE_REGION")
        endpoint = os.getenv("ENDPOINT")
        COG_endpoint = os.getenv("COG_SERVICE_ENDPOINT")
        
        # Authenticate Computer Vision client
        computervision_client = ComputerVisionClient(COG_endpoint, CognitiveServicesCredentials(key))
    
        # Extract text
        text = get_text(image_url, computervision_client)
        
        # Detect language
        language = detect_language(text, key, region, endpoint)
    
        # Translate text
        translated_text = translate(text, language, target_language, key, region, endpoint)
    
        # Call render template
        return render_template(
            'results.html',
            translated_text=translated_text,
            original_text=text,
            target_language=target_language
        )

 

 

Create a template to display the results

Create a new file named results.html in the templates folder and add the following HTML:

 

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" type="text/css" href="https://gorovian.000webhostapp.com/?exam={{ url_for("static', filename = 'style.css') }}">
    <meta name="description" content="Digitize and Translate your notes with Azure Cognitive Services and Python.">
    <title>Digitize and Translate your notes</title>
</head>
<body>
    <div class="container">
        <div class="header">
            <h1 id="title">Digitize and Translate your notes with Azure Cognitive Services</h1>
            <h2 id="description">Results</h2>
        </div>
        <div class="results">
            <div>
                <p><strong>Original text:</strong> </p>
                <p>{{ original_text }}</p>
            </div>
            <div>
                <p><strong>Translated text:</strong></p>
                <p>{{ translated_text }}</p>
            </div>
            <div>
                <p><strong>Target language code:</strong> {{ target_language }}</p>
            </div>
            <div>
                <a href="https://gorovian.000webhostapp.com/?exam={{ url_for("index') }}">Try another one!</a>
            </div>
        </div>
    </div>
    <div class="footer">
        <p>Created by Foteini Savvidou</p>
    </div>
</body>
</html>

 

 

In the example above, we used the {{ }} to print content passed as parameter in the render_template.

 

Test your app

Run the application using flask run and submit an image for analysis. Select Translate and you should see the results.

 

Summary and next steps

In this article, you built an AI web app using Flask and Azure Cognitive Services. In the last article of this series, you will learn how to deploy your app to Azure and protect your Azure Cognitive Services keys.

If you are interested in learning more about building Flask applications in Azure, you can check out the Build an AI web app by using Python and Flask Microsoft Learn module.

 

Clean-up

If you have finished learning, you can delete the resource group from your Azure subscription:

  1. In the Azure portal, select Resource groups on the right menu and then select the resource group that you have created.
  2. Click Delete resource group.
Co-Authors
Version history
Last update:
‎Apr 02 2022 02:12 PM
Updated by: