Search⌘ K
AI Features
Home
Courses
Mastering Artificial Intelligence with Azure Cognitive Services

Object Detection in the Images

Explore how to implement object detection in images using Azure Computer Vision API. Learn to detect objects with location coordinates and confidence scores. Understand how to use detect_objects and detect_objects_in_stream functions to analyze images from both URLs and local files, and how to visualize detected objects with bounding boxes.

We'll cover the following...

Introduction

In this lesson, we’re going to build a script to detect objects in an image using the Computer Vision API. The Computer Vision API detects the objects in an image and returns the bounding-boxthe co-ordinates of the objects present in the image. This differs from generating the tags for the image because the tags are generated for contextual terms too (like indoor, outdoor, etc.) which cannot be represented using the bounding-boxes.

Implementation

We’ll be using the detect_objects() function from the ComputerVisionClient class. This function accepts the URL of an image and helps to detect all the objects with their corresponding locations from an image. The response is in the JSON format and contains the objects identified with their location coordinates and the confidence score with which these objects are identified from the image. If an error occurs on the computer vision end, this function will return an error code with an error message.

Later in the lesson, we’ll also explore the detect_objects_in_stream() function. This function works the same way that detect_objects() works. The only difference is that the detect_objects_in_stream() function accepts an image in bytes format. So, we can read the image from the local machine and pass the image data to this function to identify the objects present in the image.

Now let’s move on to the implementation of this functionality.

Python
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials
from draw import draw_rectangles
import shutil
client = ComputerVisionClient(
                    computer_vision_endpoint, 
                    CognitiveServicesCredentials(computer_vision_key)
                )
image_url = "https://cdn.pixabay.com/photo/2019/01/25/13/47/beautiful-people-3954533_960_720.jpg"
detect_objects_results = client.detect_objects(image_url)
if len(detect_objects_results.objects) == 0:
    print("No objects detected.")
else:
    print("Detected objects in image:")
        
    draw_rectangles(image_url, detect_objects_results.objects)
    for obj in detect_objects_results.objects:
        print("Type of object:", obj.object_property)
        print("Confidence Score:", obj.confidence)
        print("object at location {}, {}, {}, {}".format( \
        obj.rectangle.x, obj.rectangle.x + obj.rectangle.w, \
        obj.rectangle.y, obj.rectangle.y + obj.rectangle.h))
    # Do not modify the below lines as these are required to view the output in the browser.
    shutil.copy2('./image.png', './output/image.png')

Explanation

The explanation of main.py is given below:

  • From lines 1-4, we import the required packages.

  • In lines 6-9, we create an instance of the ComputerVisionClient class and pass the endpoint and subscription key to its constructor.

  • In line 11, we define the URL to fetch the image(You can replace this URL with the one with your image).

  • In line 13, we call the detect_objects() function using the client object that we have just created in the above steps and passed the image URL.

  • In line 15, we check whether the computer vision resource identified any object or not.

  • If it has identified an object, then in line 20, we call the draw_rectangle() function to create a rectangle over the objects identified and then from lines 22-26, we print the type of the object that has been identified, the confidence score with which the object is identified, and the coordinates of the object that has been detected by our computer vision resource.

The explanation of draw.py is given below:

  • From lines 1-3, we import the required packages.

  • From line 5-11, we create the coordinates to generate a rectangle on top of the objects identified.

  • From lines 13-21, we create a draw_rectangles() function which first reads the image from the URL and then iterate over all the object identified. Each object is then marked with a red bounding box.

So, in this way, we can use the Computer Vision’s Object Detection capabilities.

Python
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials
from draw import draw_face_rectangles
import shutil
client = ComputerVisionClient(
                    computer_vision_endpoint, 
                    CognitiveServicesCredentials(computer_vision_key)
                )
image = open("CourseAssets/object_detection_image.jpg","rb")
detect_objects_results = client.detect_objects_in_stream(image)
image.close()
if len(detect_objects_results.objects) == 0:
    print("No objects detected.")
else:
    
    draw_face_rectangles(detect_objects_results.objects)
    print("Detected objects in image:")
    for obj in detect_objects_results.objects:
        print("Type of object:", obj.object_property)
        print("Confidence Score:", obj.confidence)
        print("object at location {}, {}, {}, {}".format( \
        obj.rectangle.x, obj.rectangle.x + obj.rectangle.w, \
        obj.rectangle.y, obj.rectangle.y + obj.rectangle.h))
    # Do not modify the below lines as these are required to view the output in the browser.
    shutil.copy2('./image.png', './output/image.png')

The only difference, is in lines 11, 13, and 15 in the file main.py, we read the image from the local system and then use the detect_objects_in_stream()function.