1. Home
2. Posts
3. Person detection in video streams using Python in 2023: a tutorial

Person detection in video streams using Python in 2023: a tutorial

I show some simple steps to implement person detection in video using Python in 2023.

Author

Affiliation

Sid Metcalfe

Cartesian Mathematics Foundation

Introduction to Person Detection in Video Streams

As someone who’s delved into the intricacies of computer vision, I can tell you firsthand that person detection in video streams is a vibrant frontier in modern AI. It’s remarkable how a few snippets of Python code can enable a machine to identify and track human presence within a sea of visual data.

What captivates me most is the sheer applicability of person detection. From security systems to crowd monitoring, retail analytics to smart homes, the implications are profound. But I understand that for beginners, the journey from raw video to actionable insights can seem daunting.

Let’s break it into digestible steps.

Consider a video stream as a sequence of images, or frames, shown rapidly to create the illusion of motion. Person detection merely requires the analysis of these individual frames. I’ve often started by exploring a single image before I step up to the complexity of video streams. Here’s an elementary Python block using OpenCV, a powerful library for computer vision tasks, which you can use to read an image:

import cv2

# Load an image using OpenCV
image = cv2.imread('person.jpg')

# Display the image in a window
cv2.imshow('Window Name', image)

# Wait and close the window with any key press
cv2.waitKey(0)
cv2.destroyAllWindows()

Now, imagine extrapolating this to a video. Videos, essentially, are image sequences. So, you fetch frames one at a time and apply detection on each frame:

import cv2

# Load a video stream
cap = cv2.VideoCapture('video.mp4')

# Loop through each frame
while True:
    # Read a frame
    success, frame = cap.read()
    if not success:
        break

    # Typically, you'll insert person detection logic here

    # Display the frame
    cv2.imshow('Frame', frame)

    # Break loop with a 'q' key press
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release the capture after finishing
cap.release()
cv2.destroyAllWindows()

With the foundations set, we then inject person detection algorithms. The choices are vast, but for newcomers, a pre-trained model could ease the ride. OpenCV conveniently offers access to several such models. Known as Haar cascades or HOG + SVM detectors, these are great to get your hands wet:

# Initialize HOG descriptor/person detector
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())

(cap = cv2.VideoCapture('video.mp4'))

while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Detect people in the frame
    (regions, _) = hog.detectMultiScale(frame, winStride=(4, 4), padding=(8, 8), scale=1.05)

    # Draw bounding boxes around detected regions
    for (x, y, w, h) in regions:
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

    cv2.imshow('Frame', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Clean up
cap.release()
cv2.destroyAllWindows()

While the code above represents the threshold of person detection, it’s worth noting that specialized models exist for higher accuracy and are often based on neural networks. TensorFlow and PyTorch are among the popular ones, hosting models like YOLO (You Only Look Once) and SSD (Single Shot Multibox Detector).

I encourage you to check out GitHub repositories like the official YOLO website for code and trained models. Additionally, comprehensive datasets for training and benchmarking, such as COCO (Common Objects in Context), are indispensable, and you can find them on their official site.

The balance between real-time performance and precision is the art behind person detection. Remember, this is just the start—you’ll need to explore the Python environment setup, model choices, specific implementation, and optimization to fully harness the potential of person detection in video streams.

Embarking on this journey is exhilarating. So gear up, stay curious, and let Python be your guide through the thrilling realm of video stream analysis.

Setting Up the Python Environment

Getting your Python environment up and running is the foundational step before diving into person detection in video streams. It’s a mix of installing the right tools, setting up a clean workspace, and ensuring your system can handle the tasks you’ll throw at it.

Let’s dive in.

First, I usually start with ensuring Python is installed. As of 2023, Python 3.8 or newer should be your go-to. You can verify Python installation and version by running:

python --version

If Python isn’t installed or you need an upgrade, head to python.org to download the latest version for your operating system.

Now, irrespective of your platform, one tool I cannot recommend enough is virtualenv. It allows you to create isolated Python environments. Trust me, it’s a lifesaver, especially when you’re juggling multiple projects with different dependencies.

You can install virtualenv using pip:

pip install virtualenv

Once installed, create a new environment within your project directory:

virtualenv person_detection_env

To activate the virtual environment, on Unix or MacOS, use:

source person_detection_env/bin/activate

On Windows, the script is in the Scripts folder:

person_detection_env\Scripts\activate

Your command line will indicate that you’re now in the virtual environment by prefacing the prompt with (person_detection_env).

Next up: let’s talk libraries. For person detection tasks, some heavy lifting is done by libraries such as opencv-python for handling video streams, and tensorflow or pytorch, depending on the model you choose for detection.

Install them using pip within your virtual environment:

pip install opencv-python tensorflow

or

pip install opencv-python torch torchvision

If you’re wondering why both TensorFlow and PyTorch are mentioned, it’s a matter of preference and the model’s compatibility. I’ll cover that in a later section.

But what’s a car without fuel? Indeed, we need datasets and pre-trained models. One remarkable source for models optimized for a variety of tasks is the TensorFlow Model Zoo or the Torchvision Models, which provide easy-to-integrate models.

Lastly, let’s grab a sample video to test our setup. Here’s how you can do it directly from Python using wget:

import wget

sample_video_url = 'http://example.com/sample_video.mp4'
wget.download(sample_video_url, 'sample_video.mp4')

Make sure you replace http://example.com/sample_video.mp4 with a legitimate URL to a video file you have permission to use.

And there you have it! With your Python environment now primed and ready, your next steps involving coding for person detection will feel less like hitting roadblocks and more like embarking on an exciting adventure. Remember to check the next sections where I’ll take you deeper into choosing the right detection model and how to implement it with actual code examples.

Choosing the Right Person Detection Model

Choosing the right person detection model is akin to picking the perfect ally in a complex game of chess. Each model comes with its unique set of strengths and weaknesses, and I’ve found that the context of the problem effectively dictates the choice of the model. Here, I’ll walk you through a couple of the popular choices for person detection and how you can leverage them in Python.

First up, let’s talk about OpenCV’s Haar Cascades. Despite being a bit old school, they’re incredibly fast and fairly reliable for straightforward scenarios. I typically start with this, mainly because they’re uncomplicated to deploy. Here’s a snippet to get a taste:

import cv2

# Load the Haar cascade file for person detection
person_cascade = cv2.CascadeClassifier('haarcascade_fullbody.xml')

# Initialize video capture
cap = cv2.VideoCapture('video.mp4')

while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Convert frame to grayscale
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    # Perform detection
    persons = person_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5)

    for (x, y, w, h) in persons:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 255, 0), 2)

    cv2.imshow('Person Detection', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

For a more robust and sophisticated approach, I turn to deep learning-based models. You’ve got models like YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) with pre-trained weights that you can find easily on GitHub repositories or shared through research papers.

Here’s an example using a pre-built YOLO model with OpenCV’s DNN module:

import cv2
import numpy as np

# Load YOLO
net = cv2.dnn.readNet('yolov3.weights', 'yolov3.cfg')
layer_names = net.getLayerNames()
output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]

# Loading image
cap = cv2.VideoCapture('video.mp4')

while True:
    ret, img = cap.read()
    if not ret:
        break
    
    height, width, channels = img.shape

    # Detecting objects
    blob = cv2.dnn.blobFromImage(img, 0.00392, (416, 416), (0, 0, 0), True, crop=False)
    net.setInput(blob)
    outs = net.forward(output_layers)

    # Information for each object detected
    for out in outs:
        for detection in out:
            scores = detection[5:]
            class_id = np.argmax(scores)
            confidence = scores[class_id]
            if class_id == 0 and confidence > 0.5:
                # Object detected
                center_x = int(detection[0] * width)
                center_y = int(detection[1] * height)
                w = int(detection[2] * width)
                h = int(detection[3] * height)

                # Rectangle coordinates
                x = int(center_x - w / 2)
                y = int(center_y - h / 2)

                cv2.rectangle(img, (x, y), (x + w, y + h), (255, 255, 0), 2)

    cv2.imshow('Person Detection', img)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Remember, while YOLO is incredibly powerful, it’s also computationally intensive. Make sure you’ve got a decent GPU to back it up, or you might find yourself watching a slideshow instead of a video stream.

Ultimately, the person detection model decision comes down to your specific needs: If you’re looking for speed and simplicity, Haar Cascades are your go-to. But for accuracy and the ability to work in complex scenarios, deep-learning models like YOLO or SSD are the heavy hitters.

While these snippets should give you a solid starting point, each model has a plethora of parameters you can tweak. I’ve always found experimenting with these parameters to be the best way to learn and to tailor the model to fit precise requirements. Keep tinkering, and you’ll find the perfect balance of speed and accuracy for person detection in your application.

Implementing Person Detection with Code Examples

Implementing person detection in video streams is a multifaceted process that demands some familiarity with Python and a willingness to tinker with code examples. Once you’ve got your environment set up and chosen a person detection model, the real fun begins. Here, I’ll offer a step-by-step walkthrough of how to implement person detection using OpenCV and a pre-trained YOLO (You Only Look Once) model.

Firstly, make sure OpenCV is installed:

pip install opencv-python-headless

YOLO models can be downloaded from the official website or directly via links. We need the weights file and the configuration file, which encapsulate the architecture and the trained model, respectively.

Start by loading the YOLO model:

import cv2
import numpy as np

# Load YOLO
net = cv2.dnn.readNet("yolov3.weights", "yolov3.cfg")
layer_names = net.getLayerNames()
output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]

Details about YOLO and other object detection models can be found in seminal research papers and repos linked on the official YOLO website.

Next, set up a function to load the video:

def load_video(video_path):
    cap = cv2.VideoCapture(video_path)

    if not cap.isOpened():
        print("Error: Could not open video.")
        exit()

    return cap

Now, create a function detect_person that takes each frame from the video, feeds it through YOLO, and returns the frame with person detections:

def detect_person(frame):
    height, width, channels = frame.shape

    # Detecting objects
    blob = cv2.dnn.blobFromImage(frame, 0.00392, (416, 416), (0, 0, 0), True, crop=False)
    net.setInput(blob)
    outs = net.forward(output_layers)

    class_ids, confidences, boxes = [], [], []
    for out in outs:
        for detection in out:
            scores = detection[5:]
            class_id = np.argmax(scores)
            confidence = scores[class_id]

            # Filter out person class (usually 0) and low confidence detections.
            if confidence > 0.5 and class_id == 0:
                # Object detected
                center_x = int(detection[0] * width)
                center_y = int(detection[1] * height)
                w = int(detection[2] * width)
                h = int(detection[3] * height)

                # Rectangle coordinates
                x = int(center_x - w / 2)
                y = int(center_y - h / 2)

                boxes.append([x, y, w, h])
                confidences.append(float(confidence))
                class_ids.append(class_id)

    # We use Non-Maximum Suppression to refine the boxes
    indexes = cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.4)

    for i in range(len(boxes)):
        if i in indexes:
            x, y, w, h = boxes[i]
            # Draw a rectangle with label
            cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)  

    return frame

Finally, put it all together to process the video stream:

cap = load_video('people_walking.mp4')

while True:
    ret, frame = cap.read()
    if not ret:
        break
    
    frame = detect_person(frame)

    # Display
    cv2.imshow("Person Detection", frame)

    # Stop if escape key is pressed
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break

# Release the VideoCapture object
cap.release()
cv2.destroyAllWindows()

This basic skeleton will process each video frame, identify persons using YOLO, and draw boxes around them. You can enhance this by considering optimization techniques, adjusting the confidence level, playing with Non-Maximum Suppression or augmenting it with further details to each detected person such as labeling.

Remember, this is a starting point. The performance of your person detection system can vary based on the video quality, the YOLO model used (there are different versions, like YOLOv3 or YOLOv4), environmental conditions, and more. Dive into the documentation of OpenCV and YOLO, play with different parameters, and see how each change affects your output.

Optimizing and Troubleshooting Your Person Detection System

The moment of truth for any person detection system comes when it’s deployed in the real world. You’ve done the groundwork—chosen your model, fed it data, and now it’s show time. But what if things don’t go exactly as planned? With a cool head and a few tricks up your sleeve, you can optimize and troubleshoot your system to better tailor it to real-world conditions.

Let’s start with some optimization strategies. Suppose you’re using OpenCV, a popular computer vision library in Python, and you’re noticing that your model isn’t quite snappy. What can you do to boost its performance? First, you might consider resizing the frames you’re processing:

import cv2

def resize_frame(frame, scale=0.75):
    width = int(frame.shape[1] * scale)
    height = int(frame.shape[0] * scale)
    dimensions = (width, height)

    return cv2.resize(frame, dimensions, interpolation=cv2.INTER_AREA)

By adjusting the scale parameter, you’re effectively reducing the amount of data your model has to work through, which can speed up detection times. But be warned, go too low, and you might miss some faraway figures.

Now, what about when your system is wrought with false positives and negatives? A bit of parameter tuning goes a long way. If you’re using a pre-trained model from OpenCV’s DNN module, adjust the confidence threshold:

net.setInput(blob)
detections = net.forward()
conf_threshold = 0.5  # Try varying this threshold

for i in range(detections.shape[2]):
    confidence = detections[0, 0, i, 2]
    if confidence > conf_threshold:
        # Process detection

If your model is overconfident and seeing people where there are none, push that conf_threshold higher. If it’s overly cautious, lower it a bit—but not too much, or you’ll be back to square one.

When you’re dealing with video streams, latency can be a killer. I can’t count the number of times threading saved my bacon. With Python’s threading library, you can read frames in a separate thread, keeping your processing pipeline chugging along smoothly:

import cv2
import threading

class VideoStreamThread(threading.Thread):
    def __init__(self, src=0):
        super(VideoStreamThread, self).__init__()
        self.capture = cv2.VideoCapture(src)
        self.grabbed, self.frame = self.capture.read()
        self.started = False

    def start(self):
        if self.started:
            print("Thread already started!")
            return None
        self.started = True
        self.thread.start()

    def run(self):
        while self.started:
            self.grabbed, self.frame = self.capture.read()

    def read(self):
        return self.frame

    def stop(self):
        self.started = False
        self.thread.join()

Instantiate this with VideoStreamThread(0) for your primary camera, start it up, and simply read() to get the latest frame when needed.

Troubleshooting can be daunting when you’re new to all this. I remember when I first dove in, a simple misstep like neglecting to release resources or handle exceptions could turn my codebase upside down. So, always wrap your capture and processing loop within a try-except block:

video_thread = VideoStreamThread(0)
try:
    video_thread.start()
    while True:
        frame = video_thread.read()
        # Your detection logic here
except KeyboardInterrupt:
    pass
finally:
    video_thread.stop()
    cv2.destroyAllWindows()

With these strategies, you’ll be taking your person detection project from a promising prototype to a tuned and robust system ready for the unpredictability of the real world. And that’s where the true excitement lies—watching your creation operate in the wild, adapting and learning as it goes. For more detailed guidance on developing your machine learning skills, consider reading A step-by-step guide to object recognition using Python. Remember, machine learning is as much about tweaking and adapting as it is about algorithms and data. Keep experimenting, stay patient, and enjoy the process!