Models

Face tracking model

class pftracker.modules.models.FaceTracking_2D(movModel, obsModel, N, size_v, first_frame, detector, v, video_stream, saveVideo=None)

Model for 2D face tracking based on particle filter.

This class describes a face tracking on video sequences model created for applying particle filter algorithms. Here are defined the main characteristics and behavior of the target model.

Parameters

• movModel (createMovModel) – Dynamic model

• obsModel (ObsModels) – Observation model

• N (int) – Number of particles

• size_v (int) – State vector size

• detector (str) – Face detector algorithm

• v (VideoCapture or VideoStream) – reference to the webcam (VideoCapture object) or .mp4 or .avi video file (VideoStream object)

• video_stream (bool) – This variable indicates if we’re working with VideoCapture or VideoStream

• saveVideo (str, optional) – Path to the output video file. Default is None

calc_error(gt_file)

Evaluate particle filter algorithm results.

This function calculate precision and recall metrics for the resulting face tracking.

Parameters

gt (str) – path to ground truth .txt file

Returns

8-element tuple containing

• P (array): precision value per frame, array with (1, number_of_frames) dimension

• R (array): recall value per frame, array with (1, number_of_frames) dimension

• P_mean (float): precision mean value

• R_mean (float): recall mean value

• P_std (float): precision standard deviation value

• R_std (float): recall standard deviation value

• F1Score (float): F-1-score metric

• F1Score_std (float): F-1-score standard deviation value

close_e(ended)

Close video file if it has reached to the end or if the window of the video has been closed.

getEstimation()

Get particle filter estimation.

initialization()

Create intial particles distribution.

Sample particles from initial distribution, init particles in x and y at bounding box center position resulting from face detector algorithm.

Returns

(array) with (size_v, N) dimension, particles at first time

prediction(particles)

Propagate particles from time k-1 to k using a dynamic model.

Parameters

particles (array) – Model specific representation of all particles, with (size_v, N) dimension.

Returns

2-element tuple containing

• particles (array): predicted particles array x_{k} with (size_v, N) dimension.

• uk (array): u_{k} array with (size_v, N) dimension. This is a characterization of x_{k}|x_{k-1} (move particles from x_{k-1} to x_{k} without include process noise).

saveEstimation(file_name)

Save the particle estimation for each frame into a .txt file

Parameters

file_name (str) – path to output .txt file

saveOutputVideo()

Save output visualization into a .avi video file.

update(particles)

Evaluate predicted particles x_{k}.

Parameters

particles (array) – predicted particles array x_{k} with (size_v, N) dimension

Returns

(array) of likelihoods p(z_{k}|x_{k}) with (1,N) dimension

update_apf(particles)

Evaluate predicted particles x_{k}^{idx}.

Evaluate predicted particles x_{k}^{idx} for the second stage weights of auxiliary particle filter algorithm. Here idx are the indixes resulting from resampling of the first stage weigths of auxiliary particle filter algorithm.

Parameters

particles (array) – predicted particles array x_{k}^{idx} with (size_v, N) dimension

Returns

(array) of likelihoods p(z_{k}|x_{k}^{idx}) with (1,N) dimension

visualizations(estimate, particles)

Visualization function for particles, resulting estimation and bounding box.

This function visualize the set of particles, the resulting estimation and the bounding box in each iteration of the particle filter algorithm, showing all these in each frame of the analyzed video

Parameters

• estimate (array) – estimated particle filter tracking result array with (size_v, 1) dimension

• particles (array) – particles array x_{k} with (size_v, N) dimension

Dynamic model definition

class pftracker.modules.models.createMovModel(estate_var)

Create a dynamic model for propagating particles to the next state.

Parameters

estate_var (str) – State space model

Supported estate_var:

• ‘dynamic_bbox’: Self updating bounding box model.

  In this model the state vector is described by [x, y, Vx, Vy], where (x,y) specify the location of the center of the face window (bounding box) in the image coordinate system and (Vx,Vy) represent the motion velocity. Here the boundig box width is calculated in a self-updating model.

• ‘5_variables’: Five variables state space model.

  The width of bounding box is inluded in the state space, being the state vector define as: [x, y, Vx, Vy, w].

• ‘6_variables’: Six variables state space model.

  Here the state vector is given by: [x, y, Vx, Vy, w, Vw], where Vw is the corresponding rate of width change.

Returns

2-element tuple containing

• dlg_model (dlg): dlg object class

• size_ve (int): state vector size

dlg Model

class pftracker.modules.models.dlgModel.dlg(F, muW, SigmaW)

Propagate particles from time k-1 to k using a Discrete-time Linear and gaussian model in the prediction step of particle filters.

Parameters

• F (array) – State transition matrix with (state_vector_size, state_vector_size) dimension

• muW (array) – Noise-system mean vector with (state_vector_size,1) dimension

• SigmaW (array) – Noise-system covariance matrix with (state_vector_size, state_vector_size) dimension

move_particles(xk_1, N)

Move particles from time k-1 to time k.

Parameters

• xk_1 (array) – particles in the previous state, with (state_vector_size,N) dimension.

• N (int) – number of samples

Returns

2-element tuple containing

• xk (array): particles in the actual state with (state_vector_size,N) dimension.

• muk (array): characterization of x_{k}|x_{k-1} where particles are moved particles from x_{k-1} to x_{k} without include the process noise.

Self-updating model

class pftracker.modules.models.self_updating_bbox(center, bbox, dl, particles, N)

Self updating bounding box model.

Parameters

• center (list) – 2-elements list of particle filter estimation (x,y coordinates of bounding box center)

• bbox (int) – bounding box width at previos time

• dl (float) – average distance from the previous frame to the target center

• particles (array) – particles array x_{k} with (size_v, N) dimension

• N (int) – number of particles

Returns

2-element tuple containing

• bbox_new (int): new bounding box width prediction

• d (float): average distance between the particles and the target center

Observation models

class pftracker.modules.models.ObsMod(model, N)

Observation Model base class.

Parameters

• bins (list) – number of bins of the HSV histogram. The list contains 3 values corresponding to the number of bins for each component of the HSV color space (h, s and v)

• N (int) – number of particles

calcDistance(image, particles, s)

Calculate the likelihood of each particle.

Parameters

• image (array) – frame at time k

• particles (array) – particles at time k

• s (int) – bounding box width

Returns

(array) of likelihoods p(z_{k}|x_{k}) with (1,N) dimension

calcHist_ref(first_frame, bounding_box)

Calculate reference histogram.

first_frame (array): first frame of the video sequences bounding_box (array): face bounding box for the first frame

HSV color-based model

class pftracker.modules.models.colorhist.HSVModel.hsvModel(roi, N, l=20)

HSV color-based model.

It defines an HSV model for calculating the likelihoods of particles at actual time k.

Parameters

• hist_ref (array) – reference HSV histogram

• hsvHistCalc (HSVHistogram) – image descriptor (3D HSV histogram)

• N (int) – number of particles

• l (int, optional) – lambda Bhattacharyya distance coefficient

calcLikelihood(image, particles, s)

Calculate the likelihood of each particle.

This function calcultes the distance between the reference histogram and the histograms obtained for the actual set of particles at time k. To do this it is used the Bhattacharyya distance metric.

Parameters

• image (array) – frame at time k

• particles (array) – particles at time k

• s (int) – bounding box width

Returns

(array) of likelihoods p(z_{k}|x_{k}) with (1,N) dimension

class pftracker.modules.models.colorhist.hsvhistogram.HSVHistogram(bins)

3D HSV histogram calculation.

Parameters

bins (list) – number of bins the histogram will use. The list contains 3 values corresponding to the number of bins for each component of the HSV color space.

calc_Hist(image)

Returns a 3D histogram for an image in the HSV colorspace.

Parameters

image (array) – image from wich to create the HSV histogram

LBP-based model

class pftracker.modules.models.lbp.LBPModel.lbpModel(roi, N)

LBP-based model.

Parameters

• hist_ref (array) – reference LBP histogram

• lbpHistCalc (HSVHistogram) – image descriptor (LBP histogram)

• N (int) – number of particles

calcLikelihood(image, particles, s)

Calculate the likelihood of each particle.

This function calcultes the distance between the reference histogram and the histograms obtained for the actual set of particles at time k. To do this it is used the Alternative CHI Square distance metric.

Parameters

• image (array) – frame at time k

• particles (array) – particles at time k

Returns

(array) of likelihoods p(z_{k}|x_{k}) with (1,N) dimension

class pftracker.modules.models.lbp.lbphistogram.LBPHistogram(numPoints, radius)

LBP histogram calculation.

Parameters

• numPoints (int) – number of sampling points

• radius (int) – radius from the center pixel

calc_Hist(image, eps=1e-07)

Returns the LBP histogram of an image.

Parameters

• image (array) – image from wich to create the LBP histogram

• eps (float, optional) – minimum for avoiding histogram non defined calculation (division by zero)