image segmentation - Determining the covariance of point clouds in real-time

So basically I have a set of multidimensional data that I need to determine the covariance of between dimensions in real-time. Each point that comes in is a vector. I have gotten the mean and variance per dimension using this answer: Determining the mean and standard deviation in real time. But I also need to find the covariance between dimensions, resulting in a covariance matrix. Each vector is 3 dimensional and is a point in space.

Does anyone know how to do this and does this?

To explain further, I want the covariance matrix so I can then use the eigenvalues and eigenvectors to characterize a point cloud. The point clouds are segments of a scanned environment using the Kinect. I need to make sure the segments are labelled consistently over frames. I will do this by matching the mean and covariance eigenvectors/eigenvalues of segments between frames. I am working with unity.

Edit:

Or what would be equally useful would be how to get the covariance from just the variances and means that I already have.