Mapping Camera Coordinates to a 2D Floor Plan
Data mining is a big business. These times everything is analysed. Everyone is analysing mouse clicks, mouse movements, customer purchase patterns. Such analysis has proven to give profitable insights that are driving businesses further than ever before. Big companies such as google, amazon and Facebook are in heavy use of it. It is used to keep track of customers and feed the
Data can be mined from many sources, actual inputs, pictures, and sensors, But not many people have considered data mining videos. What about all that security footage that has stacked up over the years? Can we mine those for profitable insights also? Of course, It is possible!
the plotting of tracked customers or staff from video footage onto a 2D floor plan.
Why would you want to do this?
Well, plotting on a 2D plane will allow you to more easily data mine for things such as common movement patterns or common places of congestion at particular times of the day. This is powerful information to possess. For example, if you can deduce what products customers reached for in what order you can make important decisions with respect to the layout of your shelves and placement of advertising. You can see the corners of shop which a customer’s mostly peek and maybe avoid a theft.
Another benefit of this technique is that it is also much easier to visualise movement patterns presented on 2D plane rather than when shown on distorted CCTV footage
However, if you have ever tried to undertake this task you may have come to the understanding that it is not as straightforward as you initially thought. A major dilemma is that your security camera images are distorted. For example, a one-pixel movement at the top of your image corresponds to a much larger movement in the real world than a one pixel movement at the bottom of your image.
Where to begin? In tackling this problem, the first thing to realise is that we are dealing with two planes in the Euclidean space. One plane (the floor in your camera footage) is “stretched out”, while the other is “laid flat”. We, therefore, need a transformation function to map points from one plane to the other.
The following image shows what we are trying to achieve (assume the chessboard is the floor in your shop/business):
The task: change the perspective without loosing important information.
The next step, then, is to deduce what kind of transformation is necessary. Once we know this, we can start to look at the mathematics behind it and use this maths accordingly in our application. Here are some possible transformations:
Different types of transformations
Translations (the first transformation in the image above) are shifts in the x and y plane that preserve orientation. Euclidean transformations change the orientation of the plane but preserve the distances between points – not our case, as mentioned earlier. Affine transformations are a combination of translation, rotation, scale, and shear. They can change the distances between points, but parallel lines remain parallel after transformation – also not our case. Lastly, we have homographic transformations that can change a square into any form of a quadrilateral. This is what we are after.
Getting the corresponding point pairs from our images is easy, too. It is possible to achieve the same thing with Adobe Photoshop or GIMP. If you move your mouse over an image, the pixel coordinates of your mouse positions are given at the bottom of the window. Jot down the pixel coordinates from one image and the corresponding pixel coordinates in the matching image. Get at least four such.
points pairs and you can then get an estimate of H and use it to calculate any other corresponding point pairs.
Example of 3 corresponding points in two images
Now you can take the tracking information from your security camera and plot the position of people on your perspective 2D floor plan. You can now analyse their walking paths, where they spent most of their time, where congestion frequently occurs, etc.