From reviewing the paper The Untapped Potential of Low Cost Photogrammery in Community-Based Archaeology, it was easy to see that photogrammetry can be used as an alternative to high resolution surveying with high costs.
Traditional softcopy photogrammetric methods require the 3-D location and pose of the camera(s), or the 3-D location of ground control points to be known to facilitate scene triangulation and reconstruction. In contrast, the SfM method solves the camera pose and scene geometry simultaneously and automatically, using a highly redundant bundle adjustment based on matching features in multiple overlapping, offset images [Westoby et al 2012].
Structure from motion is the geometry of creating 3D models from 2D images. If you are interested in the maths behind SfM, visit Prof. Rob Fergus’s lecture notes on Computer Vision, or geometric vision, in particular Lecture 6 on Multiview Stereo & Structure from Motion. It is quite interesting to see the process. Thankfully, to Wu Changchang, we don’t need to understand the mathematics behind how Structure from Motion works, to be able to use it. In his post-doc year at University of Washington Seattle, he developed Visual SfM.
[Fergus, 2013]
VisualSfM is a GUI application for 3D reconstruction from structure from motion [Changchang 2011]. Open-source, free for personal, non-profit, or academic use, it allows you to upload a series of photographs and turn them into 3D models. I downloaded Visual SfM, and the Multicore Bundle Adjustment, a series of other small programs you need to run Visual SfM. In order to create dense point clouds, you need these extra code/programs, as they run the dense matching and point cloud generation on larger sets of images. However, I found the instructions difficult to follow, and Visual SfM did not work for me the first time round.
After several searches, eventually, I found an excellent tutorial Make a 3D Model from Pictures on Instructables, a website for fans of DIY, to share videos and instructions on what they have made. In his tutorial, JuiceW, has the most clear instructions on how to install all the various components, and with direct links to them. If you are a member of Instructables, you can download the instructions as a PDF, otherwise, you can just view them on the screen.
Once I have all the software downloaded and installed, I began the program. I created a series of screen-shots of the process, to help chart the way.
Using Visual SfM
Firstly, I took a series of photographs of one of the Ogham Stones at Cólaiste Íde, in Burnham. (I have described Burnham house in a previous blog post, you can view it here).
To properly create a 3D model from photographs, it is recommended you use a lot of photographs, 50 or more. For the purpose of testing out Visual SfM, I used less than half of that. The process can take a while depending on the size of the random-access memory (RAM), the computer data storage, and I wanted to explore and learn how to use the program, with a minimum of ease, before embarking on a larger scale project.
STEP 1: Uploading Images
The first step is to open the Visual SfM program. Just note, there is a smaller, separate Log Window, where you can see the process in action. This Log Window, has a habit of disappearing, and if it does, don’t panic! Firstly I move it to the left of the screen, it seems less likely to disappear while there. Another way to check the Log Window is open; just under File, you will see a page icon, this will Show/Hide Log Window.
To access and upload the images, simply click on File, and select Open+ Multiple Image. Or you can click the icon that looks like a document folder, folder+. This will open a new window, where you can select the images to be uploaded. If you want to select all the images, simple click Ctrl + A, and then click Open
STEP 2: Computing Missing Matches
You can see in the Log Window, all the image files being uploaded. It will tell you when it is complete. The next step is to match the images. You will do this by pressing the icon Computing Missing Matches that has four arrows in each direction, and each a different colour.
STEP 3: Sparse Reconstruction
In next step, the program will begin to find all the relevant matching points between the images and start creating the 3D construct, calculated for their 3D positions in a relative coordinate system. To start this process, click on the Sparce Reconstruction button. This button looks like two arrows on top of each other, the foreground one is red, the background one is blue.
The 3D structure that is being reconstructed from the series of 2 D images. The Sparce here refers to a sparse set of points and lines on the object surface: the process of inferring sparse 3D structure of a static scene from a moving camera. [SMALL Project, 2016].
Sparsity corresponds to systems which are loosely coupled. Consider a line of balls connected by springs from one to the next: this is a sparse system as only adjacent balls are coupled. By contrast, if the same line of balls had springs connecting each ball to all other balls, the system would correspond to a dense matrix. The concept of sparsity is useful in combinatorics and application areas such as network theory, which have a low density of significant data or connections. [Wikipedia 2016]
STEP 4: Dense Reconstruction
After you have completed your Sparse Reconstruction, the next step is to create the Dense Reconstruction, or CMVS. When you click on the CMVS icon, a new window will open, and ask you to name the data files, or created cloud point, and the location you want to save it. Once you have chosen, you will see the relative process of dense reconstruction in the Log Window. This may take some time, depending on the quantity of images.
CMVS was developed by Yasutaka Furukawa, who like Wu Changchang, was also a postdoc at University of Washington. Many multi-view stereo (MVS) algorithms do not scale well to a large number of input images due to the lack of lack of computational and memory resources. This software (CMVS) takes the output of a structure-from-motion (SfM) software as input, then decomposes the input images into a set of image clusters of manageable size. [Furukawa, 2010].
He also developed PMVS, which is usually now included in the CMVS package. PMVS is another MVS; it takes a set of images and camera parameters, then reconstructs 3D structure of an object or a scene visible in the images. Only rigid structure is reconstructed, in other words, the software automatically ignores non-rigid objects such as pedestrians in front of a building. The software outputs a set of oriented points instead of a polygonal (or a mesh) model, where both the 3D coordinate and the surface normal are estimated at each oriented point [Furukawa, Ponce, 2010].
STEP 5: Save as NView Match
The final step in Visual SfM, is to save the file. This is done, by clicking on the SfM icon, scrolling down to Save NView Match. Click on this, give the file a name and then save.
VisualSfM saves SfM workspaces into NVM files, which contain input image paths and multiple 3D models [Furukawa 2010]. NVM are essentially text files, and could be opened with a text editor, such as Notepad, but because of the huge quantity of 3D points, it would crash it.
Finish
The next step is now to open up the file in MeshLab, which, again, is an open source program you can use in the construction of 3D models.
MeshLab is an open source, portable, and extensible system for the processing and editing of unstructured 3D triangular meshes. The system is aimed to help the processing of the typical not-so-small unstructured models arising in 3D scanning, providing a set of tools for editing, cleaning, healing, inspecting, rendering and converting this kind of meshes. [Meshlab]
I wont be going to MeshLab just yet. The 3D model I made, in this example, is not of great quality. As I said previously, I wanted to test out the program, and see how long it takes for just a handful of images to work. Also, the processing system; would my laptop be able to handle it, and is it possible to work on other things, while the program is running. It is interesting to use the program just to get an idea of the kind of photographs that need to be taken. What you look for in a ‘nice’ photo, is not what you want for a photogrammetry exercise. The greyer the day the better, with minimum amount of light variation. Also, making the ‘object’ as full in the frame as possible, cuts out on other potential unimportant points.
I will update the blog, as I learn more about Visual SfM, on tweaking images, deleting unnecessary points, etc.
Bibliography:
Changchang, Wu, ‘VisualSFM : A Visual Structure from Motion System’, 2011 <http://ccwu.me/vsfm/>
Changchang, Wu, Sameer Agarwal, Brian Curless, and Steven Seitz M., ‘VisualSFM : A Visual Structure from Motion System’ <http://ccwu.me/vsfm/#demo>
Fergus, Rob. (2013). ‘Lecture 6: Multi-View Stereo & Structure from Motion – 11_12_multiview.pdf’ <http://cs.nyu.edu/~fergus/teaching/vision/11_12_multiview.pdf>
Furukawa, Yasutaka, (2010) ‘CMVS’, Clustering Views for Multi-View Stereo (CMVS), <http://www.di.ens.fr/cmvs/>
Furukawa, Yasutaka, and Jean Ponce, ‘PMVS2’, Patch-Based Multi-View Stereo Software (PMVS – Version 2) <http://www.di.ens.fr/pmvs/>
SMALL Project (2016) ‘Learning Structures — SMALL Project’ <http://www.small-project.eu/keyresults/learning-structures-in-dictionaries>
W, Juice, ‘Using the First Program: Visual SFM’, Instructables.com <http://www.instructables.com/id/Make-a-3D-model-from-pictures/>
Westoby, M. J., J. Brasington, N. F. Glasser, M. J. Hambrey, and J. M. Reynolds, ‘“Structure-from-Motion” Photogrammetry: A Low-Cost, Effective Tool for Geoscience Applications’, Geomorphology, 179 (2012), 300–314 <http://dx.doi.org/10.1016/j.geomorph.2012.08.021>
Wikipedia (2016). ‘Sparse Matrix’, Wikipedia, the Free Encyclopedia, 2016 <https://en.wikipedia.org/w/index.php?title=Sparse_matrix&oldid=731601498>
One Comment