10-19-2018, 05:05 AM
Creating 3D Models From Photos Using Meshroom
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom.png" width="554" height="515" title="" alt="" /></div><div><p><a href="https://alicevision.github.io/#meshroom">Meshroom</a> is a new, free and open source photogrammetry software from AliceVision. Photogrammetry software enable you to create a 3D scene using a series of photographs, generally the more the merrier. Currently documentation is a bit lacking, so I’ve decided to create this quick tutorial. In this tutorial we are going to quickly walk through the process of using Meshroom using a photoset <a href="https://blenderartists.org/t/meshroom-free-photogrammetry/1120286/98">available here</a>. That post links to a zip file containing 50 images that are confirmed to work with Meshroom. Simply extract them somewhere on your drive. Of course you need to download Meshroom, which is available for download <a href="https://github.com/alicevision/meshroom/releases/tag/v2018.1.0">right here</a>. Simply download the archive, extract then execute the Meshroom application. Note <strong>Meshroom requires a CUDA GPU and works on Windows and Linux!</strong> So this process will only work on nVidia GPUs, at least as of time of writing.</p>
<p>Once you’ve got Meshroom loaded, follow the following simple steps.</p>
<p>Drag extracted images into the Images pane on the left.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3400.png"><img width="554" height="515" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom.png" border="0" /></a></p>
<p>Save your project somewhere</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3401.png"><img width="207" height="173" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-1.png" border="0" /></a></p>
<p>Click the green Start button.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3402.png"><img width="176" height="47" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-2.png" border="0" /></a></p>
<p>As it’s running, you will see the progress across the top:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3409.png"><img width="804" height="34" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-3.png" border="0" /></a></p>
<p>This bar indicates a problem occurred. You can divine more details by locating the current task in the Graph, like so:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3410.png"><img width="424" height="176" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-4.png" border="0" /></a></p>
<p>With the node selected, check the Log (bottom right corner) for details. This is the error message you receive if you run the process on a non-CUDA (nVidia) GPU. Keep in mind, it can also be caused by the process running on a laptop with Optimus, not automatically selecting the right GPU.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3411.png"><img width="554" height="274" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-5.png" border="0" /></a></p>
<p>Hopefully however you encounter no such errors and the process is entirely green. On the bright side, it should pick right up where the error occured if you run the project again in the same directory, as Meshroom caches the results of each step as it goes. You will find the vast majority of time is spent on the DepthMap section, this is normal. As the process continues, you should start seeing results in the 3D viewer.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3413.png"><img width="604" height="619" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-6.png" border="0" /></a></p>
<p>You can pan, zoom and orbit the mouse using the LMB, scroll wheel and MMB respectively. More detail in the point cloud will fill in as the process runs. Once it completes successfully, you will see a button Load Model.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3414.png"><img width="804" height="77" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-7.png" border="0" /></a></p>
<p>You can now preview the results of your effort!</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=FinalGif.gif"><img width="354" height="355" title="FinalGif" alt="FinalGif" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom.gif" /></a></p>
<p>A few steps in, it will have evaluated all of your photos, acceptable/usable photos will be marked with a green checkmark.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3412.png"><img width="504" height="439" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-8.png" border="0" /></a></p>
<p>Review the remaining photos for flaws and inconsistency if you run it again.</p>
<p>Go make some tea… it’s going to be between 10 minutes and an hour depend on the speed of your machine. Once the process is complete, there will be a folder called MeshroomCache, with the following contents:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3403.png"><img width="453" height="268" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-9.png" border="0" /></a></p>
<p>This is a folder with all of the output files from each step in the process. Generally what you are going to be interested in is the obj, mtl and texture file in the Texturing folder. This can be imported into your 3D modelling application of choice, the obj format is fairly universal. The resulting mesh is extremely dense and you may consider checking out <a href="https://www.youtube.com/watch?v=PR4KrKHqVTI">Instant Meshes</a> for optimizing the results.</p>
<p>Now that you know it works, it’s time to start refining the process or providing your own picture set. I would recommend the following tips from my own experience:</p>
<ul>
<li>use an actual camera, not a phone. I got terrible results from my Pixel phone, but my Canon DLSR gave much better results. YMMV</li>
<li>DO NOT green/white screen your background. Unique markers in the background help Meshroom position each virtual camera</li>
<li>try to get the entire object in frame on each shot</li>
<li>get rid of any image with any blurring</li>
</ul>
<p>So far we just default settings in the Graph Editor. This graph represents a graph of nodes in the process, one for each directory shown in the screenshot above. Note when you select a node, there are a number of properties you can edit:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3404.png"><img width="581" height="263" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-10.png" border="0" /></a></p>
<p>You can also connect pins to multiple nodes to create multiple results. For example, if you wanted to create a set of TFF and lower resolution PNG textures, you can do the following. Right click the graph editor and select Texturing:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3405.png"><img width="340" height="241" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-11.png" border="0" /></a></p>
<p>This will create a new node in the graph. Now drag the output node from MeshFiltering and connect it to ini and inputMesh. </p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3406.png"><img width="414" height="238" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-12.png" border="0" /></a></p>
<p>Select the new Texturing node and have it create a lower detail texture set:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3408.png"><img width="579" height="272" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-13.png" border="0" /></a></p>
<p>Now when it runs, it will create two sets of textures for you. Note there are other nodes such as Decimation that aren’t in the process by default. You can see the entire thing in process in the <a href="https://www.youtube.com/watch?v=R0PDCp0QF1o">video</a> attached below:</p>
<p align="center">[embedded content]</p>
<p class="under"> <span class="categories"><a href="https://www.gamefromscratch.com/category/Art.aspx">Art</a></span> <span class="tags"><a href="https://www.gamefromscratch.com/?tag=/Tutorial" rel="tag">Tutorial</a> <a href="https://www.gamefromscratch.com/?tag=/3D" rel="tag">3D</a></span>
</p>
<hr />
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom.png" width="554" height="515" title="" alt="" /></div><div><p><a href="https://alicevision.github.io/#meshroom">Meshroom</a> is a new, free and open source photogrammetry software from AliceVision. Photogrammetry software enable you to create a 3D scene using a series of photographs, generally the more the merrier. Currently documentation is a bit lacking, so I’ve decided to create this quick tutorial. In this tutorial we are going to quickly walk through the process of using Meshroom using a photoset <a href="https://blenderartists.org/t/meshroom-free-photogrammetry/1120286/98">available here</a>. That post links to a zip file containing 50 images that are confirmed to work with Meshroom. Simply extract them somewhere on your drive. Of course you need to download Meshroom, which is available for download <a href="https://github.com/alicevision/meshroom/releases/tag/v2018.1.0">right here</a>. Simply download the archive, extract then execute the Meshroom application. Note <strong>Meshroom requires a CUDA GPU and works on Windows and Linux!</strong> So this process will only work on nVidia GPUs, at least as of time of writing.</p>
<p>Once you’ve got Meshroom loaded, follow the following simple steps.</p>
<p>Drag extracted images into the Images pane on the left.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3400.png"><img width="554" height="515" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom.png" border="0" /></a></p>
<p>Save your project somewhere</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3401.png"><img width="207" height="173" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-1.png" border="0" /></a></p>
<p>Click the green Start button.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3402.png"><img width="176" height="47" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-2.png" border="0" /></a></p>
<p>As it’s running, you will see the progress across the top:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3409.png"><img width="804" height="34" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-3.png" border="0" /></a></p>
<p>This bar indicates a problem occurred. You can divine more details by locating the current task in the Graph, like so:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3410.png"><img width="424" height="176" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-4.png" border="0" /></a></p>
<p>With the node selected, check the Log (bottom right corner) for details. This is the error message you receive if you run the process on a non-CUDA (nVidia) GPU. Keep in mind, it can also be caused by the process running on a laptop with Optimus, not automatically selecting the right GPU.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3411.png"><img width="554" height="274" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-5.png" border="0" /></a></p>
<p>Hopefully however you encounter no such errors and the process is entirely green. On the bright side, it should pick right up where the error occured if you run the project again in the same directory, as Meshroom caches the results of each step as it goes. You will find the vast majority of time is spent on the DepthMap section, this is normal. As the process continues, you should start seeing results in the 3D viewer.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3413.png"><img width="604" height="619" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-6.png" border="0" /></a></p>
<p>You can pan, zoom and orbit the mouse using the LMB, scroll wheel and MMB respectively. More detail in the point cloud will fill in as the process runs. Once it completes successfully, you will see a button Load Model.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3414.png"><img width="804" height="77" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-7.png" border="0" /></a></p>
<p>You can now preview the results of your effort!</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=FinalGif.gif"><img width="354" height="355" title="FinalGif" alt="FinalGif" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom.gif" /></a></p>
<p>A few steps in, it will have evaluated all of your photos, acceptable/usable photos will be marked with a green checkmark.</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3412.png"><img width="504" height="439" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-8.png" border="0" /></a></p>
<p>Review the remaining photos for flaws and inconsistency if you run it again.</p>
<p>Go make some tea… it’s going to be between 10 minutes and an hour depend on the speed of your machine. Once the process is complete, there will be a folder called MeshroomCache, with the following contents:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3403.png"><img width="453" height="268" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-9.png" border="0" /></a></p>
<p>This is a folder with all of the output files from each step in the process. Generally what you are going to be interested in is the obj, mtl and texture file in the Texturing folder. This can be imported into your 3D modelling application of choice, the obj format is fairly universal. The resulting mesh is extremely dense and you may consider checking out <a href="https://www.youtube.com/watch?v=PR4KrKHqVTI">Instant Meshes</a> for optimizing the results.</p>
<p>Now that you know it works, it’s time to start refining the process or providing your own picture set. I would recommend the following tips from my own experience:</p>
<ul>
<li>use an actual camera, not a phone. I got terrible results from my Pixel phone, but my Canon DLSR gave much better results. YMMV</li>
<li>DO NOT green/white screen your background. Unique markers in the background help Meshroom position each virtual camera</li>
<li>try to get the entire object in frame on each shot</li>
<li>get rid of any image with any blurring</li>
</ul>
<p>So far we just default settings in the Graph Editor. This graph represents a graph of nodes in the process, one for each directory shown in the screenshot above. Note when you select a node, there are a number of properties you can edit:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3404.png"><img width="581" height="263" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-10.png" border="0" /></a></p>
<p>You can also connect pins to multiple nodes to create multiple results. For example, if you wanted to create a set of TFF and lower resolution PNG textures, you can do the following. Right click the graph editor and select Texturing:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3405.png"><img width="340" height="241" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-11.png" border="0" /></a></p>
<p>This will create a new node in the graph. Now drag the output node from MeshFiltering and connect it to ini and inputMesh. </p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3406.png"><img width="414" height="238" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-12.png" border="0" /></a></p>
<p>Select the new Texturing node and have it create a lower detail texture set:</p>
<p><a href="https://www.gamefromscratch.com/image.axd?picture=image_3408.png"><img width="579" height="272" title="image" alt="image" src="http://www.sickgaming.net/blog/wp-content/uploads/2018/10/creating-3d-models-from-photos-using-meshroom-13.png" border="0" /></a></p>
<p>Now when it runs, it will create two sets of textures for you. Note there are other nodes such as Decimation that aren’t in the process by default. You can see the entire thing in process in the <a href="https://www.youtube.com/watch?v=R0PDCp0QF1o">video</a> attached below:</p>
<p align="center">[embedded content]</p>
<p class="under"> <span class="categories"><a href="https://www.gamefromscratch.com/category/Art.aspx">Art</a></span> <span class="tags"><a href="https://www.gamefromscratch.com/?tag=/Tutorial" rel="tag">Tutorial</a> <a href="https://www.gamefromscratch.com/?tag=/3D" rel="tag">3D</a></span>
</p>
<hr />
</div>