Clemens Schöll

The Artist's Guide to BundleFusion with a Kinect

Let's assume happen to have the right hardware (see below) and you'd like to work with BundleFusion. It sure looks cool.
Except, it's not so easy to get to work as it maybe promised. Here is my somewhat-more-in-depth (as compared to the README) setup process:

Pre-requisites

Hardware

  • Kinect360 with the slightly overpriced USB 2.0 adapter or KinectOne (aka. v2) with the overpriced USB 3.0 adapter (not)
  • Reasonable PC with a reasonable Nvidia GPU (don't ask me what that means)
  • MSFT Windows 10 :(

Drivers

  • Nvidia drivers (figure this out yourself, for my VR things I needed the most up-to-date available)
  • CUDA 7.0 (do a custom install to preserve your shiny new drivers)
  • MSFT Visual Studio 2013 (you can get that at their website after registration for free)
  • for Kinect360
    • Kinect SDK 1.8
  • for KinectOne

Don't bother thinking about using a recent CUDA or Visual Studio version. Just don't.

Downloading and setting up the code

If you use git it might be more straight-forward, but since I never properly set up my MSFT Windows install, I downloaded the ZIPs directly from GitHub. 1. Grab a ZIP of the master of the repo. Unpack it to BundleFusion-master. 1. Grab a ZIP of the mLib submodule at the correct revision. Confirm this by navigating to /external/. Unpack this and replace BundleFusion-master/external/mLib. 1. Grab the mLib external libraries from this Dropbox link they provide. This seems to be a bunch of dependencies or builds -- but no idea, really. Unpack and place next to BundleFusion-master.

This should leave you with the following folder structure. Double check this, it'll screw you up. txt BundleFusion-master/ external/ mLib/ # this is the submodule you replaced data/ src/ [...] FriedLiver/ [...] FriedLiver.sln [...] mLibExternal/ # you downloaded this from Dropbox include libsWindows [...]  

Adjusting the code for your needs

Open BundleFusion-master/FriedLiver/FriedLiver.sln with Visual Studio 2013 (make sure it's really the 2013 version, Help > About Microsoft Visual Studio). Open the file GlobalAppState.h and change line 3 to 10 to the following to enable your Kinect(s): cpp #define KINECT #define KINECT_ONE //#define OPEN_NI #define BINARY_DUMP_READER //#define INTEL_SENSOR //#define REAL_SENSE //#define STRUCTURE_SENSOR #define SENSOR_DATA_READER  

Now switch to a production (Release) build. The Debug builds don't work for me, missing FreeImage.dll. Check that your code and build config look somewhat like this:

VS screenshot

Build

Go to Build > Build Solution of CTRL + SHIFT + B and hope for the best. If you get any errors, you're on your own now. Otherwise: Congratulations! You have now compiled BundleFusion, which is about 80% of the hastle to get to an actual 3D scan.

Configure

Copy the two files zParametersBundlingDefault.txt and zParametersDefault.txt from to BundleFusion-master/FriedLiver to BundleFusion-master/FriedLiver/x64/Release. Open them with a reasonable editor (e.g. Notepad++, not the regular Notepad, it doesn't understand line-endings).

As you can see, there is a lot of things to configure. Trial and error FTW. Run it after every change to see if it helped. First of all, in zParametersDefault.txt in line 2 specify the sensor you're going to use. I only got the Kinect360 to work, so that means cpp s_sensorIdx = 0;

Another thing that was crucial for getting it to work for me was changing line 49 in zParametersDefault.txt to cpp s_hashNumSDFBlocks = 100000; //smaller voxels require more space because 100000 is suggested in a comment in the original file.

Try to run it

Start BundleFusion-master/FriedLiver/x64/Release/FriedLiver.exe. If that crashes with txt bundlefusion-master\friedliver\source\depthsensing\VoxelUtilHashSDF.h(135) : cudaSafeCall() Runtime API error 2: out of memory. You'll want to go back to configuration and randomly change some numbers.

Shortcuts for interaction

Once you got the little preview window up and running, you'll quickly wonder how the hell you're supposed to interact with this software: Where's the colored view? And how do you export anything at all?

This bug hints at reading the source code (obvious place to look for shortcuts), specifically Source/DepthSensing/DepthSensing.cpp, where starting at line 205 we find this: cpp g_pTxtHelper->DrawTextLine(L"Controls "); g_pTxtHelper->DrawTextLine(L" \tF1:\t Hide help"); g_pTxtHelper->DrawTextLine(L" \tF2:\t Screenshot"); g_pTxtHelper->DrawTextLine(L" \t'R':\t Reset scan"); g_pTxtHelper->DrawTextLine(L" \t'9':\t Extract geometry (Marching Cubes)"); g_pTxtHelper->DrawTextLine(L" \t'8':\t Save recorded input data to sensor file (if enabled)"); g_pTxtHelper->DrawTextLine(L" \t'7':\t Stop scanning"); g_pTxtHelper->DrawTextLine(L" \t'6':\t Print Timings"); g_pTxtHelper->DrawTextLine(L" \t'<tab>':\t Switch to free-view mode"); g_pTxtHelper->DrawTextLine(L" \t"); g_pTxtHelper->DrawTextLine(L" \t'1':\t Visualize reconstruction (default)"); g_pTxtHelper->DrawTextLine(L" \t'2':\t Visualize input depth"); g_pTxtHelper->DrawTextLine(L" \t'3':\t Visualize input color"); g_pTxtHelper->DrawTextLine(L" \t'4':\t Visualize input normals"); g_pTxtHelper->DrawTextLine(L" \t'5':\t Visualize phong shaded"); g_pTxtHelper->DrawTextLine(L" \t'H':\t GPU hash statistics"); g_pTxtHelper->DrawTextLine(L" \t'T':\t Print detailed timings"); g_pTxtHelper->DrawTextLine(L" \t'M':\t Debug hash"); g_pTxtHelper->DrawTextLine(L" \t'N':\t Save hash to file"); g_pTxtHelper->DrawTextLine(L" \t'N':\t Load hash from file"); The most important keys are 9 (export) and 2 (color).

Look at the results

The exported .ply files are stored in BundleFusion-master\FriedLiver\x64\Release\scans and named scan.ply, scan1.ply et cetera. Work out a system for yourself.

Post-process

Have a look at this two part tutorial by Intel to get your model simplified, textured and (optionally) into Unity.

To be continued

Okay, I lied. Things aren't working super well and most likely you couldn't follow the instructions. But I hope you're a little further now.