3d-scanner

realsense-3d-scanner

3D Scanning Software for intel realsense Depth cameras

How to start

First create a build folder

mkdir build && cd build 

Then run cmake to create a Makefile

cmake ..

Run make to build the project

make

Execute the compiled project (use sudo when on macOS, see known issue(s))

./3d-scanner

How to use

After startup you will be presented with a small Menu:

Welcome to the 3d-scanner, choose your option: 
(1) Scan
(2) Align and save pointclouds
(3) Wipe old scans
(4) exit 

(1) - will do exactly one scan in the current position and sae it. You will have to do multiple scans from different perspectives to align them later on.

(2) - will take your scans and use the iterative closest point algorithm to stitch them together (this takes ages in the current version and definitly needs improvement). After the stitching is finished the scans will be saved seperately in the "scans_fitted" folder and the originals will be saved in the "scans_original" folder as .ply files.

Afterwards all the aligned files can be imported in a programm which allows viewing .ply files (like Meshlab).

(3) - will delete all scans scince startup

(4) - will shutdown the 3d-scanner

How it works

1. scanning

When scanning an object, the intel realsense camera will take multiple pictures to adjust the autofocus. Then the software extracts the depth image, appliies a couple of filters to enhance the image and finally creates a pointcloud from it.

This pointcloud will then be transformed from the "realsense" pointcloud format to the "pointcloudlibrary" format for further processing.

Multiple scans will create multiple pointclouds.

2. alignment of multiple scans

When there are multiple scans from more than one perspective they need alignment so they are not floating at different locations when viewing them. Therefore the iterative closest point alorithm is used. Simplified, this alorithm matches each point of one pointcloud to each point of the base pointcloud (the first scan) and then transforms it so its points are aligned with the base cloud to create a coherent scan.

At the end of this process there are still as many pointclouds as scnans. However they are now aligned with each other.

3. saving the scans

When the alignment is done, all the pointclouds will get exported as .ply files and saved in a folder named "scans_fitted". The original scans without alignment will be saved in a folder called "scans_original" beforehand.

All of them need to get imported in Meshlab or a similar tool in order to view the whole scan.

What is there still to do?

1. everything is reaaaally slow

• doing one scan takes a while, because its always necessary to take a couple of shots for the autofocus to settle
  • How to fix it: Maybe it is possible to take all scnas by using one videostream instead of taking pictures
• the ICP Algorithm is the slowest part because it is not optimized yet
  • How to fix it: Optimize the ICP by setting an initial matrix which guids it towards the right transformation. There shoudl also be a couple more options for optimization

2. improve the scan in order to improve the alignment

• when the scan covers not only the desired object but also the floor or other surroundings the alignemnt doesnt work. Therefore the scan needs to get cropped or somehow seperated from its surroundings

3. transform the pointclouds to a mesh

• currently all pointclouds are seperate .ply files at the end. However it would be nice to merge all of them together and then create a mesh from them.

4. an interface

• When everything else works it would be great to have a grfaphical user interface to interact with the scanning software.

known issue(s)

On Mac there can occure either segmentation faults or a realsense error (Timeout when waiting for camera frames) when running the executable. Somehow it will work again if you just execute the 3d-scanner-startup-helper often enough (this is basically the realsense hello-world example). This will also fail a lot with segmentation faults but after a while it will work again. At least as long as the camera is connected, when reconnecting the error usually appears again. I am not sure if this Error is Mac specific or wheater I am doing something wrong. However I did not manage to resolve it properly. If you have any Idea what the reason could be, feel free to fix it.

Another inconvenience I discovered on Mac is, that the executable can only be executed using sudo, otherwise it will always fail with a segmentation fault.