Introducing a new an easy way to create a single high-resolution Linux desktop using multiple projectors. Anyone can achieve this with the following easy-to-use tools:
lt-align: the LightTwist projector alignment tool is a simple point-and-click interface that lets the user specify a single projection area with multiple overlapping projectors and generate distortion and blending maps.
lt-compiz-plugin: A plugin for Compiz 0.9x that warps and blends the output of your projectors using the deformation maps produced by lt-align, creating one single seamless high resolution desktop area.
Requirements
The LightTwist projector alignment tool and compiz plugin have a few dependencies which must be met. It has been developed on Ubuntu 11.04 which includes Compiz 0.9 in its distribution.
The plugin cannot be used with Compiz versions prior to 0.9x.
Ubuntu 11.04 users can simply execute the following command:
sudo apt-get install subversion cmake cmake-curses-gui libpng12-dev\ libnetpbm10-dev libavcodec-dev libavformat-dev libavutil-dev\ libswscale-dev libcv-dev libcvaux-dev libhighgui-dev libfftw3-dev\ libopenscenegraph-dev libboost-dev libboost-serialization-dev\ compiz-dev compizconfig-settings-manager xsltproc
For earlier releases of Ubuntu and other distributions, you can get the latest version of Compiz on their website: www.compiz.org
Downloads and installation
IMGU
IMGU is our extensive image manipulation library, which is used in both the plugin and the projector alignment tool.
First, checkout the source using subversion:
svn co https://svn3d.iro.umontreal.ca/svn/imgu/devel imgu
then go to the newly created directory and build the library:
cd imgu cmake . make sudo make install
lt-align
Download, compile and install lt-align:
mkdir ~/svn3d cd ~/svn3d svn co https://svn3d.iro.umontreal.ca/svn/vision3d/lt-align cd lt-align cmake . make sudo make install
lt-compiz-plugin
Download, compile and install lt-compiz-plugin (Compiz plugins cannot be built from the base directory. You must create a ‘build’ directory and compile from there):
mkdir ~/svn3d
cd ~/svn3d
svn co https://svn3d.iro.umontreal.ca/svn/vision3d/lt-compiz-plugin
cd lt-compiz-plugin
mkdir build
cd build
cmake ..
make
make install ## no sudo! it will be installed
## in .compiz-1/ in your home directory
How to use lt-align
command line options
$lt-align -h
LightTwist projector alignment tool
Version 0.1
Usage:
-h Displays this help message
-screen<n> <x0> <y0> <width> <height> Defines a screen and its
dimensions
-o <filename> Saves setup to file
-i <filename> Loads setup from file
(overrides -screen args)
no arguments Uses Xinerama to initialize
all screens
Controls
<PageUp> or <PageDn> : Cycle through all overlaps. Pairs of projectors will become blue. When no projector pair is selected, you may define/adjust the four white crosshairs defining your desired projection area.
<space> Deselect current projector pair, so you can edit your desired projection area (white crosshairs).
Keys <1> and <2>: Select which of the two screens of the current overlap you wish to define/modify points.
Arrow keys: subpixel adjustment.
<s>: Save setup and generate distortion maps.
<ESC>: quit.
Instructions
1 – Define the desired projection area:
Make sure no projector pair is selected (press <space>).
Click on the four full screen corners (left top, right top, right bottom, left bottom)
2 – Specifiy matching points for each projector overlap
First, press <PageDn> to select the first projector pair. If they do not overlap on the projection surface, continue cycling with <PageDn>. When you have selected a pair of projectors that do overlap, click on 4 points in the first screen WITHIN the overlapping zone.
Then press <2> to select the second projector of the overlap pair. Now, click on the same 4 points, but in the second screen. Be as precise as you can. The crosshairs with 2 circles should be together. Since the crosshairs and lines in the first screen are red, and green in the second screen, the crosshairs and lines will turn yellow if they are well aligned.
You may make sub-pixel adjustments to a crosshair’s positions with the arrow keys while the mouse points hovers over it.
Do this for every pair of overlapping projectors, until lines appear connecting the 4 white crosshairs defining your desired projection area.
When you are satisfied with your alignment, press <s> to save your setup and generate your distortion maps.
You may now use those maps with the LightTwist Compiz plugin
How to use lt-compiz-plugin
First of all you must manually restart compiz un a terminal:
compiz --replace ccp
The default Ubuntu installation does not include the ‘ccp’ option when launching Compiz at startup. Without the ‘ccp’ option, Compiz will not load the LightTwist plugin at all. Until that is fixed, you will have to manually restart Compiz every time you want to use th plugin.
Enabling the plugin
Start the CompizConfig Settings manager from the System>Preference menu, or start it from a terminal by typing ‘ccsm’.
Go to the Utility section, to the Mouse Position Polling and set the Mouse Poll Interval to 1 (default value is 40)
Go to Effects section. Enable the lighttwist plugin
Locate the fragment shader : set it to
<your home directory>/.compiz-1/lighttwist/data/deform.frag
Leave the vertex shader field empty.
Now enter the distortion maps and parameters for each screen:
click “New”
In the Edit window:
Lut: select the lut_<n>.png file generated for each projector
Blend : (leave empty)
Blend embeded : checked
Proj num : set it to the projector number <n>
Gamma : set to 1 initially. You may adjust it to your liking afterwards
Your settings should look something like this:
And now you are ready to activate the LightTwist Compiz plugin! Simply use the following hot keys:
<shift>F9 : activate/deactivate plugin
<shift>F7 : activate/deactivate deformation
… and enjoy!


Pingback: LGM 2011 report | Animtim home
Pingback: Desktop multi-projecteur simple « institut Arts Cultures et Technologies
Dear sir,
I’m trying to compile the imgu library. I’m having the following error :
[ 92%] Building C object examples/CMakeFiles/imguMinGray.dir/imguMinGray.o
/root/multi-projector/imgu/examples/imguMinGray.c: In function ‘main’:
/root/multi-projector/imgu/examples/imguMinGray.c:86:3: warning: implicit declaration of function ‘guptaPattern’
/root/multi-projector/imgu/examples/imguMinGray.c:86:3: warning: nested extern declaration of ‘guptaPattern’
Linking C executable imguMinGray
CMakeFiles/imguMinGray.dir/imguMinGray.o: In function `main’:
/root/multi-projector/imgu/examples/imguMinGray.c:86: undefined reference to `guptaPattern’
collect2: ld returned 1 exit status
make[2]: *** [examples/imguMinGray] Error 1
make[1]: *** [examples/CMakeFiles/imguMinGray.dir/all] Error 2
make: *** [all] Error 2
I don’t know if this error is relative to some “no found” libraries at configure stage:
– Found NETPBM: /usr/lib/libnetpbm.so
Could not find cminpack
Could not find rqueue. Building without thread support.
Could not find prosilica
– Found V4L:
– Found OSG library: /usr/lib/libosg.so;/usr/lib/libosgFX.so;/usr/lib/libosgViewer.so;/usr/lib/libosgText.so;/usr/lib/libosgGA.so;/usr/lib/libosgDB.so;/usr/lib/libOpenThreads.so;/usr/lib/libosgUtil.so
– Found OSG library dirs: /usr/lib
– Found OSG inc dirs: /usr/include
osg include dir is /usr/include
– Checking GNUCXX version 3/4 to determine OpenCV /opt/net/ path
– Found BMC: /usr/local/lib/libbmc.so
Could not find pt (pantilt control)
CMake Warning at CMakeLists.txt:208 (find_package):
Could not find module FindPROFILER.cmake or a configuration file for
package PROFILER.
Adjust CMAKE_MODULE_PATH to find FindPROFILER.cmake or set PROFILER_DIR to
the directory containing a CMake configuration file for PROFILER. The file
will have one of the following names:
PROFILERConfig.cmake
profiler-config.cmake
Could not find profiler
Could not find profilometre
Could not find ANN library
– Found FFMPEG library: /usr/lib/libavcodec.so;/usr/lib/libavutil.so;/usr/lib/libavformat.so;/usr/lib/libswscale.so
– Found FFMPEG library dirs: /usr/lib
– Found FFMPEG inc dirs: /usr/include
– Found FFT3W: /usr/lib/libfftw3_threads.so;/usr/lib/libfftw3.so
– Found Doxygen: /usr/bin/doxygen
– Configuring done
CMake Warning (dev) at src/CMakeLists.txt:129 (add_library):
Policy CMP0003 should be set before this line. Add code such as
if(COMMAND cmake_policy)
cmake_policy(SET CMP0003 NEW)
endif(COMMAND cmake_policy)
as early as possible but after the most recent call to
cmake_minimum_required or cmake_policy(VERSION). This warning appears
because target “imgu” links to some libraries for which the linker must
search:
stdc++, bmc, m, m
and other libraries with known full path:
/root/multi-projector/imgu/src/libmatrixmath.so
CMake is adding directories in the second list to the linker search path in
case they are needed to find libraries from the first list (for backwards
compatibility with CMake 2.4). Set policy CMP0003 to OLD or NEW to enable
or disable this behavior explicitly. Run “cmake –help-policy CMP0003″ for
more information.
This warning is for project developers. Use -Wno-dev to suppress it.
– Generating done
– Build files have been written to: /root/multi-projector/imgu
Could you help me please?
Thanks in advance,
Gustavo Adolfo
Dear Gustavo,
It seems we have a small problem with a test program called imguMinGray.
If you are using the svn (it seems so), just update now (svn up). I removed the imguMinGray program from the CMakeList.txt in examples.
Cheers,
Sebastien
Actually, the rqueue library is needed to fully compile imgu (see link to install). However, it is not needed to get lt-align working.
From a clean install of 11.04 following the above, all compiled and installed great! BTW, I did start by trying to hack an opensuse 11.4 but ran into too many missing components (in case other try… I’m sure you can get it working there as well but I lost patients).
I’m wanting to play with this code in a dome we have set up at LA Center Studios (Los Angeles), where is the code for using a camera to create the deformation maps/and or how do we do that?
Almost done… We are putting the finishing touch on a camera-based version of lt-align. The full lighttwist software is much more complicated and experimental than the new lt-align/lt-compiz software. It requires a camera and support cylindrical projections, both standard and in 3D.
Its probably better to wait for the new lt-align-camera which will feature new inter-reflexion resistant scanning, full 3d models of the screen, and remove the requirement to have a single camera that can see the whole screen (of course you then have to move the camera around).
For maximum simplicity, we intend to support webcams as well as industrial cameras (we use prosilica ge650 typically). Also, to get a 3D screen model, two cameras will be initially be required. We’ll see. Our initial tests are promising.
Is possible to use for 6 or 9 projectors, for example, for a planetariums domes?
The user interface assumes that the projection surface is planar, so it will not work for a dome.
However, the compiz plugin itself, the one that makes the live deformation of the desktop, uses a “deformation map” (or “LUT”, “Lookup Table”) which can easily account for a curved projection surface.
We are currently working on a version of lt-align that will rely on cameras instead of manually clicking points. In that case, it will be easy to support a dome surface. The compiz plugin will stay the same.
If you already know the equation that maps each screen pixel to a projector pixel, it is relatively easy to create a LUT image yourself. We can document the image format for you.
Thanks. We have a big dome; and actually we trying to make it digital. Still we are looking the projector that we need, and your software sound very great for this project. Do you have a e-mail? To explain it better.
Hi; I’m very interesed in the system.
Dear Sébastien,
Can you, please, share some info about lt-align-camera. Will it use multiple calibrated cameras and projectors? Will it use 3D surface reconstruction? Or will it still require a camera in the center of the eye-point with a full projection area view?
Do you already have some code to share?
Thanks in advance.
Nikola
Receiving the following error when trying to compile imgu:
Building CXX object src/CMakeFiles/imgu.dir/plugins/BlackmagicInputCallback.o
In file included from /home/ubuntu/svn3d/imgu/src/plugins/BlackmagicInputCallback.cpp:1:0:
/home/ubuntu/svn3d/imgu/src/plugins/BlackmagicInputCallback.h:8:25: fatal error: DeckLinkAPI.h: No such file or directory
compilation terminated.
make[2]: *** [src/CMakeFiles/imgu.dir/plugins/BlackmagicInputCallback.o] Error 1
make[1]: *** [src/CMakeFiles/imgu.dir/all] Error 2
make: *** [all] Error 2
Any ideas? I would really like to give this a go.
Thanks In Advance!
Sorry for that.
We added a proper check for BlackMagic frame grabber in the CMakeList.txt.
It should work fine now. SImply update your version with “svn up” and tell me if there are any more problems.
Ok, everything compiles now, but when trying to activate the plugin, I get a “cannot use FBO” error.
Shader Compile Error : 0:1(10): error: GLSL 1.30 is not supported. Supported versions are: 1.10, and 1.20
This is on a stock Ubuntu 11.04 install. No extras, no security patches. I have tried before and after upgrading (apt-get upgrade) and I get the same results. Using ATI proprietary drivers, or the built in ATI drivers.
Also, using the normal ubuntu drivers, when running lt-align, if I press the S key to save, Xorg freezes. I can still ALT+F1 to get to the terminal, but Xorg will not recover. I have to reboot to get the desktop back. The .png files are not created.
Hey there guys I am working together with Pikoro. We really want to give LightTwist a try. We are currently using ATI cards…Is it a possibility that we need to use a Nvidia card…We have tried everything..Even different computers with different configurations. If you can let us know what graphics cards you are using or perhaps if you have some insite into his questions above that would be a huge help. We really love what we have seen in your videos so far and are very exited to try this. Please feel free to email either of us and can hash this out. Once again you guys have made a very cool project….
Would this work inside a virtual machine?
If so, it would be great if you could create VM (vmware convertor?) so it could be used for people to try it easily.
Cheers
In Ubuntu 11.10 I’m having a problem with the cmake:
[WARNING] One or more dependencies for compiz plugin lighttwist not found. Skipping plugin.
Missing dependencies : compiz-mousepoll
The mousepoll seems to be part of compiz now, maybe the check is no longer needed?
Although this is a warning, `make` (and `make install`) are of no use after that.
Thanks
For ubuntu 11.10 I had to use:
sudo apt-get install compiz-plugins-main-dev
It’s building fine, will test it tomorrow
Product looks great. Will this work on any form of linux? We are running opensuse 12.1 with 2 nvidia gtx580 cards.
With my configuration i want to have one pc run 4 video outputs. Will this still be possible using your software?
I’m trying to get this working on a clean install of 12.0 but in calling “cmake ..” for the plugin, it complains “– package ‘compiz-mousepoll’ not found” even thought it’s installed and set to a value of 1. Any idea if this is something to do with 12.04, my setup or lighttwist?
Solved, turns out I was missing the compiz-plugins-main-dev package
I managed to get this working, however the blended screen appears very pixelated (I’m guessing maybe 256 horizontal pixels). Is this a limitation given the LUT is only 8-bits in the PNG, or is something else happening that’s capping the resolution?