1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
|
\section{Problems}
The number of test cases to implement was the first small problem the group had encountered. However, it was not the only one. The main problem was due to the chosen input device; the Wiimote\footnote[1]{Wiimote is a nickname for the Wii remote which is the primary controller (6DOF) used with the Nintendo Wii.}. As the Wiimote supports 6DOF only 3DOF were used in the project. This was enough to be able to place a block inside the correct hole.
%\subsubsection{System latency}
%In order to support a Wiimote in a program like MatchBlox, a number of development libraries are available on the web. The library that was initially used for the application suffered a lot of lag (or latency). It took some time to determine the error and switch to another library.
\subsubsection{Depleted batteries}
A second Wiimote is used to track the head using the infrared camera in front of the Wiimote. The user places two infrared LED's which the Wiimote can keep track of. A wireless Wii sensorbar\footnote[1]{The Wiimote senses IR light from the sensor bar. The light emitted from each end of the sensor bar is focused onto the image sensor which sees the light as two bright dots separated by a distance "mi" on the image sensor. The second distance "m" between the two clusters of light emitters in the sensor bar is a fixed distance. From these two distances m and mi, the distance between the Wiimote and the sensor bar can be calculated.} was used for this. This has as great advantages that it's wireless (as opposed the the wired variant shipped with the Wii console) and no home made infrared LED's need to be used. \\
A great disadvantage however, is that it need batteries and it's not visible when the batteries are almost depleted. When the batteries are running low, it is harder for the Wiimote to detect the IR dots. With the use of a camera from a mobile phone, the infrared LED's can be seen. This is a good way to check if the sensor bar is still emitting IR light.
\subsubsection{Cross talk}
Cross talk is a common problem in all implementations of stereo vision. It occurs when left images reach the right eye and right images reach the left eye. In other words, you see things with an eye you're only supposed to see with the other eye. In the case of red and blue stereo vision it means that the colors are not properly filtered out. \\
There exist a technique that reduces this effect. What it basically does is subtract the leakage from the displayed intensity. This technique was not applied to the MatchBlox application. Instead, a more detailed and moving background is used. This makes the crosstalk less visible and overall very useable.
\subsubsection{Stability of the 3D mouse cursor}
During the implementation of the 3D mouse control scheme described in section 3.1, we ran into some problems, or actually one problem with multiple causes: the cursor was impossible to keep steady when holding the wii remote in hand. The cursor seemed to oscillate around a point, while frequently jumping to another position. Only when the controller lay perfectly still on a flat surface, the cursor on screen seemed to be steady.
One cause for the cursor instability was the method with which we selected the coordinates for $ g_l $ and $ g_r $ from the points returned by the wiimote. As mentioned the wiimote can track up to four infra-red sources. If the wiimote is close enough to the sensor bar it will recognize the positions of the individual LED's withing the LED groups. When this happens more than two coordinates are returned by the wiimote. In our original implementation we selected the two points with the greatest distance between them. As a solution, we implemented an algorithm that sorts the points returned by the wiimote into two groups based on their proximities, one group per LED group. The algorithm returns the average coordinates per group as the coordinates for $ g_l $ and $ g_r $. This solved the large jumps in the cursor position, but the did not help the stability of the cursor very much.
Another cause, which is more apparent, is the fact that it is very hard for humans to keep their arm perfectly still when holding a wiimote. As a solution we implemented an exponential smoothing algorithm to calculate a weighted average over $ g_l $ and $ g_r $ before they were processed. The smoothing stabilized the cursor in the x and y-direction while still being quite responsive. In the z-direction however the cursor remained unsteady, albeit to some lesser extend.
The reason for the stability problem in the z-direction is the limited resolution of the mapping in the z-direction as opposed to the x and y directions. For the latter two, the resolution corresponds to the resolution of the camera. For the z-direction the resolution is limited by the chosen values of $ d_{min} $ and $ d_{max} $ and the distance between the LED groups in the sensor bar. Suppose we define $ d_{min} $ as 1 meter and $ d_{max} $ as 2 meters and use a standard sensor bar such that $ \Delta $ in the distance calculation is 0.0205 meters (see Figure \ref{fig:wiimote_dist_calc}). The distance in pixels between the $ g_l $ and $ g_r $ at distance $ d_{min} $ is:
\[ \frac{2\mathsf{arctan}(\frac{\frac{1}{2}\Delta}{d_{max}})}{\theta_{pix}} = 29.832867646 \approx 30 \text{pixels} \]
Compared to the distance in pixels at $ d_{max} $ which is approximately $ 15 $ pixels, gives a resolution in the z-direction of a mere $ 15 $ camera pixels. The resolution can be increased by using a wider sensor bar or by sitting closer to the sensor bar, reducing $ d_{min} $ and $ d_{max} $. Choosing a bounding box $ \mathcal{B} $ with a depth as small as possible can also make vibrations in the z-direction less visible. In our implementation we severely smoothed the z-coordinate of the cursor position to finally stabilize the cursor. The downside of the smoothing is that we are left with a noticeable lag in movements in the z-direction.
%\subsection{Evaluation}
%The conclusions of the MatchBlox project are explained in chapter 4 of this report. This paragraph deals with the evaluation of the project and how it was executed. \\
%A lot of new techniques during the implementation were applied; head tracking, stereo vision, a Wiimote as input device, etc. To fit this all into one big project was a bit over the top and therefor unfeasible. The troubles with the infra red LED's and the Wiimote in general consumed a lot of time and energy of the project. \\
%Although the project was able to complete the application, it's not useable for an actual experiment. This doesn't mean all was in vein and no conclusion can be drawn from the results of the project. The conclusions will be explained in the last chapter.
\section{Conclusions}
In this section we'll give our conclusions of the project, although
we weren't enable to let persons experiment with our application, we
can drawn some conclusions of the project.
\subsection{Head Tracking}
With the OpenGL library is't not difficult to make a good perspective
world. It can be done by mainly one function. The difficulty for the
head tracking was to cooperate good with the Nintendo Wii controller
and its sensor bar, which is being discussed in the problem section.
A disadvantage of using Wii controller is, the small angle of the
camera. To have a good moving space in which the camera can track the
dots, you must be on a long distance of the camera, and so the
screen. And for that you must have a big screen to have a nice view.
If you are to close to the camera, you easily move out of the range
and your head cannot be tracked anymore.
\subsection{Stereo Vision}
The applied red-blue stereo vision is one of the least realistic alternatives of stereo vision. Yet, it is the easiest to implement and still very effective. The created images provide the user the illusion of depth in a very usable way. The distance between objects can easily been seen from a still image. This is also a big disadvantage of head tracking; without moving, the image provides no additional "depth info".
\subsection{Wii remote as 3D mouse}
We can conclude that the Wii remote is not fit to be used as a 3D mouse. The ridiculously low resolution in depth measurements when using standard Nintendo hardware makes the wiimote unusable as a 3D mouse. However, because the resolution of the camera in the wiimote is relatively high, compared to the resolution of a standard definition television set, the wiimote is ideal to be used on the Wii console as a 2D mouse. The third dimension should only be used for tasks that do not require great precision or resolution, for example simulating the action of pushing a button or for switching between zoomed and normal view in a 3D shooting game.
|
