212 lines
13 KiB
TeX
212 lines
13 KiB
TeX
\section{Experiments}
|
|
|
|
\newcommand{\refSeg}[1]{seg. $#1$}
|
|
|
|
% introduction
|
|
Evaluation took place within all floors (0 to 3) of the
|
|
faculty building, each of which about \SI{77}{\meter} x \SI{55}{\meter} in size.
|
|
%
|
|
We conducted 4 distinct walks, for testing short distances, long distances, critical sections
|
|
and ignoring the shortest-path suggested by the system.
|
|
Due to an in-house exhibition during that time, many places were crowded and \docWIFI{} signals
|
|
are attenuated more than usual.
|
|
Each acquired path is backed by ground truth information to enable error calculation.
|
|
This ground truth is measured by recording a timestamp at a marked spot on the walking route.
|
|
During the walk, the pedestrian had to click a button on the smartphone application
|
|
when passing a marker. Between two consecutive points, a constant movement speed is assumed.
|
|
Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough to conduct
|
|
error measurements. All walks were performed using a Motorola Nexus 6 and a Samsung Galaxy S5.
|
|
|
|
As the Samsung Galaxy S5's \docWIFI{} can not be limited to the \SI{2.4}{\giga\hertz} band only,
|
|
its scans take much longer than those of the Motorola Nexus 6:
|
|
\SI{3500}{\milli\second} vs. \SI{600}{\milli\second}.
|
|
Also, the Nexus' barometer sensor provides readings both more frequent and far more accurate than
|
|
the Galaxy does. This results in a better localisation using the Nexus smartphone.
|
|
|
|
Despite being fast enough to run in realtime on the smartphone itself, computation was done offline using
|
|
the \mbox{CONDENSATION} particle filter with \SI{7500}{} particles as realization.
|
|
The weighted arithmetic mean of the particles was used as state estimation.
|
|
|
|
As mentioned earlier, the position of all \docAP{}s (about 5 per floor) is known beforehand.
|
|
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.
|
|
Additionally, we used three \docIBeacon{}s for slight enhancements in some areas.
|
|
The empirically chosen values for \docWIFI{} were
|
|
$P_{0_{\text{wifi}}} = \SI{-46}{\dBm}, \mPLE_{\text{wifi}} = \SI{2.7}{}, \mWAF_{\text{wifi}} = \SI{8}{\dB}$,
|
|
and $\mPLE_{\text{ib}} = \SI{1.5}{}$ for the \docIBeacon{}s, respectively.
|
|
%
|
|
Due to omitting a time-consuming calibration process for those values we expect the localisation process to
|
|
perform generally worse compared to standard fingerprinting methods \cite{Ville09}. However, incorporating
|
|
prior knowledge will often compensate for those poorly chosen system parameters.
|
|
|
|
As uncertainties we used $\sigma_\text{wifi} = \sigma_\text{ib} = 8.0$, both growing with each measurement's age.
|
|
While the pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters
|
|
are determined automatically (see \ref{sec:sensBaro}). The step size $\mStepSize$ for the transition was configured to be
|
|
\SI{70}{\centimeter} with an allowed derivation of \SI{10}{\percent}. The heading deviation in
|
|
\refeq{eq:transSimple}, \refeq{eq:transShortestPath} and \refeq{eq:transMultiPath} was \SI{25}{\degree}.
|
|
Edges departing from the pedestrian's destination are downvoted using $\mUsePath = 0.9$.
|
|
\commentByFrank{$\mUsePath$ erklaert}
|
|
|
|
|
|
As we start with a discrete uniform distribution for $\mStateVec_0$ (random position and heading), the first few estimations
|
|
are omitted from the error calculation to allow the system to somewhat settle its initial state. Even though, the error
|
|
during the following few seconds is expected to be much higher than the error when starting with a well known initial
|
|
position and heading.
|
|
%
|
|
|
|
|
|
The following evaluations will depict the improvements that the prior path knowledge is able to provide,
|
|
even when other system parameters are badly chosen.
|
|
Just adding importance-factors (described in \ref{sec:wallAvoidance} and \ref{sec:doorDetection})
|
|
to the simple transition \refeq{eq:transSimple} addresses only minor local errors
|
|
% like not sticking too close to walls. In most cases this lead only to slight improvements
|
|
and is therefore not further evaluated.
|
|
To examine the contribution our approach is able to provide, we will have a closer look
|
|
at a long walk with many stairs, intentionally leaving the shortest path several times,
|
|
named path 4 (see fig. \ref{fig:paths}).
|
|
%
|
|
% all paths we evaluated
|
|
\begin{figure}
|
|
\input{gfx/eval/paths}
|
|
\caption{The four paths that were part of the evaluation.
|
|
Starting positions are marked with black circles.
|
|
For a better visualisation they were slightly shifted to avoid overlapping.
|
|
\commentByFrank{font war korrekt, aber die groesse war zu gross im vgl. zu den anderen}
|
|
}
|
|
\label{fig:paths}
|
|
\end{figure}
|
|
% error development over time while walking along a path
|
|
\begin{figure}
|
|
\input{gfx/eval/error_timed_nexus}
|
|
\caption{Error development while walking along Path 4 using the Motorola Nexus 6.
|
|
When leaving the suggested route \refSeg{3}, the error of \textbf{shortest} path \refeq{eq:transShortestPath}
|
|
and \textbf{multi}path \refeq{eq:transMultiPath} increases.
|
|
The same issues arise when facing multimodalities between two staircases just before the destination \refSeg{9}.
|
|
\commentByFrank{hilft das bold vlt. schon um die legende zu verstehen?}
|
|
}
|
|
\label{fig:errorTimedNexus}
|
|
\end{figure}
|
|
% detailed analysis of path 4
|
|
\begin{figure}
|
|
\input{gfx/eval/path_nexus_detail}
|
|
\caption{Detailed path analysis depicting the individual segments of path 4 using \refeq{eq:transMultiPath}.
|
|
For a better visualisation, the segments are divided using an outline of alternating grey levels.
|
|
The corresponding segment-error can be seen in fig. \ref{fig:errorTimedNexus}. Even though the shortest path
|
|
suggested by the system is ignored multiple times (\refSeg{3'} and \refSeg{3''}) our approach is still able to improve
|
|
the overall localisation error.}
|
|
\label{fig:nexusPathDetails}
|
|
\end{figure}
|
|
%
|
|
|
|
|
|
Fig. \ref{fig:errorTimedNexus} depicts the error for path 4 recorded with the Motorola Nexus 6.
|
|
For a better understanding of the following discussion, the path was divided into $10$ individual segments.
|
|
Remember that we start with a uniform distribution instead of a well known pedestrian location.
|
|
Therefore, the first few estimations
|
|
reside somewhere near the centre of the building and result in a very high error contribution
|
|
as illustrated in fig. \ref{fig:nexusPathDetails} \refSeg{1}.
|
|
%
|
|
Even when removing those initial estimations from the error calculation, the next few seconds are still erroneous
|
|
due to (intentionally) bad system parameters introduced in section \ref{sec:sensors}.
|
|
Furthermore, as the pedestrian is not yet walking, our proposed method is also not yet able to address those errors.
|
|
This can be seen at the red area in the upper left corner of fig. \ref{fig:nexusPathDetails} \refSeg{1} and within
|
|
segment \refSeg{1} of fig. \ref{fig:errorTimedNexus}.
|
|
%
|
|
Starting with both, known position and heading, reduced the error by about \SI{15}{\percent} when using prior knowledge and
|
|
by \SI{25}{\percent} when omitting prior knowledge. As prior knowledge directs the density towards a known target,
|
|
it is able to compensate unknown initial headings which explains the \SI{10}{\percent} difference.
|
|
\commentByFrank{bekannter startpunkt getestet und kurz beschrieben}
|
|
%
|
|
However, as soon as the pedestrian starts moving down the hallway \refSeg{2} the error is reduced dramatically.
|
|
Adding prior knowledge centres the density in the middle of the floor, ensures that the heading is directed towards
|
|
the shortest path and thus produces even better localisation results.
|
|
%
|
|
Directly hereafter, we ignore the shortest path \refSeg{3'} determined by the system and walk along \refSeg{3}
|
|
instead. Of course, this leads to a temporally increasing error, as the system needs to detect this path change
|
|
and takes some time to recover (see fig. \ref{fig:errorTimedNexus} \refSeg{3}). The new path to the desired destination
|
|
is \refSeg{3''} which is also ignored. Instead, we took a much longer route down the stairwell \refSeg{4}.
|
|
After this change is detected by the system, prior knowledge is again able to reduce the error for segment \refSeg{5}.
|
|
%
|
|
Immediately hereafter follows a long, straight walk down the hallway. While the \docWIFI{} component pulls
|
|
the pedestrian into the rooms on the right side, the actual walking route was located on the left side
|
|
of the floor (see ground truth in fig. \ref{fig:nexusPathDetails} \refSeg{6}). While prior knowledge prevents
|
|
the density from being dragged into the office-rooms, the estimated path is still located on the wrong side
|
|
of the hallway. As both sides of the floor result in a route with almost the same length,
|
|
just knowing the pedestrian's destination is not able to provide further improvements.
|
|
Thus, a constant error of approximately the floor's width remains.
|
|
This is clearly visible in fig. \ref{fig:nexusPathDetails} \refSeg{6}.
|
|
%
|
|
Due to the excellent barometer installed within the Nexus 6, changing the floor provides only small estimation
|
|
errors in segment \refSeg{7}.
|
|
It follows a critical area with high errors and multimodalities.
|
|
Due to an in-house exhibition during the time of recording, we had to leave the ground truth by a few meters.
|
|
Furthermore, the overcrowded areas lead to attenuated \docWIFI{} signals. Both reasons move the
|
|
density into another stairwell (see fig. \ref{fig:nexusPathDetails}, red lines in the lower right).
|
|
The resulting multimodality (two staircases possible at the same time) leads to a rising error
|
|
\refSeg{8}, \refSeg{9}. At the end of the walk \refSeg{10} the system is able to recover, again.
|
|
|
|
|
|
% overall error-distribution for nexus and galaxy
|
|
\begin{figure}
|
|
\input{gfx/eval/error_dist_nexus}
|
|
\caption{Error distribution of all walks conducted with the Motorola Nexus 6 for distinct percentile values.
|
|
Our proposed methods clearly provide an enhancement for the overall localization process.
|
|
\commentByFrank{percentile erwaehnt}}
|
|
\label{fig:errorDistNexus}
|
|
\end{figure}
|
|
%\begin{figure}
|
|
% \input{gfx/eval/error_dist_galaxy}
|
|
% \caption{Nicht so markant beim galaxy, denke aber der platz reicht eh nicht, also einfach kurz erwaehnen}
|
|
%\end{figure}
|
|
|
|
The median error values for all other paths and the other smartphone are listed in table
|
|
\ref{tbl:errNexus} and \ref{tbl:errGalaxy}. Furthermore, fig. \ref{fig:errorDistNexus}
|
|
depicts the error development for several percentile values. As can be seen, adding prior
|
|
knowledge is able to improve the localisation for all examined situations, even when
|
|
leaving the suggested path or when facing bad/slow sensor readings.
|
|
\commentByFrank{fig. \ref{fig:errorDistNexus} erwaehnt}
|
|
|
|
% error values
|
|
\begin{table}
|
|
\caption{Median error for walks conducted with the Nexus 6.}
|
|
\label{tbl:errNexus}
|
|
\centering
|
|
\begin{tabular}{|l|c|c|c|c|}
|
|
\hline
|
|
& Path1 & Path2 & Path3 & Path4 \\\hline
|
|
Simple (\refeq{eq:transSimple}) & \SI{6.68}{\meter} & \SI{5.25}{\meter} & \SI{4.32}{\meter} & \SI{3.84}{\meter} \\\hline
|
|
Shortest (\refeq{eq:transShortestPath}) & \SI{2.72}{\meter} & \SI{2.98}{\meter} & \SI{2.48}{\meter} & \SI{3.06}{\meter} \\\hline
|
|
Multipath (\refeq{eq:transMultiPath}) & \SI{2.62}{\meter} & \SI{2.14}{\meter} & \SI{2.46}{\meter} & \SI{2.75}{\meter} \\\hline
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
\begin{table}
|
|
\caption{Median error for walks conducted with the Galaxy S5.}
|
|
\label{tbl:errGalaxy}
|
|
\centering
|
|
\begin{tabular}{|l|c|c|c|c|}
|
|
\hline
|
|
& Path1 & Path2 & Path3 & Path4 \\\hline
|
|
Simple (\refeq{eq:transSimple}) & \SI{10.03}{\meter} & \SI{7.65}{\meter} & \SI{6.03}{\meter} & \SI{7.54}{\meter} \\\hline
|
|
Shortest (\refeq{eq:transShortestPath}) & \SI{ 5.86}{\meter} & \SI{4.14}{\meter} & \SI{5.14}{\meter} & \SI{5.20}{\meter} \\\hline
|
|
Multipath (\refeq{eq:transMultiPath}) & \SI{ 6.35}{\meter} & \SI{4.21}{\meter} & \SI{5.03}{\meter} & \SI{6.79}{\meter} \\\hline
|
|
\end{tabular}
|
|
\end{table}
|
|
|
|
%\begin{figure}
|
|
% \includegraphics{gfx/eval/bergwerk_path1_galaxy}
|
|
% \caption{Path 1 recorded with the Galaxy S5. Using prior knowledge improves the staircase (left) and the target area (right) where
|
|
% both the barometer and \docWIFI{} provided bad readings.}
|
|
% \label{fig:bergwerkPath1Galaxy}
|
|
%\end{figure}
|
|
%
|
|
%\begin{figure}
|
|
% \includegraphics{gfx/eval/bergwerk_path3_galaxy}
|
|
% \caption{Path 3 recorded with the Galaxy S5. Even though both paths look similar, the version with prior knowledge ended
|
|
% much closer to the real destination due to reduced delays.}
|
|
% \label{fig:bergwerkPath3Galaxy}
|
|
%\end{figure}
|
|
|
|
%\commentByFrank{sensorausfall simulieren, z.b. in der mitte, oder auf einer treppe}
|
|
%\commentByFrank{zwischendrin mal stehenbleiben und schauen ob auch das klappt}
|
|
%\commentByFrank{zu grosser einfluss vom pfad ist also kein allheilmittel.. kann, wie beim treppenhaus, auch nach hinten los gehen}
|