203 lines
12 KiB
TeX
203 lines
12 KiB
TeX
\section{Experiments}
|
|
|
|
% introduction
|
|
Evaluation took place within all floors (0 to 3) of the
|
|
faculty building, each of which about \SI{77}{\meter} x \SI{55}{\meter} in size.
|
|
%
|
|
We conducted 4 distinct walks, for testing short distances, long distances, critical sections
|
|
and ignoring the shortest-path suggested by the system.
|
|
Due to an inhouse exhibition during that time, many places were crowded and \docWIFI{} signals
|
|
are attenuated more than usual.
|
|
Each acquired path is backed by ground truth information to enable error calculation.
|
|
This ground truth is measured by recording a timestamp at a marked spot on the walking route.
|
|
During the walk, the pedestrian had to click a button on the smartphone application
|
|
when passing a marker. Between two consecutive points, a constant movement speed is assumed.
|
|
Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough to conduct
|
|
error measurements. All walks were conducted using a Google Nexus 6 and a Samsung Galaxy S5.
|
|
|
|
As the Samsung Galaxy S5's \docWIFI{} can not be limited to the \SI{2.4}{\giga\hertz} band only,
|
|
its scans take much longer than those of the Google Nexus 6:
|
|
\SI{3500}{\milli\second} vs. \SI{600}{\milli\second}.
|
|
Also, the Nexus' barometer sensor provides readings both more frequent and far more accurate than
|
|
the Galaxy does. This results in a much better localisation of the Nexus smartphone.
|
|
|
|
Despite being fast enough to run in realtime on the smartphone itself, computation was done offline using
|
|
the condensation algorithm with \SI{7500}{} particles as realization of the recursive density estimation \cite{todo}.
|
|
The weighted arithmetic mean of the particles was used as state estimation.
|
|
|
|
As mentioned earlier, the position of all \docAP{}s (about 5 per floor) is known beforhand.
|
|
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.
|
|
Additionally we used three \docIBeacon{}s for slight enhancements in some areas.
|
|
The empirically chosen values for \docWIFI{} were $P_{0_{\text{wifi}}} = \SI{-46}{\dBm}, \mPLE_{\text{wifi}} = \SI{2.7}{}$,
|
|
and $\mPLE_{\text{ib}} = \SI{1.5}{}$ for the \docIBeacon{}s, respectively.
|
|
%
|
|
Due to omitting a time-consuming calibration process for those values we expect the localistation
|
|
process to perform generally worse compared to fingerpring methods \todo{cite}. However,
|
|
incorporating prior knowledge will often compensate for those poorly chosen system parameters.
|
|
|
|
As uncertainties we used $\sigma_\text{wifi} = \sigma_\text{ib} = 8.0$, both growing with each measurement's age.
|
|
While the pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters
|
|
are determined automatically (see \ref{sec:sensBaro}). The step size for the transition was configured to be \SI{70}{\centimeter}
|
|
with an allowed derivation of \SI{10}{\percent}. The heading deviation in \refeq{eq:transSimple} was \SI{25}{\degree}.
|
|
|
|
As we start with a uniformation distribution for $\mStateVec_0$ (random position and heading), the first few estimations
|
|
are omitted from the error calculation to allow the system to somewhat settle its initial state. Even though, the error
|
|
during the follwing few seconds is expected to be much higher than the error when starting with a well known initial
|
|
position and heading.
|
|
|
|
The follwing evaluations will depict the improvements prior path knowledge is able to provide
|
|
even when other system parameters are badly chosen.
|
|
Just adding importance-factors described in \ref{sec:wallAvoidance} and \ref{sec:doorDetection}
|
|
to the simple transition \refeq{eq:transSimple} addresses only minor local errors
|
|
% like not sticking too close to walls. In most cases this lead only to slight improvements
|
|
and is therefore not further evaluated.
|
|
To examine the contribution our approach is able to provided, we will have a closer look
|
|
at a long walk with many stairs, intentionally leaving the shortest path several times,
|
|
named path 4 (see \ref{fig:paths}).
|
|
%
|
|
|
|
|
|
% all paths we evaluated
|
|
\begin{figure}
|
|
\input{gfx/eval/paths}
|
|
\caption{The four paths that were part of the evaluation.
|
|
Starting positions are marked with black circles.
|
|
For a better visualisation they were slightly shifted to avoid overlapping.}
|
|
\label{fig:paths}
|
|
\end{figure}
|
|
|
|
\commentByFrank{verlassen vom shortest path fuehrt zu weniger verbesserung, aber es wird nach wie vor besser als ohne!}
|
|
|
|
\commentByFrank{in den ersten paar sec ist die pfad-info teils hinderlich, da die genaue position noch sehr unklar ist und sich erst einstellen muss.
|
|
deshalb geht der fehler hier oft leicht hoch}
|
|
|
|
\newcommand{\refSeg}[1]{$(#1)$}
|
|
Fig. \ref{errorTimedNexus} shows the error for the individual segments of path 1 and path 4 recorded with the Google Nexus 6.
|
|
Remember that we start with a uniform distribution instead of a well known pedestrian location. Therefor the first few estimations
|
|
reside somewhere near the center of the building and result in a very high error contribution
|
|
(see fig. \ref{nexusPathDetails} \refSeg{1}).
|
|
%
|
|
Even when removing those initial estimations from the error calculation, the next few seconds are still erroneous
|
|
due to (intentionally) bad system parameters (see \ref{sec:sensors}). Furthermore, as the pedestrian is not yet walking,
|
|
our proposed method is not yet able to addres those error. This can be seen in both
|
|
fig. \ref{fig:nexusPathDetails} \refSeg{1} (the red are in the upper left)
|
|
and fig \ref{fig:errorTimedNexus} \refSeg{1}.
|
|
%
|
|
However, as soon as the pedestrian starts moving down the hallway \refSeg{2} the error is reduced dramatically.
|
|
Adding prior knowledge centers the density in the middle of the floor, ensures the heading is directed towards
|
|
the shortest path and thus produces even better localisation results.
|
|
%
|
|
Directly hereafter, we ignore the shortest path \refSeg{3'} determined by the system and walk along \refSeg{3}
|
|
instead. Of course, this leads to a temporally increasing error, as the system needs to detect this path change
|
|
and takes some time to recover (see \ref{fig:errorDistNexus} \refSeg{3}). The new path to the desired destination
|
|
is \refSeg{3''} which is also ignored. Instead, we took a much longer route down the stairwell \refSeg{4}.
|
|
After this change is detected by the system, prior knowledge is able to reduce the error for segment \refSeg{5}.
|
|
%
|
|
Immediately hereafter follows a long, straight walk down the hallway. While the \docWIFI{} component pulls
|
|
the pedestrian into the rooms on the right side, the actual walking route was located on the left side
|
|
of the wall (see ground truth in fig. \ref{fig:nexusPathDetails} \refSeg{6}). While prior knowledge prevents
|
|
the density being draged into the office-rooms, the estimated path is still located on the wrong side
|
|
of the hallway. As both sides of the floor result in a route with almost the same length,
|
|
just knowing the pedestrian's destination is not able to provide further improvments.
|
|
Thus, a constant error of approximately the floor's width remains (see \ref{fig:nexusPathDetails} \refSeg{6}).
|
|
%
|
|
Due to the excellent barometer installed within the Nexus 6, the stair provides were small estimation
|
|
errors \refSeg{7}. Hereafter follows a critical area with high errors and multimodalities. Due to an
|
|
inhouse exhibition during the time of recording, we had to leave the ground truth by a few meters.
|
|
Furthermore, the overcrowded areas lead to attenuated \docWIFI{} signals. Both reasons lead to the
|
|
density being dragged into another stairwell (see \ref{fig:nexusPathDetails}, red lines in the lower right).
|
|
The resulting multimodality (two staircases possible at the same time) leads to a rising error
|
|
\refSeg{8}, \refSeg{9}. At the end of the walk \refSeg{10} the system is able to recover, again.
|
|
|
|
|
|
% error development over time while walking along a path
|
|
\begin{figure}
|
|
\input{gfx/eval/error_timed_nexus}
|
|
\caption{Development of the error while walking along
|
|
%path 1 (upper) and
|
|
path 4 (lower) using the Google Nexus 6.
|
|
Path 4 shows increasing errors for our methods when leaving the shortest path (3) and when facing multimodalities between two
|
|
staircases just before the destination (9).}
|
|
\label{fig:errorTimedNexus}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\input{gfx/eval/path_nexus_detail}
|
|
\caption{Detailed path analysis depicting the individual segments of path 4. Their corresponding error contribution can
|
|
be seen in fig. \ref{fig:errorTimedNexus}. Even though the shortest path suggested by the system is ignored multiple
|
|
times ($3'$ and $3''$) our approach is still able to improve the overall localisation error.}
|
|
\label{fig:nexusPathDetails}
|
|
\end{figure}
|
|
|
|
% overall error-distribution for nexus and galaxy
|
|
\begin{figure}
|
|
\input{gfx/eval/error_dist_nexus}
|
|
\caption{Error distribution for all walks conducted with the Google Nexus 6. Our proposed methods
|
|
clearly provide an enhancement for the overall localization process.}
|
|
\label{fig:errorDistNexus}
|
|
\end{figure}
|
|
%\begin{figure}
|
|
% \input{gfx/eval/error_dist_galaxy}
|
|
% \caption{Nicht so markant beim galaxy, denke aber der platz reicht eh nicht, also einfach kurz erwaehnen}
|
|
%\end{figure}
|
|
|
|
The error values for all other paths and the other smartphone are listed in table
|
|
\ref{tbl:errGalaxy} and \ref{tbl:errNexus}. As can be seen, adding prior knowledge
|
|
is able to improve the localisation for all examined situations, even when
|
|
leaving the suggested path or when facing bad/slow sensor readings.
|
|
|
|
% error values
|
|
\begin{table}
|
|
\centering
|
|
\begin{tabular}{|l|c|c|c|c|}
|
|
\hline
|
|
& Path1 & Path2 & Path3 & Path4 \\\hline
|
|
Simple (\refeq{eq:transSimple}) & \SI{6.68}{\meter} & \SI{5.25}{\meter} & \SI{4.32}{\meter} & \SI{3.84}{\meter} \\\hline
|
|
Shortest (\refeq{eq:transShortestPath}) & \SI{2.72}{\meter} & \SI{2.98}{\meter} & \SI{2.48}{\meter} & \SI{3.06}{\meter} \\\hline
|
|
Multipath (\refeq{eq:transMultiPath}) & \SI{2.62}{\meter} & \SI{2.14}{\meter} & \SI{2.46}{\meter} & \SI{2.75}{\meter} \\\hline
|
|
\end{tabular}
|
|
\label{tbl:errNexus}
|
|
\caption{Median error for walks conducted with the Nexus 6.}
|
|
\end{table}
|
|
|
|
\begin{table}
|
|
\centering
|
|
\begin{tabular}{|l|c|c|c|c|}
|
|
\hline
|
|
& Path1 & Path2 & Path3 & Path4 \\\hline
|
|
Simple (\refeq{eq:transSimple}) & \SI{10.03}{\meter} & \SI{7.65}{\meter} & \SI{6.03}{\meter} & \SI{7.54}{\meter} \\\hline
|
|
Shortest (\refeq{eq:transShortestPath}) & \SI{ 5.86}{\meter} & \SI{4.14}{\meter} & \SI{5.14}{\meter} & \SI{5.20}{\meter} \\\hline
|
|
Multipath (\refeq{eq:transMultiPath}) & \SI{ 6.35}{\meter} & \SI{4.21}{\meter} & \SI{5.03}{\meter} & \SI{6.79}{\meter} \\\hline
|
|
\end{tabular}
|
|
\label{tbl:errGalaxy}
|
|
\caption{Median error for walks conducted with the Galaxy S5.}
|
|
\end{table}
|
|
|
|
\begin{figure}
|
|
\includegraphics{gfx/eval/bergwerk_path1_galaxy}
|
|
\caption{Path 1 recorded with the Galaxy S5. Using prior knowledge improves the staircase (left) and the target area (right) where
|
|
both the barometer and \docWIFI{} provided bad readings.}
|
|
\label{fig:bergwerkPath1Galaxy}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\includegraphics{gfx/eval/bergwerk_path3_galaxy}
|
|
\caption{Path 3 recorded with the Galaxy S5. Even though both paths look similar, the version with prior knowledge ended
|
|
much closer to the real destination due to reduced delays.}
|
|
\label{fig:bergwerkPath3Galaxy}
|
|
\end{figure}
|
|
|
|
|
|
|
|
\begin{itemize}
|
|
\item Nochmal kurz auf die Probleme des letzten Systems eingehen (schon teil der introduction)
|
|
\item Da letztes mal nur 1 Pfad, machen wir dieses mal mehrere!
|
|
\item Stelle normale Lokalisation der Pfad Lokalisation gegenüber und überlege wo Probleme auftreten
|
|
\item nutze den "natürlichen Pfad" und einen normalen dijkstra
|
|
\item Analysiere Probleme ggf. mit schönen Grafiken.
|
|
\item Vergleich zum Schluss das neue System mit dem Alten um eine schöne Conclusion der Verbesserungen einzuleiten.
|
|
\end{itemize}
|
|
\commentByFrank{sensorausfall simulieren, z.b. in der mitte, oder auf einer treppe}
|
|
\commentByFrank{zwischendrin mal stehenbleiben und schauen ob auch das klappt}
|
|
\commentByFrank{zu grosser einfluss vom pfad ist also kein allheilmittel.. kann, wie beim treppenhaus, auch nach hinten los gehen}
|