163 lines
8.7 KiB
TeX
163 lines
8.7 KiB
TeX
\section{Experiments}
|
|
|
|
% introduction
|
|
Evaluation took place within all floors (0 to 3) of the
|
|
faculty building, each of which about \SI{77}{\meter} x \SI{55}{\meter} in size.
|
|
%
|
|
We conducted 4 distinct walks, for testing short distances, long distances, critical sections
|
|
and ignoring the shortest-path suggested by the system.
|
|
Due to an inhouse exhibition during that time, many places were crowded and \docWIFI{} signals
|
|
are attenuated more than usual.
|
|
Each acquired path is backed by ground truth information to enable error calculation.
|
|
This ground truth is measured by recording a timestamp at a marked spot on the walking route.
|
|
During the walk, the pedestrian had to click a button on the smartphone application
|
|
when passing a marker. Between two consecutive points, a constant movement speed is assumed.
|
|
Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough to conduct
|
|
error measurements. All walks were conducted using a Google Nexus 6 and a Samsung Galaxy S5.
|
|
|
|
As the Samsung Galaxy S5's \docWIFI{} can not be limited to the \SI{2.4}{\giga\hertz} band only,
|
|
its scans take much longer than those of the Google Nexus 6:
|
|
\SI{3500}{\milli\second} vs. \SI{600}{\milli\second}.
|
|
Also, the Nexus' barometer sensor provides readings both more frequent and far more accurate than
|
|
the Galaxy does. This results in a much better localisation of the Nexus smartphone.
|
|
|
|
Despite being fast enough to run in realtime on the smartphone itself, computation was done offline using
|
|
the condensation algorithm with \SI{7500}{} particles as realization of the recursive density estimation \cite{todo}.
|
|
The weighted arithmetic mean of the particles was used as state estimation.
|
|
|
|
As mentioned earlier, the position of all \docAP{}s (about 5 per floor) is known beforhand.
|
|
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.
|
|
Additionally we used three \docIBeacon{}s for slight enhancements in some areas.
|
|
The empirically chosen values for \docWIFI{} were $P_{0_{\text{wifi}}} = \SI{-46}{\dBm}, \mPLE_{\text{wifi}} = \SI{2.7}{}$,
|
|
and $\mPLE_{\text{ib}} = \SI{1.5}{}$ for the \docIBeacon{}s, respectively.
|
|
%
|
|
Due to omitting a time-consuming calibration process for those values we expect the localistation
|
|
process to perform generally worse compared to fingerpring methods \todo{cite}. However,
|
|
incorporating prior knowledge will often compensate for those poorly chosen system parameters.
|
|
|
|
As uncertainties we used $\sigma_\text{wifi} = \sigma_\text{ib} = 8.0$, both growing with each measurement's age.
|
|
While the pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters
|
|
are determined automatically (see \ref{sec:sensBaro}). The step size for the transition was configured to be \SI{70}{\centimeter}
|
|
with an allowed derivation of \SI{10}{\percent}. The heading deviation in \refeq{eq:transSimple} was \SI{25}{\degree}.
|
|
|
|
As we start with a uniformation distribution for $\mStateVec_0$ (random position and heading), the first few estimations
|
|
are omitted from the error calculation to allow the system to somewhat settle its initial state. Even though, the error
|
|
during the follwing few seconds is expected to be much higher than the error when starting with a well known initial
|
|
position and heading.
|
|
|
|
The follwing evaluations will depict the improvements prior path knowledge is able to provide
|
|
even when other system parameters are badly chosen.
|
|
Just adding importance-factors described in \ref{sec:wallAvoidance} and \ref{sec:doorDetection}
|
|
to the simple transition \refeq{eq:transSimple} addresses only minor local errors
|
|
% like not sticking too close to walls. In most cases this lead only to slight improvements
|
|
and is therefore not further evaluated.
|
|
%
|
|
|
|
|
|
\commentByFrank{bergwerk\_path3\_galaxy}
|
|
|
|
\begin{figure}
|
|
\input{gfx/eval/paths}
|
|
\caption{The four paths that were part of the evaluation.
|
|
Starting positions are marked with black circles.
|
|
For a better visualisation they were slightly shifted to avoid overlapping.}
|
|
\label{fig:paths}
|
|
\end{figure}
|
|
|
|
\commentByFrank{verlassen vom shortest path fuehrt zu weniger verbesserung, aber es wird nach wie vor besser als ohne!}
|
|
|
|
\commentByFrank{in den ersten paar sec ist die pfad-info teils hinderlich, da die genaue position noch sehr unklar ist und sich erst einstellen muss.
|
|
deshalb geht der fehler hier oft leicht hoch}
|
|
|
|
% error development over time while walking along a path
|
|
\begin{figure}
|
|
\input{gfx/eval/error_timed_nexus}
|
|
\caption{Development of the error while walking along path 1 (upper) and path 4 (lower) using the Google Nexus 6.
|
|
Path 4 shows increasing errors for our methods when leaving the shortest path (3) and when facing multimodalities between two
|
|
staircases just before the destination (9).}
|
|
\label{fig:errorTimedNexus}
|
|
\end{figure}
|
|
|
|
% overall error-distribution for nexus and galaxy
|
|
\begin{figure}
|
|
\input{gfx/eval/error_dist_nexus}
|
|
\caption{Error distribution for all walks conducted with the Google Nexus 6. Our proposed methods
|
|
clearly provide an enhancement for the overall localization process.}
|
|
\label{fig:errorDistNexus}
|
|
\end{figure}
|
|
\begin{figure}
|
|
\input{gfx/eval/error_dist_galaxy}
|
|
\caption{Nicht so markant beim galaxy, denke aber der platz reicht eh nicht, also einfach kurz erwaehnen}
|
|
\end{figure}
|
|
|
|
\begin{table}
|
|
\centering
|
|
\begin{tabular}{|l|c|c|c|c|}
|
|
\hline
|
|
& Path1 & Path2 & Path3 & Path4 \\\hline
|
|
Simple (\refeq{eq:transSimple}) & \SI{6.68}{\meter} & \SI{5.25}{\meter} & \SI{4.32}{\meter} & \SI{3.84}{\meter} \\\hline
|
|
Shortest (\refeq{eq:transShortestPath}) & \SI{2.72}{\meter} & \SI{2.98}{\meter} & \SI{2.48}{\meter} & \SI{3.06}{\meter} \\\hline
|
|
Multipath (\refeq{eq:transMultiPath}) & \SI{2.62}{\meter} & \SI{2.14}{\meter} & \SI{2.46}{\meter} & \SI{2.75}{\meter} \\\hline
|
|
\end{tabular}
|
|
\caption{Median error for walks conducted with the Nexus 6.}
|
|
\end{table}
|
|
|
|
\begin{table}
|
|
\centering
|
|
\begin{tabular}{|l|c|c|c|c|}
|
|
\hline
|
|
& Path1 & Path2 & Path3 & Path4 \\\hline
|
|
Simple (\refeq{eq:transSimple}) & \SI{10.03}{\meter} & \SI{7.65}{\meter} & \SI{6.03}{\meter} & \SI{7.54}{\meter} \\\hline
|
|
Shortest (\refeq{eq:transShortestPath}) & \SI{ 5.86}{\meter} & \SI{4.14}{\meter} & \SI{5.14}{\meter} & \SI{5.20}{\meter} \\\hline
|
|
Multipath (\refeq{eq:transMultiPath}) & \SI{ 6.35}{\meter} & \SI{4.21}{\meter} & \SI{5.03}{\meter} & \SI{6.79}{\meter} \\\hline
|
|
\end{tabular}
|
|
\caption{Median error for walks conducted with the Galaxy S5.}
|
|
\end{table}
|
|
|
|
\begin{figure}
|
|
\includegraphics{gfx/eval/bergwerk_path1_galaxy}
|
|
\caption{Path 1 recorded with the Galaxy S5. Using prior knowledge improves the staircase (left) and the target area (right) where
|
|
both the barometer and \docWIFI{} provided bad readings.}
|
|
\label{fig:bergwerkPath1Galaxy}
|
|
\end{figure}
|
|
|
|
\begin{figure}
|
|
\includegraphics{gfx/eval/bergwerk_path3_galaxy}
|
|
\caption{Path 3 recorded with the Galaxy S5. Even though both paths look similar, the version with prior knowledge ended
|
|
much closer to the real destination due to reduced delays.}
|
|
\label{fig:bergwerkPath3Galaxy}
|
|
\end{figure}
|
|
|
|
\begin{itemize}
|
|
\item Nochmal kurz auf die Probleme des letzten Systems eingehen (schon teil der introduction)
|
|
\item Da letztes mal nur 1 Pfad, machen wir dieses mal mehrere!
|
|
\item Stelle normale Lokalisation der Pfad Lokalisation gegenüber und überlege wo Probleme auftreten
|
|
\item nutze den "natürlichen Pfad" und einen normalen dijkstra
|
|
\item Analysiere Probleme ggf. mit schönen Grafiken.
|
|
\item Vergleich zum Schluss das neue System mit dem Alten um eine schöne Conclusion der Verbesserungen einzuleiten.
|
|
\end{itemize}
|
|
|
|
\commentByFrank{we start with a uniform distribution $\mStateVec_0$}
|
|
|
|
\commentByFrank{hinweis auf die verschiedenen geraete (smartphones) und unterschiede, wlan/baro}
|
|
\commentByFrank{
|
|
PATH4 HAELT SICH NICHT AN DEN SHORTEST PATH.
|
|
GUTES BEISPIEL.
|
|
der pfad wechselt sogar 2x! (3. stock)
|
|
der shortest wird am ende etwas ungenau bei der treppe
|
|
}
|
|
|
|
\commentByFrank{sensorausfall simulieren, z.b. in der mitte, oder auf einer treppe}
|
|
\commentByFrank{zwischendrin mal stehenbleiben und schauen ob auch das klappt}
|
|
\commentByFrank{pfad verlassen und ganz wo anders hingehen}
|
|
|
|
\commentByFrank{die reine importance selbst auf dem graphen hilft, aber nur minimal weiter}
|
|
|
|
\commentByFrank{pfad4 nexus. pfadlos laeuft mit ach und krach richtig (treppenhaus, wlan schlecht)
|
|
mit pfad laeuft es falsch, weil die andere treppe kuerzer zum ziel ist und das wlan dort besser passt}
|
|
|
|
\commentByFrank{zu grosser einfluss vom pfad ist also kein allheilmittel.. kann, wie beim treppenhaus, auch nach hinten los gehen}
|
|
|
|
|
|
\commentByFrank{path1: bad start due to nearby AP and bad parameters (path-loss too high): high starting errors: median better}
|