FHWS/Fusion2016
Archived
3
0
Fork 0
Code Issues 1 Pull Requests Releases Wiki Activity

eval parts written and refactored

Browse Source 
completely changed the eval gfx
This commit is contained in:
FrankE
2016-02-13 11:53:43 +01:00
parent c12502cf8b
commit 601a7161e5
8 changed files with 7630 additions and 1109 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
tex/chapters/experiments.tex
View File

@@ -50,12 +50,14 @@
Just adding importance-factors described in \ref{sec:wallAvoidance} and \ref{sec:doorDetection}
to the simple transition \refeq{eq:transSimple} addresses only minor local errors
% like not sticking too close to walls. In most cases this lead only to slight improvements
and is therefore not further evaluated.
and is therefore not further evaluated.
To examine the contribution our approach is able to provided, we will have a closer look
at a long walk with many stairs, intentionally leaving the shortest path several times,
named path 4 (see \ref{fig:paths}).
%
\commentByFrank{bergwerk\_path3\_galaxy}
% all paths we evaluated
\begin{figure}
\input{gfx/eval/paths}
\caption{The four paths that were part of the evaluation.
@@ -68,16 +70,65 @@
\commentByFrank{in den ersten paar sec ist die pfad-info teils hinderlich, da die genaue position noch sehr unklar ist und sich erst einstellen muss.
deshalb geht der fehler hier oft leicht hoch}
\newcommand{\refSeg}[1]{$(#1)$}
Fig. \ref{errorTimedNexus} shows the error for the individual segments of path 1 and path 4 recorded with the Google Nexus 6.
Remember that we start with a uniform distribution instead of a well known pedestrian location. Therefor the first few estimations
reside somewhere near the center of the building and result in a very high error contribution
(see fig. \ref{nexusPathDetails} \refSeg{1}).
%
Even when removing those initial estimations from the error calculation, the next few seconds are still erroneous
due to (intentionally) bad system parameters (see \ref{sec:sensors}). Furthermore, as the pedestrian is not yet walking,
our proposed method is not yet able to addres those error. This can be seen in both
fig. \ref{fig:nexusPathDetails} \refSeg{1} (the red are in the upper left)
and fig \ref{fig:errorTimedNexus} \refSeg{1}.
%
However, as soon as the pedestrian starts moving down the hallway \refSeg{2} the error is reduced dramatically.
Adding prior knowledge centers the density in the middle of the floor, ensures the heading is directed towards
the shortest path and thus produces even better localisation results.
%
Directly hereafter, we ignore the shortest path \refSeg{3'} determined by the system and walk along \refSeg{3}
instead. Of course, this leads to a temporally increasing error, as the system needs to detect this path change
and takes some time to recover (see \ref{fig:errorDistNexus} \refSeg{3}). The new path to the desired destination
is \refSeg{3''} which is also ignored. Instead, we took a much longer route down the stairwell \refSeg{4}.
After this change is detected by the system, prior knowledge is able to reduce the error for segment \refSeg{5}.
%
Immediately hereafter follows a long, straight walk down the hallway. While the \docWIFI{} component pulls
the pedestrian into the rooms on the right side, the actual walking route was located on the left side
of the wall (see ground truth in fig. \ref{fig:nexusPathDetails} \refSeg{6}). While prior knowledge prevents
the density being draged into the office-rooms, the estimated path is still located on the wrong side
of the hallway. As both sides of the floor result in a route with almost the same length,
just knowing the pedestrian's destination is not able to provide further improvments.
Thus, a constant error of approximately the floor's width remains (see \ref{fig:nexusPathDetails} \refSeg{6}).
%
Due to the excellent barometer installed within the Nexus 6, the stair provides were small estimation
errors \refSeg{7}. Hereafter follows a critical area with high errors and multimodalities. Due to an
inhouse exhibition during the time of recording, we had to leave the ground truth by a few meters.
Furthermore, the overcrowded areas lead to attenuated \docWIFI{} signals. Both reasons lead to the
density being dragged into another stairwell (see \ref{fig:nexusPathDetails}, red lines in the lower right).
The resulting multimodality (two staircases possible at the same time) leads to a rising error
\refSeg{8}, \refSeg{9}. At the end of the walk \refSeg{10} the system is able to recover, again.
% error development over time while walking along a path
\begin{figure}
\input{gfx/eval/error_timed_nexus}
\caption{Development of the error while walking along path 1 (upper) and path 4 (lower) using the Google Nexus 6.
\caption{Development of the error while walking along
%path 1 (upper) and
path 4 (lower) using the Google Nexus 6.
Path 4 shows increasing errors for our methods when leaving the shortest path (3) and when facing multimodalities between two
staircases just before the destination (9).}
\label{fig:errorTimedNexus}
\end{figure}
\begin{figure}
\input{gfx/eval/path_nexus_detail}
\caption{Detailed path analysis depicting the individual segments of path 4. Their corresponding error contribution can
be seen in fig. \ref{fig:errorTimedNexus}. Even though the shortest path suggested by the system is ignored multiple
times ($3'$ and $3''$) our approach is still able to improve the overall localisation error.}
\label{fig:nexusPathDetails}
\end{figure}
% overall error-distribution for nexus and galaxy
\begin{figure}
\input{gfx/eval/error_dist_nexus}
@@ -85,11 +136,17 @@
clearly provide an enhancement for the overall localization process.}
\label{fig:errorDistNexus}
\end{figure}
\begin{figure}
\input{gfx/eval/error_dist_galaxy}
\caption{Nicht so markant beim galaxy, denke aber der platz reicht eh nicht, also einfach kurz erwaehnen}
\end{figure}
%\begin{figure}
% \input{gfx/eval/error_dist_galaxy}
% \caption{Nicht so markant beim galaxy, denke aber der platz reicht eh nicht, also einfach kurz erwaehnen}
%\end{figure}
The error values for all other paths and the other smartphone are listed in table
\ref{tbl:errGalaxy} and \ref{tbl:errNexus}. As can be seen, adding prior knowledge
is able to improve the localisation for all examined situations, even when
leaving the suggested path or when facing bad/slow sensor readings.
% error values
\begin{table}
\centering
\begin{tabular}{|l|c|c|c|c|}
@@ -99,6 +156,7 @@
Shortest (\refeq{eq:transShortestPath}) & \SI{2.72}{\meter} & \SI{2.98}{\meter} & \SI{2.48}{\meter} & \SI{3.06}{\meter} \\\hline
Multipath (\refeq{eq:transMultiPath}) & \SI{2.62}{\meter} & \SI{2.14}{\meter} & \SI{2.46}{\meter} & \SI{2.75}{\meter} \\\hline
\end{tabular}
\label{tbl:errNexus}
\caption{Median error for walks conducted with the Nexus 6.}
\end{table}
@@ -111,6 +169,7 @@
Shortest (\refeq{eq:transShortestPath}) & \SI{ 5.86}{\meter} & \SI{4.14}{\meter} & \SI{5.14}{\meter} & \SI{5.20}{\meter} \\\hline
Multipath (\refeq{eq:transMultiPath}) & \SI{ 6.35}{\meter} & \SI{4.21}{\meter} & \SI{5.03}{\meter} & \SI{6.79}{\meter} \\\hline
\end{tabular}
\label{tbl:errGalaxy}
\caption{Median error for walks conducted with the Galaxy S5.}
\end{table}
@@ -128,6 +187,8 @@
\label{fig:bergwerkPath3Galaxy}
\end{figure}
\begin{itemize}
\item Nochmal kurz auf die Probleme des letzten Systems eingehen (schon teil der introduction)
\item Da letztes mal nur 1 Pfad, machen wir dieses mal mehrere!
@@ -136,27 +197,6 @@
\item Analysiere Probleme ggf. mit schönen Grafiken.
\item Vergleich zum Schluss das neue System mit dem Alten um eine schöne Conclusion der Verbesserungen einzuleiten.
\end{itemize}
\commentByFrank{we start with a uniform distribution $\mStateVec_0$}
\commentByFrank{hinweis auf die verschiedenen geraete (smartphones) und unterschiede, wlan/baro}
\commentByFrank{
PATH4 HAELT SICH NICHT AN DEN SHORTEST PATH.
GUTES BEISPIEL.
der pfad wechselt sogar 2x! (3. stock)
der shortest wird am ende etwas ungenau bei der treppe
}
\commentByFrank{sensorausfall simulieren, z.b. in der mitte, oder auf einer treppe}
\commentByFrank{zwischendrin mal stehenbleiben und schauen ob auch das klappt}
\commentByFrank{pfad verlassen und ganz wo anders hingehen}
\commentByFrank{die reine importance selbst auf dem graphen hilft, aber nur minimal weiter}
\commentByFrank{pfad4 nexus. pfadlos laeuft mit ach und krach richtig (treppenhaus, wlan schlecht)
mit pfad laeuft es falsch, weil die andere treppe kuerzer zum ziel ist und das wlan dort besser passt}
\commentByFrank{zu grosser einfluss vom pfad ist also kein allheilmittel.. kann, wie beim treppenhaus, auch nach hinten los gehen}
\commentByFrank{path1: bad start due to nearby AP and bad parameters (path-loss too high): high starting errors: median better}