FHWS/IPIN2018
Archived
4
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

schreibe schreibe eval schreibe

Browse Source
This commit is contained in:
toni
2018-06-21 16:47:08 +02:00
parent 3cebfb8839
commit c99b276c4f
2 changed files with 109 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tex/bare_conf.tex
View File

@@ -26,6 +26,8 @@
\usepackage{eqparbox}
\usepackage{epstopdf}
\usepackage{siunitx}
\usepackage{array}
\usepackage{multirow}
%\updates{yes} % If there is an update available, un-comment this line

125
tex/chapters/experiments.tex
View File

@@ -13,6 +13,8 @@ For the sake of simplicity we did not incorporate such knowledge into the floorp
Thus, the floorplan consists only of walls, ceilings, doors, windows and stairs.
It was created using our 3D map editor software based on architectural drawings from the 1980s.
Sensor measurements are recorded using a simple mobile application that implements the standard Android sensor functionalities.
As smartphones we used either a Samsung Note 2, Google Pixel One or Motorola Nexus 6.
The computation of the state estimation as well as the \docWifi{} optimization are done offline using an Intel Core i7-4702HQ CPU with a frequency of \SI{2.2}{GHz} running \SI{8}{cores} and \SI{16}{GB} main memory.
However, similar to our previous, award-winning system, the setup is able to run completely on commercial smartphones as well as it uses C++ code \cite{torres2017smartphone}.
%Sensor measurements are recorded using a simple mobile application that implements the standard Android SensorManager.
@@ -25,7 +27,7 @@ We try to resolve sample impoverishment with the here presented method and compa
\subsection{Transition}
To make a statement about the performance of our novel transition model presented within section \ref {}, we
To make a statement about the performance of our novel transition model presented within section \ref {}, we chose a simple scenario, in which a tester walks up and down a staircase three times.
\todo{Unser liebes Treppensteigen. Vergleich altes und neues Bewegungsmodell.}
@@ -57,16 +59,16 @@ Other transmitters like smart TVs or smartphone hotspots are ignored as they mig
\subsection{Location Estimation Error}
\begin{figure}[ht]
\centering
\includegraphics[width=0.9\textwidth]{gfx/floorplanDummy.png}
\includegraphics[width=0.9\textwidth]{gfx/floorplanDummy.png}
\caption{Floorplan Dummy}
\label{fig:floorplan}
\end{figure}
%
The 4 arranged walks can be seen in fig. \ref{fig:floorplan}.
The 4 chosen walking paths can be seen in fig. \ref{fig:floorplan}.
They were carried out be 4 different male testers using either a Samsung Note 2, Google Pixel One or Motorola Nexus 6 for recording the measurements.
\todo{walks noch weng erläutern, länge und dauer, many different other walks were made, however those 4 where chosen because most challenging etc. pp.}
All in all, we recorded \SI{28}{} distinct measurement series, \SI{7}{} for each walk.
A walk is indicated by a set of numbered markers, fixed to the ground.
Small icons on those markers give the direction of the next marker and in some cases provide instructions to pause walking for a certain time.
@@ -78,32 +80,119 @@ Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough
The approximation error is then calculated by comparing the interpolated ground truth position with the current estimation \cite{Fetzer-16}.
%computation und monte carlo runs
For each walk we deployed 50 runs using a varying size of particles.
For each walk we deployed 100 runs using \SI{5000}{particles} particles.
Instead of an initial position and heading, all walks start with a uniform distribution (random position and heading) as prior.
The overall localisation results can be see in table \ref{table:overall}.
Here, we differ between the single impoverishment techniques.
For a better overview, we only used the KDE-based estimation, as the errors compared to the weighted average estimation differ by only a few centimetres.
a simple filter (weighted average estimation + simple impoverishment solution) and an advanced filter (KDE estimation + $D_\text{KL}$-based impoverishment solution).
It can be seen that...
the results include also all failed walks, to show the benefits of using an anti impoverishment method. in walk 0 at walk 1 20 percent of walks failed ...
of course it would be possible to resets the system at this point, but thats not our anspruch...
we want to show a real worst case scenario!
\todo{providing a penalty for wrong floors. sonst haben wir das problem das der overall error einfach nicht unterschiedlich genug ist. }
\newcommand{\STAB}[1]{\begin{tabular}{@{}c@{}}#1\end{tabular}}
\begin{table}[!h]
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
\hline
Method & \multicolumn{3}{c|}{none} & \multicolumn{3}{c|}{simple} & \multicolumn{3}{c|}{kde}\\
\hline
& $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ \\
\hline \hline
Walk 0 & \SI{1315}{\centi\meter} & \SI{1136}{\centi\meter} & \SI{2266}{\centi\meter} & \SI{1189}{\centi\meter} & \SI{1092}{\centi\meter} & \SI{2019}{\centi\meter} & \SI{301}{\centi\meter} & \SI{252}{\centi\meter} & \SI{376}{\centi\meter} \\ \hline
Walk 1 & \SI{318}{\centi\meter} & \SI{243}{\centi\meter} & \SI{402}{\centi\meter} & \SI{318}{\centi\meter} & \SI{240}{\centi\meter} & \SI{403}{\centi\meter} & \SI{342}{\centi\meter} & \SI{256}{\centi\meter} & \SI{440}{\centi\meter} \\ \hline
Walk 2 & \SI{589}{\centi\meter} & \SI{403}{\centi\meter} & \SI{843}{\centi\meter} & \SI{321}{\centi\meter} & \SI{210}{\centi\meter} & \SI{431}{\centi\meter} & \SI{342}{\centi\meter} & \SI{219}{\centi\meter} & \SI{455}{\centi\meter} \\ \hline
Walk 3 & \SI{462}{\centi\meter} & \SI{337}{\centi\meter} & \SI{701}{\centi\meter} & \SI{407}{\centi\meter} & \SI{306}{\centi\meter} & \SI{599}{\centi\meter} & \SI{341}{\centi\meter} & \SI{253}{\centi\meter} & \SI{462}{\centi\meter} \\
\hline
\end{tabular}
\caption{Overall localization results using the different impoverishment methods. The error is given by the \SI{75}{\percent}-quantil Used only kde for estimation, since kde and avg nehmen sich nicht viel. fehler kleiner als 10 cm im durchschnitt deshalb der übersichtshalber weggelassen. }
\label{table:overall}
\end{table}
%vielleicht die avg / kde unterscheidung weg lassen? dafür avg, std und 75%
It is clearly visible that bla outperformce blah at path 4.
Exemplary estimation results for walk 4 can be seen in fig. \ref{}.
The 75 quantil gibt aufschluss, über aufgretenedes impoverishment. due the 100 runs, some of the walks, where impoverishment occured are average out.
neff = 0.85
boxkde 0.2 point2(1,1)
wifi useregionalopt=true
BILD: Von einem Pfad der steckenbleibt und den beiden anderen verfahren mit fehler über die zeit.
BILD: WIFI-Fehler unten bei den Kellern.
BILD: Estimation Fehler
%To analyse the drawbacks and benefits of the here presented method to resolve sample impoverishment,
%The benefits of the here presented solution to resolve sample impoverishment can be seen in the example shown in fig. \ref{}.
%probleme mit impoverishment aufzeigen, wo bringt es was, was macht es kaputt etc pp
%%estimation
\begin{figure}
\centering
\input{gfx/walk.tex}
\caption{Occurring bimodal distribution caused by uncertain measurements in the first \SI{13.4}{\second} of the walk. After \SI{20.8}{\second}, the distribution gets unimodal. The weigted-average estimation (blue) provides a high error compared to the ground truth (solid black), while the BoxKDE approach (orange) does not. }
\label{fig:realWorldMulti}
\end{figure}
As discussed in chapter \ref{}, the main advantage of a KDE-based estimation is that it provides the "correct" mode of a density, even under a multimodal setting.
A situation in which the system highly benefits from this is illustrated in fig. \ref{fig:realWorldMulti}.
Here, a set of particles splits apart, due to uncertain measurements and multiple possible walking directions.
Indicated by the black dotted line, the resulting bimodal posterior reaches its maximum distance between the modes at \SI{13.4}{\second}.
Thus, a weighted average estimation (blue line) results in a position of the pedestrian somewhere outside the building (light green area).
The ground truth is given by the black solid line.
The KDE-based estimation (orange line) is able to provide reasonable results by choosing the "correct" mode of the density.
After \SI{20.8}{\second} the setting returns to be unimodal again.
Due to a right turn the lower red particles are walking against a wall and thus punished with a low weight.
Although, situations as displayed in fig. \ref{fig:realWorldMulti} frequently occur, the KDE-estimation is not able to improve the overall estimation results.
This can be seen in the corresponding error development over time plot given by fig. \ref{fig:realWorldTime}.
Here, the KDE-estimation performs slightly better then the weighted-average, however after deploying \SI{100}{} Monte Carlo runs, the difference becomes insignificant.
It is obvious, that the above mentioned "correct" mode, not always provides the lowest error.
In some situations the weighted-average estimation is often closer to the ground truth.
Within our experiments this happened especially when entering or leaving thick-walled rooms, causing slow and attenuated Wi-Fi signals.
While the system’s dynamics are moving the particles outside, the faulty Wi-Fi readings are holding back a majority by assigning corresponding weights.
Only with new measurements coming from the hallway or other parts of the building, the distribution and thus the KDE-estimation are able to recover.
\begin{figure}
\centering
\input{gfx/errorOverTime.tex}
\caption{Error development over time of a single Monte Carlo run of the walk calculated between estimation and ground truth. Between \SI{230}{\second} and \SI{290}{\second} to pedestrian was not moving.}
\label{fig:realWorldTime}
\end{figure}
As already mentioned in our previous work \cite{}.
A KDE-based approach for estimation is able to resolve multimodalities.
It does not always provide the lowest error, since it depends more on an accurate sensor model then a weighted-average approach, but is very suitable as a good indicator about the real performance of a sensor fusion system.
At the end, in the here shown examples we only searched for a global maxima, even though this approach approach opens a wide range of other possibilities for finding a best estimate.
%wie in bulli paper.
%letzer absatz nochmal gesamtergebniss des gesamten systems
%was läuft noch schief? wo macht was probleme?
\begin{itemize}
\item Noch ein paar Dinge über das gebäude und das setup an sich
\item auf was wurde geachtet, wie wurden die ap's gesetzt. etc pp.
\item wie wurde ground truth gemacht
\item wie viele testaufnahmen...
\item die einzelnen pfade gegenüberstellen. (videos irgendwie bereitstellen?)
\item schätzung der ap positionen vs reale ap positionen
\item fehler gegenüberstellen, genauigkeiten.
\item allgemein an den parametern rumspielen: Anzahl Partikel (da wir das eig noch nicht so wirklich oft gemacht haben)
\item estimation methoden gegenüberstellen (diskussion aus bulli paper)
\item aktive probleme aufzeigen (verlaufen, hängenblieben, schlechte ap signale ...
\end{itemize}
wir experimentieren auf allen vieren.