FHWS/IPIN2017
Archived
3
0
Fork 0
Code Issues 4 Pull Requests Releases Wiki Activity

tex v2 - without experiments

Browse Source
This commit is contained in:
toni
2017-05-10 23:33:01 +02:00
parent 5b2e1b0c65
commit 210fae56b7
11 changed files with 4856 additions and 10840 deletions
Download Patch File Download Diff File Expand all files Collapse all files

132
tex/chapters/experiments.tex
View File

@@ -5,51 +5,41 @@
\begin{figure}
\centering
\input{gfx/eval/paths.tex}
\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
\caption{The three paths that were part of the experiments. Starting positions are marked with black circles. The red squares illustrate the \docWIFI{} quality in this sector. The intensity of red indicates a low coverage and thus a bad quality for localisation.}
\label{fig:paths}
\end{figure}
%
%Gebäude
All upcoming experiments were carried out on four floors (0 to 3) of a \SI{77}{m} x \SI{55}{m} sized faculty building.
It includes several staircases and elevators and has a ceiling height of about \SI{3}{m}.
Nevertheless, the grid was generated for the complete campus and thus outdoor areas like the courtyard are also walkable.
As Wi-Fi is attenuated by obstacles and walls, it does not provide a consistent quality over the complete building.
In fig. \ref{fig:paths} we illustrate the quality obtained by the wall attenuation factor model presented earlier.
Here, the intensity of red indicates a low coverage and thus a bad quality for localisation.
To obtain this information we interpolated the Wi-Fi quality factor given by all test walks using $l_{\text{max}} = \SI{-75}{\dBm}$ and $l_{\text{min}} = \SI{-90}{\dBm}$.
As mentioned before, we omit any time-consuming calibration processes and use the same values for all access-points. That would be $P_{0_{\text{wifi}}} = \SI{-46}{\dBm}, \mPLE_{\text{wifi}} = \SI{2.7}{}, \mWAF_{\text{wifi}} = \SI{8}{\dB}$.
The position of the access-points (about five per floor) is known beforehand.
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.
The gridded graph was generated for the complete campus and thus outdoor areas like the courtyard are also walkable.
As \docWIFI{} is attenuated by obstacles and walls, it does not provide a consistent quality over the complete building.
To get an idea about the \docWIFI{} quality, we interpolated the \docWIFI{} quality factor $q(\mObsVec_t^{\mRssiVec_\text{wifi}})$, with $l_{\text{max}} = \SI{-75}{\dBm}$ and $l_{\text{min}} = \SI{-90}{\dBm}$, between some measurements using a nearest-neighbour procedure.
In fig. \ref{fig:paths} the resulting colourized floorplan is illustrated.
Here, the intensity of red indicates a low signal strength and thus a bad quality for localising a pedestrian within this area using \docWIFI{}.
% gewählte parameter (auch mal die optimieren wifi parameter testen)
%Pfade
We arranged three distinct walks (see also fig. \ref{fig:paths}).
The measurements for the walks were recorded using a Motorola Nexus 6 at 2.4 GHz band only.
The computation was done offline as described in algorithm \ref{fig:paths}.
For each walk we deployed $50$ MC runs using 5000 Particles for each mode.
Instead of an initial position and heading, all walks start with a uniform distribution (random position and heading) as prior.
For the filtering we used $\sigma_\text{wifi} = 8.0$ as uncertainties, both growing with each measurement's age.
While the pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters are determined automatically.
The step size $\mStepSize$ for the transition was configured to be \SI{70}{\centimeter} with an allowed derivation of \SI{10}{\percent}.
The heading deviation was set to \SI{25}{\degree}.
The pedestrian's position (state) was estimated using the weighted arithmetic mean of
the particle set.
For each walk we deployed $50$ runs using 5000 particles for each mode.
Instead of an initial position and heading, all walks start with a uniform distribution (random position and heading) as prior $q_1$.
In the graph-based transition of the dominant filter, the to-be-walked distance is given by the number of steps using a step size of \SI{70}{\centimeter} with an allowed deviation of \SI{10}{\percent}.
The deviation for the walking direction was set to \SI{25}{\degree}.
Since the simple transition randomly scatters particles within a specific range, we choose a covariance matrix that allows a variance of \SI{200}{\centimeter} in $x$- and $y$-direction for the multivariate normal distribution.
Here, floors were changed by deploying a discrete distribution for every floor level, providing a chance of \SI{27}{\percent} for changing one floor and \SI{5}{\percent} for two floors in a particular $z$-direction.
% wie für die kld gezogen? begründen warum wir nun keine parzenschätzung machen (weil ähnliche ergebnisse)
To calculate \eqref{equ:KLD} and thus the Kullback-Leibler divergence, we need to sample densities from both modes likewise.
The grid is suitable for this purpose.
However, sampling at any vertex $\mVertexA$ of the grid, given just a set of random variables (particles), is not the easiest task.
We need to estimate the posterior distribution given by the respective particle sets.
A common way is to deploy a kernel density estimation using a Gaussian distribution as kernel.
The density of a specific point $\hat\mStateVec_{t} = \fPos{\mVertexA}$ is then given by
%
\begin{equation}
p(\hat\mStateVec_{t} \mid m_t, \mObsVec_{1:t}) = \sum_{i=1}^{N_{m_t}} \mathcal{N}(d^i_{\text{KL}} \mid 0, \sigma_{\text{KL}})
\enspace ,
\end{equation}
%
while $d^i_{\text{KL}}$ is the euclidean distance between the considered point's $\hat\mStateVec_{t}$ and all particles $\fPos{\vec{X}_t^{i,m_t}}$ of the mode. The variance $\sigma_{\text{KL}}$ is set to \SI{1}{m}.
It is well known, that the computation of the kernel density estimation is rather slow, thus we also used a much simpler estimation by assuming a multivariate Gaussian distribution for both modes.
Here, the mean is given by weighted arithmetic mean of the particles and the variance is defined by the sample covariance matrix.
Calculating a meaningful $D_{\text{KL}}$, both estimation methods performed almost identical and therefore we used the multivariate Gaussian distribution for both modes with $\lambda = 0.03$ for the upcoming experimental discussion.
We omit any time-consuming calibration processes and therefore use the same parameters for all \docWIFI{} access-points, similar to \cite{Ebner-15}.
The position of the access-points (about five per floor) is known beforehand.
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.
To model the uncertainty of the \docWIFI{}, a white Gaussian noise with $\sigma_\text{wifi} = \SI{8}{\dBm}$, growing with each measurement's age, was chosen.
The pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters are determined automatically as described in \cite{Fetzer2016OMC}.
As expected before, the kernel density estimation in \eqref{eq:KDE} and the much simpler multivariate normal distribution estimation are both producing similar results in context of the divergence measure given in \eqref{equ:KLD}.
For this reason and because of the faster processing time, especially in context of calculating on a smartphone, we estimated the posteriors of the modes by approximating a multivariate normal distribution (cf. section \ref{sec:divergence}).
For calculating \eqref{equ:KLD} we set $\lambda = 0.03$ based on best practices.
Finally, the pedestrian's most likely position (state) was then estimated using the weighted arithmetic mean of the particle set.
% ground truth
The ground truth is measured by recording a timestamp at marked spots on the walking route. When passing a marker, the pedestrian clicked a button on the smartphone application.
@@ -65,24 +55,24 @@ The approximation error is then calculated by comparing the interpolated ground
\centering
\input{gfx/eval/path3.tex}
\input{gfx/eval/path3-kld.tex}
\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
\caption{Exemplary results on path 3 for the common particle filter using the graph-based (red) or simple transition model (blue) and our IMMPF approach (green). The Kullback-Leibler divergence $D_{\text{KL}}$ between the standalone filters (purple) proves itself as a good indicator, if one filter gets stuck or loses track.}
\label{fig:path3}
\end{figure}
%
At first, we discuss the results of path 3, starting at the left-hand side of the building.
Exemplary estimation results, using the modes standalone and combined within the IMMPF, can be seen in fig. \ref{fig:path3}.
As mentioned above, every run of a walk starts with a uniform distribution as prior.
Due to a low Wi-Fi coverage at the starting point, the pedestrian's position is falsely estimated into a room instead of the corridor.
All three filters are able to overcome this false detection.
However, the common particle filter (red) gets then indissoluble captured within a room, because of its restrictive behaviour and the aftereffects of the initial Wi-Fi readings.
It provides an \SI{75}{\percent}-quantil of $\tilde{x}_{0.75} = \SI{3884}{\centimeter}$ and got captured in \SI{100}{\percent} of all runs.
As expected and discussed earlier, the simple transition (blue) is less prone to bad observations and provides not so accurate, but very robust results of $\tilde{x}_{0.75}= \SI{809}{\centimeter}$ and a standard deviation over all results of $\bar{\sigma} = \SI{529}{\centimeter}$.
It provides an \SI{75}{\percent}-quantil of $\tilde{x}_{75} = \SI{3884}{\centimeter}$ and got captured in \SI{100}{\percent} of all runs.
As expected and discussed earlier, the simple transition (blue) is less prone to bad observations and provides not so accurate, but very robust results of $\tilde{x}_{75}= \SI{809}{\centimeter}$ and a standard deviation over all results of $\bar{\sigma} = \SI{529}{\centimeter}$.
Looking at $D_{\text{KL}}$ over time confirms our assumption made in section \ref{sec:immpf}.
It is clearly visible, that the Kullback-Leibler divergence between both modes (purple) is a very good indicator to observe, if the dominant filter gets stuck or loses track.
The graph-based filter (red) gets stuck and is not able to recover, starting on from this point, the Kullback-Leibler divergence $D_{\text{KL}}$ (purple) further increases due to the growing distance between both filters (blue and red).
It is clearly visible, that this divergence between both filters is a very good indicator to observe, if a filter gets stuck or loses track.
Following, the IMMPF (green) results in a very natural and straight path estimation and a low $D_{\text{KL}}$ between modes and no sticking.
The benefits of mixing both filtering schemes within the scenario of path 3 are thus obvious.
The IMMPF filters with an error of $\tilde{x}_{0.75} = \SI{667}{\centimeter}$ and $\bar{\sigma} = \SI{558}{\centimeter}$.
The IMMPF filters with an error of $\tilde{x}_{75} = \SI{667}{\centimeter}$ and $\bar{\sigma} = \SI{558}{\centimeter}$.
% zeigen das schlechtes wi-fi (zu hohe diversity) behoben wird.
% bild: lauf auf der rechten seite des gebäudes zeige mit und ohne wifi faktor (schlechtes wifi einzeichnen)
@@ -90,7 +80,7 @@ The IMMPF filters with an error of $\tilde{x}_{0.75} = \SI{667}{\centimeter}$ an
\centering
\input{gfx/eval/path2.tex}
\input{gfx/eval/path2-wifi-quality.tex}
\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
\caption{Comparison of the estimation results on path 2 with (green) and without (red) the Wi-Fi quality factor in the Markov transition matrix. The low Wi-Fi quality and thus high errors between the \SI{80}{th} and \SI{130}{th} second are caused by the high attenuation and low signal coverage inside the zig-zag stairwell on the building's backside.}
\label{fig:path2}
\end{figure}
%
@@ -99,11 +89,11 @@ Here, the overall Wi-Fi quality is rather low, especially in the zig-zag stairwe
Path 2 starts in the second floor, walking town the centred stairs into the first floor, then making a right turn and walking the stairs down to zero floor, from there we walk back to second floor using the zig-zag stairwell and after finally crossing a room we are back at the start.
This is a very challenging scenario, at first the estimation got stuck on the first floor in a room's corner and after that the Wi-Fi is highly attenuated.
Looking at fig. \ref{fig:path2}, one can observe the impact of the Wi-Fi quality factor within the Markov transition matrix.
Without it, the position estimation (red) is drifting in the courtyard, missing the stairwell and producing high errors between \SI{80}{th} and \SI{130}{th} second.
Without it, the position estimation (red) is drifting in the courtyard, missing the stairwell and producing high errors between the \SI{80}{th} and \SI{130}{th} second.
As described before, the bad Wi-Fi readings are causing $D_{\text{KL}}$ to grow.
It follows that the accurate dominant filter draws new particles from the uncertain support and therefore worsen the position estimation.
In this scenario it is cold comfort that the system is able to recover thanks to its high diversity during situations with uncertain measurements.
Only by adding the Wi-Fi quality factor the system is able to improve the approximated path (green) and the overall estimation results from $\tilde{x}_{0.75} = \SI{1278}{\centimeter}$ with $\bar{\sigma} = \SI{948}{\centimeter}$ to $\tilde{x}_{0.75} = \SI{811}{\centimeter}$ with $\bar{\sigma} = \SI{340}{\centimeter}$.
Only by adding the Wi-Fi quality factor the system is able to improve the approximated path (green) and the overall estimation results from $\tilde{x}_{75} = \SI{1278}{\centimeter}$ with $\bar{\sigma} = \SI{948}{\centimeter}$ to $\tilde{x}_{75} = \SI{953}{\centimeter}$ with $\bar{\sigma} = \SI{543}{\centimeter}$.
However, this is far from perfect and in some cases ($\sim \SI{9}{\percent}$) the quality factor was not able to prevent the estimation to drift in the courtyard.
This solely happened when particles were sampled directly onto the courtyard while changing from first to zero floor.
Those particles then received a high weight due to the attenuated measurements, causing a weight degeneracy.
@@ -115,43 +105,49 @@ Adapting the bounds $l_{\text{max}}$ and $l_{\text{min}}$ of the quality factor
\centering
\input{gfx/eval/path1.tex}
\input{gfx/eval/path1-time.tex}
\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
\caption{Estimation results and error development while walking alongside path 1. In \SI{20}{\percent} of cases, the the graph-based particle filter failed to detect the first floor change. Therefore, a good (blue) and a bad (red) result are shown. The here presented approach (green) never lost track.}
\label{fig:path1}
\end{figure}
%
An exemplary result for path 1 is illustrated in fig. \ref{fig:path1}.
The path starts on the first floor and finishes on the third after walking two straight stairs.
Using the grid-based particle filter for localisation, we were able to provide an very accurate path (blue) in \SI{80}{\percent} of the MC runs providing $\tilde{x}_{0.75} = \SI{526}{\centimeter}$ with $\bar{\sigma} = \SI{316}{\centimeter}$.
Using the graph-based particle filter for localisation, we were able to obtain a very accurate path (blue) in \SI{80}{\percent} of the runs providing $\tilde{x}_{75} = \SI{526}{\centimeter}$ with $\bar{\sigma} = \SI{316}{\centimeter}$.
Due to a lack of particles near the stairs, the other \SI{20}{\percent} failed to detect the first floor change (red).
Using our approach (green), we were able detect all floor changes and thus never lost track.
It performs with $\tilde{x}_{0.75} = \SI{544}{\centimeter}$ and $\bar{\sigma} = \SI{281}{\centimeter}$.
It performs with $\tilde{x}_{75} = \SI{544}{\centimeter}$ and $\bar{\sigma} = \SI{281}{\centimeter}$.
Those very similar estimation results confirm the efficiency of the mixing and how it is able to keep the accuracy while providing a higher robustness against failures.
\todo{mehr die ergebnisse von bild 5 diskutieren. an manchen stellen verlieren wir genauigkeit, an anderen wird es besser.}
% gegenüberstellung aller pfade und werte in tabelle
\definecolor{header}{rgb}{.8, .8, .8}
\begin{table}
\caption{Median error for all conducted walks.}
\label{tbl:err}
\centering
\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}
\hline
& \multicolumn{3}{c}{Path 1} & \multicolumn{3}{|c|}{Path 2} & \multicolumn{3}{|c|}{Path 3}\\
\hline
& $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{0.75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{0.75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{0.75}$ \\
\hline
PF_{\text{grid}}& $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ \\
\hline
PF_{\text{simple}}& $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ \\
\hline
IMMPF & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ \\
\hline
\end{tabular}
\end{table}
% \begin{table}
% \caption{Resulting Errors for all conducted walks in meter.}
% \label{tbl:err}
% \centering
% \scalebox{0.93}{
% \begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}
% \hline
% & \multicolumn{3}{c|}{Path 1} & \multicolumn{3}{c|}{Path 2} & \multicolumn{3}{c|}{Path 3}\\
% \hline
% & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{75}$ \\
% \hline
% $\text{PF}_{\text{graph}}$ & $4.0$ & $3.2$ & $5.3$ & $8.2$ & $4.0$ & $10.7$ & $30.3$ & $12.8$ & $38.8$ \\
% \hline
% $\text{PF}_{\text{simple}}$ & $4.9$ & $2.8$ & $6.2$ & $7.3$ & $2.9$ & $9.4$ & $6.8$ & $5.4$ & $8.1$ \\
% \hline
% IMMPF & $4.2$ & $2.8$ & $5.4$ & $7.7$ & $5.4$ & $9.5$ & $6.3$ & $5.6$ & $6.7$ \\
% \hline
% \end{tabular}
% }
% \end{table}
An overview of all localisation results can be seen in table \ref{tbl:err}.
Again, it should be noted that the localisation system used for the experiments is very basic and can be seen as a slimmed version of our previous works \cite{}.
%An overview of all localisation results can be seen in table \ref{tbl:err}.
The here presented walks were selected because they fail in some way using a restrictive transition model and thus are well suited to represent the benefits and drawbacks of the IMMPF approach.
%So the results of table \ref{tbl:err} should not be seen as best case localization results, but more as proofing robustness while providing room for further improvements.
In this context, it should be noted that the localisation system used for the experiments is very basic and can be seen as a slimmed version of our previous works \cite{Fetzer2016OMC, Ebner-16}.
Optimizing the Wi-Fi parameters and adding additional methods will improve the localisation results significantly.
Especially, the graph-based transition model allows many optimizations and performance boosts.
More importantly, the here presented approach was able to recover in all situations and thus never got completely stuck within a demarcated area.
All results were similar or more accurate then the ones provided by the standalone filters, even when the localisation did not suffer from any problems.