\section{Experiments}

% allgemeine infos über pfade und gebäude. wo 
% bild: mit pfaden drauf und eventl. wifi qualität in jeweiligen bereichen? (kann frank das)
	\begin{figure}
		\centering
		\input{gfx/eval/paths.tex}
		\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
		\label{fig:paths}
	\end{figure}
%
All upcoming experiments were carried out on four floors (0 to 3) of a \SI{77}{m} x \SI{55}{m} sized faculty building. 
It includes several staircases and elevators and has a ceiling height of about \SI{3}{m}. 
Nevertheless, the grid was generated for the complete campus and thus outdoor areas like the courtyard are also walkable.
As Wi-Fi is attenuated by obstacles and walls, it does not provide a consistent quality over the complete building. 
In fig. \ref{fig:paths} we illustrate the quality obtained by the wall attenuation factor model presented earlier.
Here, the intensity of red indicates a low coverage and thus a bad quality for localisation.
To obtain this information we interpolated the Wi-Fi quality factor given by all test walks using $l_{\text{max}} = \SI{-75}{\dBm}$ and $l_{\text{min}} = \SI{-90}{\dBm}$.
As mentioned before, we omit any time-consuming calibration processes and use the same values for all access-points. That would be $P_{0_{\text{wifi}}} = \SI{-46}{\dBm}, \mPLE_{\text{wifi}} = \SI{2.7}{}, \mWAF_{\text{wifi}} = \SI{8}{\dB}$.
The position of the access-points (about five per floor) is known beforehand. 
Due to legal terms, we are not allowed to depict their positions and therefore omit this information within the figures.

% gewählte parameter (auch mal die optimieren wifi parameter testen)
We arranged three distinct walks (see also fig. \ref{fig:paths}).
The measurements for the walks were recorded using a Motorola Nexus 6 at 2.4 GHz band only. 
The computation was done offline as described in algorithm \ref{fig:paths}. 
For each walk we deployed $50$ MC runs using 5000 Particles for each mode.
Instead of an initial position and heading, all walks start with a uniform distribution (random position and heading) as prior.
For the  filtering we used $\sigma_\text{wifi} = 8.0$ as uncertainties, both growing with each measurement's age.
While the pressure change was assumed to be \SI{0.105}{$\frac{\text{\hpa}}{\text{\meter}}$}, all other barometer-parameters are determined automatically. 
The step size $\mStepSize$ for the transition was configured to be \SI{70}{\centimeter} with an allowed derivation of \SI{10}{\percent}. 
The heading deviation was set to \SI{25}{\degree}.
The pedestrian's position (state) was estimated using the weighted arithmetic mean of
the particle set.

% wie für die kld gezogen? begründen warum wir nun keine parzenschätzung machen (weil ähnliche ergebnisse)
To calculate \eqref{equ:KLD} and thus the Kullback-Leibler divergence, we need to sample densities from both modes likewise. 
The grid is suitable for this purpose. 
However, sampling at any vertex $\mVertexA$ of the grid, given just a set of random variables (particles), is not the easiest task. 
We need to estimate the posterior distribution given by the respective particle sets.
A common way is to deploy a kernel density estimation using a Gaussian distribution as kernel. 
The density of a specific point $\hat\mStateVec_{t} = \fPos{\mVertexA}$ is then given by 
	%
	\begin{equation}
	p(\hat\mStateVec_{t} \mid m_t, \mObsVec_{1:t}) = \sum_{i=1}^{N_{m_t}} \mathcal{N}(d^i_{\text{KL}} \mid 0, \sigma_{\text{KL}})
	\enspace ,
	\end{equation}
	%
while $d^i_{\text{KL}}$ is the euclidean distance between the considered point's $\hat\mStateVec_{t}$ and all particles $\fPos{\vec{X}_t^{i,m_t}}$ of the mode. The variance $\sigma_{\text{KL}}$ is set to \SI{1}{m}.
It is well known, that the computation of the kernel density estimation is rather slow, thus we also used a much simpler estimation by assuming a multivariate Gaussian distribution for both modes.  
Here, the mean is given by weighted arithmetic mean of the particles and the variance is defined by the sample covariance matrix. 
Calculating a meaningful $D_{\text{KL}}$, both estimation methods performed almost identical and therefore we used the multivariate Gaussian distribution for both modes with $\lambda = 0.03$ for the upcoming experimental discussion.

% ground truth
The ground truth is measured by recording a timestamp at marked spots on the walking route. When passing a marker, the pedestrian clicked a button on the smartphone application. 
Between two consecutive points, a constant movement speed is assumed.
Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough for error measurements. 
The approximation error is then calculated by comparing the interpolated ground truth position with the current estimation \cite{Fetzer2016OMC}.

%error at the beginning always very high. about 44 meters. therefore the median is better value oder 75 quantil.

% zeigen das es stucken verhindert (eventl. hier eine andere aufnahme die mitten drinnen stecken bleibt)
% bild: stucken im raum + nicht mehr stucken im raum + kld mit anzeigen
	\begin{figure}
		\centering
		\input{gfx/eval/path3.tex}
		\input{gfx/eval/path3-kld.tex}
		\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
		\label{fig:path3}
	\end{figure}
	%

At first, we discuss the results of path 3, starting at the left-hand side of the building. 
Exemplary estimation results, using the modes standalone and combined within the IMMPF, can be seen in fig. \ref{fig:path3}.
As mentioned above, every run of a walk starts with a uniform distribution as prior.
Due to a low Wi-Fi coverage at the starting point, the pedestrian's position is falsely estimated into a room instead of the corridor.   
All three filters are able to overcome this false detection. 
However, the common particle filter (red) gets then indissoluble captured within a room, because of its restrictive behaviour and the aftereffects of the initial Wi-Fi readings. 
It provides an \SI{75}{\percent}-quantil of $\tilde{x}_{0.75} = \SI{3884}{\centimeter}$ and got captured in \SI{100}{\percent} of all runs.
As expected and discussed earlier, the simple transition (blue) is less prone to bad observations and provides not so accurate, but very robust results of $\tilde{x}_{0.75}= \SI{809}{\centimeter}$ and a standard deviation over all results of $\bar{\sigma} = \SI{529}{\centimeter}$.
Looking at $D_{\text{KL}}$ over time confirms our assumption made in section \ref{sec:immpf}. 
It is clearly visible, that the Kullback-Leibler divergence between both modes (purple) is a very good indicator to observe, if the dominant filter gets stuck or loses track.
Following, the IMMPF (green) results in a very natural and straight path estimation and a low $D_{\text{KL}}$ between modes and no sticking.
The benefits of mixing both filtering schemes within the scenario of path 3 are thus obvious.
The IMMPF filters with an error of $\tilde{x}_{0.75} = \SI{667}{\centimeter}$ and $\bar{\sigma} = \SI{558}{\centimeter}$.

% zeigen das schlechtes wi-fi (zu hohe diversity) behoben wird. 
% bild: lauf auf der rechten seite des gebäudes zeige mit und ohne wifi faktor (schlechtes wifi einzeichnen)
	\begin{figure}[b]
		\centering
		\input{gfx/eval/path2.tex}
		\input{gfx/eval/path2-wifi-quality.tex}
		\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
		\label{fig:path2}
	\end{figure}
	%
Next, we investigate the performance of our approach by considering the scenario in path 2. 
Here, the overall Wi-Fi quality is rather low, especially in the zig-zag stairwell on the buildings back and the small entrance area at floor 1 (cf. fig. \ref{fig:paths}). 
Path 2 starts in the second floor, walking town the centred stairs into the first floor, then making a right turn and walking the stairs down to zero floor, from there we walk back to second floor using the zig-zag stairwell and after finally crossing a room we are back at the start. 
This is a very challenging scenario, at first the estimation got stuck on the first floor in a room's corner and after that the Wi-Fi is highly attenuated.
Looking at fig. \ref{fig:path2}, one can observe the impact of the Wi-Fi quality factor within the Markov transition matrix. 
Without it, the position estimation (red) is drifting in the courtyard, missing the stairwell and producing high errors between \SI{80}{th} and \SI{130}{th} second. 
As described before, the bad Wi-Fi readings are causing $D_{\text{KL}}$ to grow.
It follows that the accurate dominant filter draws new particles from the uncertain support and therefore worsen the position estimation.
In this scenario it is cold comfort that the system is able to recover thanks to its high diversity during situations with uncertain measurements.
Only by adding the Wi-Fi quality factor the system is able to improve the approximated path (green) and the overall estimation results from $\tilde{x}_{0.75} = \SI{1278}{\centimeter}$ with $\bar{\sigma} = \SI{948}{\centimeter}$ to $\tilde{x}_{0.75} = \SI{811}{\centimeter}$ with $\bar{\sigma} = \SI{340}{\centimeter}$.
However, this is far from perfect and in some cases ($\sim \SI{9}{\percent}$) the quality factor was not able to prevent the estimation to drift in the courtyard. 
This solely happened when particles were sampled directly onto the courtyard while changing from first to zero floor. 
Those particles then received a high weight due to the attenuated measurements, causing a weight degeneracy.
Adapting the bounds $l_{\text{max}}$ and $l_{\text{min}}$ of the quality factor or optimizing the access-points parameters can resolve this problem \cite{}.

% zeigen das immpf nicht viel schlechter als normaler pf (ohne stucken) ist. 
% bild: er schafft es nicht die treppe rauf + er schafft es immpf + er schafft es normal filter
	\begin{figure}[t]
		\centering
		\input{gfx/eval/path1.tex}
		\input{gfx/eval/path1-time.tex}
		\caption{Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy }
		\label{fig:path1}
	\end{figure}
	%
An exemplary result for path 1 is illustrated in fig. \ref{fig:path1}. 
The path starts on the first floor and finishes on the third after walking two straight stairs. 
Using the grid-based particle filter for localisation, we were able to provide an very accurate path (blue) in \SI{80}{\percent} of the MC runs providing $\tilde{x}_{0.75} = \SI{526}{\centimeter}$ with $\bar{\sigma} = \SI{316}{\centimeter}$. 
Due to a lack of particles near the stairs, the other \SI{20}{\percent} failed to detect the first floor change (red). 
Using our approach (green), we were able detect all floor changes and thus never lost track. 
It performs with $\tilde{x}_{0.75} = \SI{544}{\centimeter}$ and $\bar{\sigma} = \SI{281}{\centimeter}$. 
Those very similar estimation results confirm the efficiency of the mixing and how it is able to keep the accuracy while providing a higher robustness against failures.


% gegenüberstellung aller pfade und werte in tabelle
	\definecolor{header}{rgb}{.8, .8, .8}
	\begin{table}
		\caption{Median error for all conducted walks.}
		\label{tbl:err}
		\centering
		\begin{tabular}{|l|c|c|c|c|c|c|c|c|c|}
			\hline
			& \multicolumn{3}{c}{Path 1} & \multicolumn{3}{|c|}{Path 2} & \multicolumn{3}{|c|}{Path 3}\\
			\hline
			 &  $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{0.75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{0.75}$ & $\bar{x}$ & $\bar{\sigma}$ & $\tilde{x}_{0.75}$ \\
			\hline
			 PF_{\text{grid}}& $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$  \\
			\hline
			 PF_{\text{simple}}& $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ \\
			\hline
			 IMMPF & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ & $x$ \\
			\hline
		\end{tabular}
	\end{table}

An overview of all localisation results can be seen in table \ref{tbl:err}.
Again, it should be noted that the localisation system used for the experiments is very basic and can be seen as a slimmed version of our previous works \cite{}. 
Optimizing the Wi-Fi parameters and adding additional methods will improve the localisation results significantly. 
More importantly, the here presented approach was able to recover in all situations and thus never got completely stuck within a demarcated area. 
All results were similar or more accurate then the ones provided by the standalone filters, even when the localisation did not suffer from any problems.

%how the Markov transition matrix regulates the impact of the supporting filter in the right amount.




