\section{Experiments}

As explained at the very beginning of this work, we wanted to explore the limits of the here presented localization system.
By utilizing it to a 13th century historic building, we created a challenging scenario not only because of the various architectural factors, but also because of its function as a museum.
During all experiments, the museum was open to the public and had a varying number of \SI{10}{} to \SI{50}{} visitors while recording.

The \SI{2500}{\square\meter} building consists of \SI{6}{} different levels, which are grouped into 4 floors (see fig. \ref{fig:apfingerprint}). 
Thus, the ceiling height is not constant over one floor and varies between \SI{2.6}{\meter} to \SI{3.6}{\meter}.
In the middle of the building is an outdoor area, which is only accessible from one side. 
While most of the exterior and ground level walls are made of massive stones, the floors above are half-timbered constructions.
Due to different objects like exhibits, cabinets or signs not all positions within the building were walkable. 
For the sake of simplicity we did not incorporate such knowledge into the floorplan. 
Thus, the floorplan consists only of walls, ceilings, doors, windows and stairs.
It was created using our 3D map editor software based on architectural drawings from the 1980s. 

The computation of the state estimation as well as the \docWifi{} optimization are done offline using an Intel Core i7-4702HQ CPU with a frequency of \SI{2.2}{GHz} running \SI{8}{cores} and \SI{16}{GB} main memory. 
However, similar to our previous, award-winning system, the setup is able to run completely on commercial smartphones as well as it uses C++ code \cite{torres2017smartphone}. 
%Sensor measurements are recorded using a simple mobile application that implements the standard Android SensorManager. 

The experiments are separated into three sections:
At first, we discuss the performance of the novel transition model and compare it to a grid-based approach.  
In section \ref{sec:exp:opti} we have a look at \docWIFI{} optimization and how the real \docAPshort{} positions differ from it.
Following, we conducted several test walks throughout the building to examine the estimation accuracy (in \SI{}{\meter}) of the localisation system. 
We try to resolve sample impoverishment with the here presented method and compare the different estimation methods as presented in section \ref{sec:estimation}. 


\subsection{Transition}
To make a statement about the performance of our novel transition model presented within section \ref {}, we 

\todo{Unser liebes Treppensteigen. Vergleich altes und neues Bewegungsmodell.}

\subsection{\docWIFI{} Optimization}
\label{sec:exp:opti}

%wie viele ap sind es insgesamt?
The \docAPshort{} positions as well as the fingerprints used for optimization can be seen in fig. \ref{fig:apfingerprint}.
As described in section \ref{sec:wifi} we used \SI{42}{} WEMOS D1 mini to provide a \docWIFI{} infrastructure throughout the building.
The position of every installed beacon was measured using a laser scanner. 
This allows a comparison with the optimized \docAPshort{} positions. 
Within all Wi-Fi observations, we only consider the beacons, which are identified by their well-known MAC address. 
Other transmitters like smart TVs or smartphone hotspots are ignored as they might cause estimation errors.
%wie fingerprints aufgenommen, wie viele ...

\begin{figure}[bt]
	\centering
  \includegraphics[width=0.9\textwidth]{gfx/floorplanDummy.png}
	\caption{Floorplan Dummy}
	\label{fig:apfingerprint}
\end{figure}


%kurze beschreibung was wir jetzt alles testen wollen.


%was kommt bei der optimierung raus. vergleichen mit ground truth. auch den fehler gegenüberstellen. 
%man sollte sehen das ohne optimierung gar nichts geht.

\subsection{Location Estimation Error}


\begin{figure}[ht]
	\centering
  \includegraphics[width=0.9\textwidth]{gfx/floorplanDummy.png}
	\caption{Floorplan Dummy}
	\label{fig:floorplan}
\end{figure}
%
The 4 arranged walks can be seen in fig. \ref{fig:floorplan}.
They were carried out be 4 different male testers using either a Samsung Note 2, Google Pixel One or Motorola Nexus 6 for recording the measurements. 
All in all, we recorded \SI{28}{} distinct measurement series, \SI{7}{} for each walk.
A walk is indicated by a set of numbered markers, fixed to the ground. 
Small icons on those markers give the direction of the next marker and in some cases provide instructions to pause walking for a certain time.
The intervals for pausing vary between \SI{10}{\second} to \SI{60}{\second}. 
The ground truth is then measured by recording a timestamp while passing a marker. 
For this, the tester clicks a button on the smartphone application. 
Between two consecutive points, a constant movement speed is assumed.
Thus, the ground truth might not be \SI{100}{\percent} accurate, but fair enough for error measurements. 
The approximation error is then calculated by comparing the interpolated ground truth position with the current estimation \cite{Fetzer-16}.

%computation und monte carlo runs
For each walk we deployed 50 runs using a varying size of particles. 
Instead of an initial position and heading, all walks start with a uniform distribution (random position and heading) as prior.


%probleme mit impoverishment aufzeigen, wo bringt es was, was macht es kaputt etc pp

%%estimation

%wie in bulli paper.

%letzer absatz nochmal gesamtergebniss des gesamten systems
%was läuft noch schief? wo macht was probleme?


\begin{itemize}
	\item Noch ein paar Dinge über das gebäude und das setup an sich
	\item auf was wurde geachtet, wie wurden die ap's gesetzt. etc pp.
	\item wie wurde ground truth gemacht
	\item wie viele testaufnahmen... 
	\item die einzelnen pfade gegenüberstellen. (videos irgendwie bereitstellen?)
	\item schätzung der ap positionen vs reale ap positionen
	\item fehler gegenüberstellen, genauigkeiten. 
	\item allgemein an den parametern rumspielen: Anzahl Partikel (da wir das eig noch nicht so wirklich oft gemacht haben) 
	\item estimation methoden gegenüberstellen (diskussion aus bulli paper)
	\item aktive probleme aufzeigen (verlaufen, hängenblieben, schlechte ap signale ... 
\end{itemize}

wir experimentieren auf allen vieren.

