FHWS/FtmPrologic
Archived
4
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Text & Meas table

Browse Source
This commit is contained in:
Markus Bullmann
2020-03-03 14:34:54 +01:00
parent aa73c73a6c
commit fd6510c770
3 changed files with 121 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
tex/bare_conf.tex
View File

@@ -30,6 +30,8 @@
\usepackage{xfrac} \usepackage{xfrac}
\usepackage{mwe} \usepackage{mwe}
\usepackage{subfig} \usepackage{subfig}
\usepackage{hyperref}
\usepackage{booktabs}
%\updates{yes} % If there is an update available, un-comment this line %\updates{yes} % If there is an update available, un-comment this line

136
tex/chapters/8_experiments.tex
View File

@@ -24,14 +24,22 @@
In all our experiments we used a Google Pixel 2 XL and Google Pixel 3a smartphone, both running Android~9. In all our experiments we used a Google Pixel 2 XL and Google Pixel 3a smartphone, both running Android~9.
Google introduced official ranging APIs based on FTM for supported devices with Android~9. Google introduced official ranging APIs based on FTM for supported devices with Android~9.
Also starting with Android~9, Google limited the number of network scans to four every two minutes. Also starting with Android~9, Google limited the number of network scans to four every two minutes.
This limitation renders the commonly used network scanning API practical useless for measuring RSSI values for localization purposes. This limitation renders the commonly used network scanning API virtually unusable for measuring RSSI values for localization purposes.
However, the new FTM ranging API has no rate limit. However, the new FTM ranging API has no rate limit.
Additionally, the new FTM API also provides RSSI values, thus with each measurement the signal strength and distance to one AP can be simultaneously obtained. Additionally, the new FTM API also provides RSSI values, thus with each measurement the signal strength and distance to one AP can be simultaneously obtained.
In our experiments every \SI{200}{ms} a FTM measurement to every known access point is queried. In our experiments every \SI{200}{ms} a FTM measurement to every known access point is queried.
This does not guarantee that distance measurements for every access point are available at that frequency.
Due to blocked sight or harsh environmental conditions the measurements can fail.
According to our observations Android issues eight FTM measurements for a single ranging request.
The API returns various values, \eg mean distance, standard deviation, RSSI, number of attempted measurements, and the number of successful measurements.
It is not documented how Android aggregates the eight measurements, but it is assumed that the arithmetic mean is calculated on the distance.
The list of access points is statically stored in the application and known beforehand. The list of access points is statically stored in the application and known beforehand.
With Android~10 it is possible to transfer the AP position dynamically using the location configuration information protocol. With Android~10 it is possible to transfer the AP position dynamically using the location configuration information protocol.
In some cases we noticed that Android did not provide FTM measurements for intervals about three up to five seconds long for unknown reasons, although multiple APs were clearly in reach. This allows a more flexible applications as access point can be added or modified dynamically without updating the client application.
In some cases we noticed that Android did not provide FTM measurements for intervals about three up to five seconds for unknown reasons, although multiple APs were clearly in reach.
Due to the rarity of this problem we decided to repeat these faulty experiment runs. Due to the rarity of this problem we decided to repeat these faulty experiment runs.
% TODO welche Daten liefert Android. Wie die konkret gemessene Distanz entshtet ist nicht bekannt. Android teilt mit wieviele FTM Messungen erfolgreich waren z.b. 7/8. etc. % TODO welche Daten liefert Android. Wie die konkret gemessene Distanz entshtet ist nicht bekannt. Android teilt mit wieviele FTM Messungen erfolgreich waren z.b. 7/8. etc.
@@ -39,25 +47,19 @@ Due to the rarity of this problem we decided to repeat these faulty experiment r
% TODO Linux kernel version; intel firmware version % TODO Linux kernel version; intel firmware version
The access points are Intel NUCs running a patched Linux to enable FTM support. The access points are Intel NUCs running a patched Linux to enable FTM support.
In total we're using eight APs based on Intel WiFi cards. In total we're using eight APs based on Intel WiFi cards.
Four of them are based on Intel Dualband-Wireless-AC 8260 cards and configured as described by \etal{Ibrahim} \cite{ibrahim2018verification}. Four of them are based on Intel Dualband-Wireless-AC~8260 cards configured as described by \etal{Ibrahim} \cite{ibrahim2018verification}.
The remaining four are based on Intel Wireless-AC 9462 modules and run a recent Linux kernel where the iwlwifi driver and hostapd are already prepared to support FTM. The remaining four are based on Intel Wireless-AC~9462 modules and run a recent Linux kernel where the iwlwifi driver and hostapd are already prepared to support FTM.
However, the driver still requires small manual changes, as a global boolean flag needs to be set to activate the FTM related code. However, the driver still requires small manual changes, as a global boolean flag needs to be set to activate the FTM related code.
In addition, the firmware of the card returns that the chip is not calibrated for FTM. In addition, the firmware of the card returns that the chip is not calibrated for FTM.
As a consequence the driver disables FTM responder functionality. As a consequence the driver disables FTM responder functionality.
Overriding this check however, allows to use the chip as responder and FTM measurements can be performed reliable. Overriding this check, however, allows to use the chip as responder and FTM measurements can be performed reliable.
At this point it is not clear to us what the exact purpose of the flag is. At this point it is not clear to us what the exact purpose of the flag is.
While it indicates that the card is not calibrated accuracy of the measurements are sufficient as shown in our experiments.
Due to regulatory limitations both wireless cards can only be configured as access points in the \SI{2.4}{GHz} frequency band. Due to regulatory limitations both wireless cards can only be configured as access points in the \SI{2.4}{GHz} frequency band.
To improve the signal quality and strength we connected two external consumer grade omni directional antennas to the wireless cards and attached them to the case of the Intel NUC. To improve the signal quality and strength we connected two external consumer grade omni directional antennas to the wireless cards and attached them to the case of the Intel NUC.
The antennas have an antenna gain of \SI{2}{dBi} as specified by the manufacturer. The antennas have an antenna gain of \SI{2}{dBi} as specified by the manufacturer.
We used a \SI{10}{cm} coaxial cable to connect the antennas to the cards, which is practically the shortest cable possible. We used a \SI{10}{cm} coaxial cable to connect the antennas to the cards, which is practically the shortest cable possible to limit the additional signal propagation delay.
All experiments were done in our university building.
The before mentioned Intel NUCs were deployed, because the existing WiFi infrastructure does not support the FTM protocol.
While the positions of the access points where chosen with localization in mind, the actual positions mimic the placement of access points for network infrastructure.
In contrast to regular stationary access points which are usually mounted on walls or ceilings, our Intel NUCs were placed on tables for practical reasons.
\subsection{Verification FTM Performance in LOS Scenario} \subsection{Verification FTM Performance in LOS Scenario}
%\begin{itemize} %\begin{itemize}
@@ -71,8 +73,8 @@ In contrast to regular stationary access points which are usually mounted on wal
% \item Bildergrid und Video % \item Bildergrid und Video
%\end{itemize} %\end{itemize}
The first experiment evaluates the precision of FTM distance measurements given different hardware configurations. The first experiment evaluates the indoor precision of FTM distance measurements given different hardware configurations.
While \etal{Ibrahim} already verified the precision of the Intel AC 8260 card in great detail, our setup differs from theirs and requires anew evaluation. While \etal{Ibrahim} \cite{ibrahim2018verification} already verified the precision of the Intel~AC~8260 card in great detail, our setup differs from theirs and requires anew evaluation.
In contrast to \etal{Ibrahim} we use smartphones as receivers and two different cards with different firmware versions as senders. In contrast to \etal{Ibrahim} we use smartphones as receivers and two different cards with different firmware versions as senders.
Additionally, it is unclear how the external antennas affect the measurements. Additionally, it is unclear how the external antennas affect the measurements.
For these reasons we did a static distance measurement experimental setup to confirm that the combination of Pixel devices and Intel cards provide reliable values. For these reasons we did a static distance measurement experimental setup to confirm that the combination of Pixel devices and Intel cards provide reliable values.
@@ -80,15 +82,19 @@ For these reasons we did a static distance measurement experimental setup to con
Our test setup consist of 10 measurement points evenly spaced on a straight line with a distance of \SI{2}{m}. Our test setup consist of 10 measurement points evenly spaced on a straight line with a distance of \SI{2}{m}.
The closest point to the AP is \SI{2}{m} away and the furthest \SI{20}{m}. The closest point to the AP is \SI{2}{m} away and the furthest \SI{20}{m}.
At every point each phone is placed on a stand and around 140 FTM measurements are recorded, which corresponds to a measure period of \SI{30}{s} per point. At every point each phone is placed on a stand.
Around 140 FTM measurements are recorded, which corresponds to a measure period of \SI{30}{s} per point.
The APs and the phones are placed on an empty cardboard box on a metal stand to allow some distance between the metal and the phone. The APs and the phones are placed on an empty cardboard box on a metal stand to allow some distance between the metal and the phone.
The phones laid flat on the box.
Both the APs and the phones are located at \SI{1.05}{m} above the floor. Both the APs and the phones are located at \SI{1.05}{m} above the floor.
While recording measurements for \SI{30}{s} at a single point is not realistic in a dynamic positioning system, it allows to evaluate the statistical properties of the method. While recording measurements for \SI{30}{s} at a single point is not realistic in a dynamic positioning system, it allows us to evaluate the statistical properties of the method.
Each distance measurement is done with every hardware combination. The whole experiment was deployed in the hallway of our university.
Each distance measurement is performed with every hardware combination.
On the receiving side we used the Google Pixel 2 XL and Pixel 3a. On the receiving side we used the Google Pixel 2 XL and Pixel 3a.
On the sending side we used the Intel AC 8260 and 9462 with internal or external \SI{2}{dBi} antennas. On the sending side we used the Intel AC 8260 and 9462 with internal and external \SI{2}{dBi} antennas.
The geometry and properties of the internal antenna are unknown. The antenna geometry and properties of the internal antenna are unknown.
In total there are eight phone AP combinations.
% TODO Table mit mean dist err, median dist err und mean RSSI % TODO Table mit mean dist err, median dist err und mean RSSI
@@ -97,24 +103,104 @@ The geometry and properties of the internal antenna are unknown.
% Erkenntnis 3: Antenne vs FTM: Keine Aussage möglich % Erkenntnis 3: Antenne vs FTM: Keine Aussage möglich
% Erkenntnis 4: Unterschied zwischen Pixel 2 und 3 ist nicht erkennbar % Erkenntnis 4: Unterschied zwischen Pixel 2 und 3 ist nicht erkennbar
\autoref{fig:DistMeasMeanNucPixel}~\subref{fig:DistMeasMeanNucPixel:a} shows the average measured distance per smartphone in respect to the ground truth distance.
Likewise, \autoref{fig:DistMeasMeanNucPixel}~\subref{fig:DistMeasMeanNucPixel:b} depicts the average measured distance per access point.
The corresponding values of these figures are shown in \autoref{tab:distvaluesPixels} and \autoref{tab:distvaluesNUCs}.
Interestingly, both the wireless cards and the Pixel devices exhibit some tendency regarding the measurement error.
As seen in \autoref{fig:DistMeasMeanNucPixel}~\subref{fig:DistMeasMeanNucPixel:a} the Pixel3a tends to underestimate the distance.
Only at \SI{6}{m} and \SI{10}{m} the estimated distance is slightly larger compared to the true distance.
The overall
\begin{figure}[ht] \begin{figure}[ht]
\begin{minipage}{.5\textwidth} \begin{minipage}{.5\textwidth}
\centering \centering
\subfloat[]{\label{main:a}\includegraphics[width=\textwidth]{DistMeasMean.png}} \subfloat[]{\label{fig:DistMeasMeanNucPixel:a}\includegraphics[width=\textwidth]{DistMeasMeanPerPixel.png}}
\end{minipage}% \end{minipage}%
\begin{minipage}{.5\textwidth} \begin{minipage}{.5\textwidth}
\centering \centering
\subfloat[]{\label{main:b}\includegraphics[width=\textwidth]{DistMeasMeanPerPixel.png}} \subfloat[]{\label{fig:DistMeasMeanNucPixel:b}\includegraphics[width=\textwidth]{DistMeasMean.png}}
\end{minipage}\par\medskip \end{minipage}\par\medskip
\centering \centering
\subfloat[CDF of error]{\label{main:c}\includegraphics[width=\textwidth]{DistMeasCDF.png}} \subfloat[CDF of error]{\label{main:c}\includegraphics[width=\textwidth]{DistMeasCDF.png}}
\caption{my fig} \caption{my fig}
\label{fig:main} \label{fig:DistMeasMeanNucPixel}
\end{figure} \end{figure}
% Pixel values
\begin{table}[ht]
\centering
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{@{}RRRRcRRR@{}}
\toprule
\theadbf{GT dist in m} & \multicolumn{3}{c}{\bfseries Pixel 2 XL} & \phantom{a} & \multicolumn{3}{c}{\bfseries Pixel 3a} \\ \cmidrule{2-4} \cmidrule{6-8}
~ & \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} && \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} \\ \midrule
2 & 2.472 & 1.165 & 0.472 && 0.032 & 1.615 & -1.968 \\
4 & 3.976 & 1.755 & -0.024 && 2.645 & 1.834 & -1.355 \\
6 & 6.605 & 1.978 & 0.605 && 6.577 & 1.780 & 0.577 \\
8 & 8.780 & 1.510 & 0.780 && 6.861 & 1.506 & -1.139 \\
10 & 11.520 & 1.522 & 1.520 && 10.454 & 2.176 & 0.454 \\
12 & 12.096 & 1.582 & 0.096 && 10.342 & 1.875 & -1.658 \\
14 & 14.327 & 1.870 & 0.327 && 12.866 & 2.309 & -1.134 \\
16 & 17.765 & 1.488 & 1.765 && 15.992 & 0.879 & -0.008 \\
18 & 22.569 & 1.458 & 4.569 && 18.962 & 1.491 & 0.962 \\
20 & 24.058 & 1.820 & 4.058 && 20.742 & 1.318 & 0.742 \\
\bottomrule
\end{tabular}
\caption{TODO}
\label{tab:distvaluesPixels}
\end{table}
% NUC values
\begin{table}[ht]
\centering
\renewcommand{\arraystretch}{1.2}
\begin{tabular}{@{}RRRRcRRR@{}}
\toprule
\theadbf{GT dist in m} & \multicolumn{3}{c}{\bfseries Intel AC 8260} & \phantom{a} & \multicolumn{3}{c}{\bfseries Intel AC 9460} \\ \cmidrule{2-4} \cmidrule{6-8}
~ & \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} && \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} \\ \midrule
2 & 0.221 & 1.460 & -1.779 && 2.282 & 1.724 & 0.282 \\
4 & 2.175 & 0.541 & -1.825 && 4.446 & 1.973 & 0.446 \\
6 & 5.599 & 0.876 & -0.401 && 7.583 & 1.922 & 1.583 \\
8 & 7.041 & 1.562 & -0.959 && 8.601 & 1.715 & 0.601 \\
10 & 9.916 & 1.787 & -0.084 && 12.058 & 1.247 & 2.058 \\
12 & 10.055 & 1.478 & -1.945 && 12.384 & 1.506 & 0.384 \\
14 & 12.312 & 2.110 & -1.688 && 14.882 & 1.182 & 0.882 \\
16 & 16.223 & 1.005 & 0.223 && 17.534 & 1.710 & 1.534 \\
18 & 20.341 & 2.417 & 2.341 && 21.190 & 2.589 & 3.190 \\
20 & 22.001 & 2.920 & 2.001 && 22.799 & 1.853 & 2.799 \\
\bottomrule
\end{tabular}
\caption{TODO}
\label{tab:distvaluesNUCs}
\end{table}
% Smartphones und Intel cards in einer table
%\begin{table}[ht]
%\renewcommand{\arraystretch}{1.2}
%\begin{tabular}{@{}RRRRcRRRcRRRRcRRR@{}}
%\toprule
%\theadbf{GT} & \multicolumn{3}{c}{\bfseries Pixel 2 XL} & \phantom{a} & \multicolumn{3}{c}{\bfseries Pixel 3a} & \phantom{a} & \multicolumn{3}{c}{\bfseries AC 8260} & \phantom{a} & \multicolumn{3}{c}{\bfseries AC 9462} \\ \cmidrule{2-4} \cmidrule{6-8} \cmidrule{10-12} \cmidrule{14-16}
%~ & \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} && \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} && \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} && \thead{$\bar{d}$} & \thead{$\sigma$} & \thead{error} \\ \midrule
%2 & 2.472 & 1.165 & 0.472 && 0.032 & 1.615 & -1.968 && 0.221 & 1.460 & -1.779 && 2.282 & 1.724 & 0.282 \\
%4 & 3.976 & 1.755 & -0.024 && 2.645 & 1.834 & -1.355 && 2.175 & 0.541 & -1.825 && 4.446 & 1.973 & 0.446 \\
%6 & 6.605 & 1.978 & 0.605 && 6.577 & 1.780 & 0.577 && 5.599 & 0.876 & -0.401 && 7.583 & 1.922 & 1.583 \\
%8 & 8.780 & 1.510 & 0.780 && 6.861 & 1.506 & -1.139 && 7.041 & 1.562 & -0.959 && 8.601 & 1.715 & 0.601 \\
%10 & 11.520 & 1.522 & 1.520 && 10.454 & 2.176 & 0.454 && 9.916 & 1.787 & -0.084 && 12.058 & 1.247 & 2.058 \\
%12 & 12.096 & 1.582 & 0.096 && 10.342 & 1.875 & -1.658 && 10.055 & 1.478 & -1.945 && 12.384 & 1.506 & 0.384 \\
%14 & 14.327 & 1.870 & 0.327 && 12.866 & 2.309 & -1.134 && 12.312 & 2.110 & -1.688 && 14.882 & 1.182 & 0.882 \\
%16 & 17.765 & 1.488 & 1.765 && 15.992 & 0.879 & -0.008 && 16.223 & 1.005 & 0.223 && 17.534 & 1.710 & 1.534 \\
%18 & 22.569 & 1.458 & 4.569 && 18.962 & 1.491 & 0.962 && 20.341 & 2.417 & 2.341 && 21.190 & 2.589 & 3.190 \\
%20 & 24.058 & 1.820 & 4.058 && 20.742 & 1.318 & 0.742 && 22.001 & 2.920 & 2.001 && 22.799 & 1.853 & 2.799 \\
%\bottomrule
%\end{tabular}
%\caption{TODO}
%\label{tab:distvalues}
%\end{table}
\subsection{Range Measurements in NLOS Scenario} \subsection{Range Measurements in NLOS Scenario}
Während der posionierung ist an gewissen stellen eine systematische abweichung zum gt aufgefallen dies trat immer an zwei stellen auf, daraus folgender die hypthese das es irgendwie an dieser gegebenheit liegen muss. als folge daraus ist dann eben entstanden das wir die brandschutztüren gesehen haben und dann eine experiment aufgebaut haben um die hypthese zu messen, inwiefern sich die türen oder besser gesagt wie sich unterschiedliche wandmateriellien, insbesondere wenn diese so extrem wie hier sind, auf die FTM Messungen auswirken. Während der posionierung ist an gewissen stellen eine systematische abweichung zum gt aufgefallen dies trat immer an zwei stellen auf, daraus folgender die hypthese das es irgendwie an dieser gegebenheit liegen muss. als folge daraus ist dann eben entstanden das wir die brandschutztüren gesehen haben und dann eine experiment aufgebaut haben um die hypthese zu messen, inwiefern sich die türen oder besser gesagt wie sich unterschiedliche wandmateriellien, insbesondere wenn diese so extrem wie hier sind, auf die FTM Messungen auswirken.
@@ -149,6 +235,10 @@ rssi grafik lassen wir weg (bst2rssi.png) und schreiben das nur im text. müssen
\item wie sind wir gelaufen / testbedingungen... wie wurde GT gemessen usw. app etc pp. \item wie sind wir gelaufen / testbedingungen... wie wurde GT gemessen usw. app etc pp.
\end{itemize} \end{itemize}
All experiments were done in our university building.
The before mentioned Intel NUCs were deployed, because the existing WiFi infrastructure does not support the FTM protocol.
While the positions of the access points where chosen with localization in mind, the actual positions mimic the placement of access points for network infrastructure.
In contrast to regular stationary access points which are usually mounted on walls or ceilings, our Intel NUCs were placed on tables for practical reasons.
\subsection{Results for Multilateration} \subsection{Results for Multilateration}
zunächst wird das einfachste und nahliegendste verfahren untersucht um die performance von ftm und rssi gegenüberzustellen. zunächst wird das einfachste und nahliegendste verfahren untersucht um die performance von ftm und rssi gegenüberzustellen.

6
tex/misc/keywords.tex
View File

@@ -5,6 +5,12 @@
\newcommand{\figref}[1]{fig.~\ref{#1}} \newcommand{\figref}[1]{fig.~\ref{#1}}
\newcommand{\etal} [1]{#1~et~al.\@\xspace} \newcommand{\etal} [1]{#1~et~al.\@\xspace}
% Table helpers
\newcommand*{\thead}[1]{\multicolumn{1}{c}{#1}}
\newcommand*{\theadbf}[1]{\multicolumn{1}{c}{\bfseries #1}}
\newcolumntype{L}{>{$}l<{$}} % math-mode version of "l" column type
\newcolumntype{R}{>{$}r<{$}} % math-mode version of "r" column type
\DeclareSIUnit{\belmilliwatt}{Bm} \DeclareSIUnit{\belmilliwatt}{Bm}
\DeclareSIUnit{\dBm}{\deci\belmilliwatt} \DeclareSIUnit{\dBm}{\deci\belmilliwatt}