403 lines
20 KiB
TeX
403 lines
20 KiB
TeX
\section{Experiments}
|
|
|
|
\todo{
|
|
alles im FHWS gebäude [korrekte groesse fuer beide gebaeude!] mit nem nexus 6
|
|
}
|
|
|
|
Within all \docWIFI{} observations (offline and online) we only consider the \docAP{}s that are permanently installed
|
|
within the building. Temporal and movable transmitters are ignored as they might cause estimation errors.
|
|
|
|
|
|
|
|
modell direkt fuer den gelaufenen pfad optimiert (also wirklich jede wifi messung direkt auf den ground-truth)
|
|
der fehler wird zwar kleiner, ist aber immernoch deutlich spürbar. das spricht dafür, dass das modell einfach nicht
|
|
gut geeignet ist.
|
|
|
|
optimierungs input: alle 4 walks samt ground-truth
|
|
dann kommt fuer die 4 typen [fixed, all same par, each par, each par pos]
|
|
log probability 50 75, meter 50, 75
|
|
|
|
|
|
|
|
% -------------------------------- optimization -------------------------------- %
|
|
|
|
\subsection{Model optimization}
|
|
|
|
As the signal strength prediction model is the heart of the absolute positioning component
|
|
described in \ref{sec:system} we start with the model parameter estimation (see \ref{sec:optimization}) for
|
|
\mTXP, \mPLE and \mWAF based on some reference measurements and compare the results
|
|
between various optimization strategies and a basic empiric choice of \mTXP = \SI{-40}{\decibel{}m} @ \SI{1}{\meter}
|
|
(defined by the usual \docAPshort{} transmit power for europe), a path loss exponent $\mPLE \approx $ \SI{2.5} and
|
|
$\mWAF \approx$ \SI{-8}{\decibel} per floor/ceiling (made of reinforced concrete) \todo{cite für werte}.
|
|
|
|
Figure \ref{fig:referenceMeasurements} depicts the location of the used 121 reference measurements.
|
|
Each location was scanned 30 times ($\approx$ \SI{25}{\second} scan time),
|
|
non permanent \docAP{}s were removed, the values were grouped per physical transmitter (see \ref{sec:vap})
|
|
and aggregated to form the average signal strength per transmitter.
|
|
|
|
% used reference measurements
|
|
\begin{figure}
|
|
{
|
|
\centering
|
|
\input{gfx/all_fingerprints.tex}
|
|
}
|
|
\label{fig:referenceMeasurements}
|
|
\caption{
|
|
Locations of the 121 reference measurements.
|
|
The size of each square denotes the number of permanently installed \docAPshort{}s
|
|
that are visible at this location,
|
|
and ranges between 2 and 22 with an average of 9.
|
|
}
|
|
\end{figure}
|
|
|
|
% visible APs:
|
|
% cnt(121) min(2.000000) max(22.000000) range(20.000000) med(8.000000) avg(9.322314) stdDev(4.386709)
|
|
|
|
\begin{figure}[b]
|
|
\centering
|
|
\input{gfx/compare-wifi-in-out.tex}
|
|
\label{fig:wifiIndoorOutdoor}
|
|
\caption{
|
|
Measurable signal strengths of a testing \docAPshort{} (black dot).
|
|
While the signal diminishes slowly along the corridor (upper rectangle)
|
|
the metallised windows (dashed outline) attenuate the signal by over \SI{30}{\decibel} (lower rectangle).
|
|
}
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:wifiIndoorOutdoor} depicts the to-be-expected issues by examining the signal strength
|
|
values of the reference measurements for one \docAP{}.
|
|
Even though the transmitter is only \SI{5}{\meter} away from the reference
|
|
measurement (small box), the metallised windows attenuate the signal as much as \SI{50}{\meter}
|
|
of corridor (wide box). The model described in section \ref{sec:sigStrengthModel} will not be able
|
|
to match such situations, due to the lack of obstacle information.
|
|
%
|
|
We will thus look at various optimization strategies and the error between
|
|
the resulting estimation model and our reference measurements:
|
|
|
|
{\em\noOptEmpiric{}} uses the same three empiric parameters \mTXP{}, \mPLE{}, \mWAF{} for each \docAPshort{} in combination
|
|
with its position, which is well known from the floorplan.
|
|
|
|
{\em\optParamsAllAP{}} is the same as above, except that the three parameters are optimized
|
|
using the reference measurements.
|
|
|
|
{\em\optParamsEachAP{}} optimizes the three parameters per \docAP{} instead of using the same
|
|
parameters for all.
|
|
|
|
{\em\optParamsPosEachAP{}} does not need any prior knowledge and will optimize all six parameters
|
|
(3D position, \mTXP, \mPLE, \mWAF) based on the reference measurements.
|
|
|
|
{\em\optPerFloor{}} and {\em\optPerRegion{}} are just like \optParamsPosEachAP{} except that
|
|
there are several sub-models that are optimized for one floor / region instead of the whole building.
|
|
|
|
Figure \ref{fig:wifiModelError} shows the optimization results for all strategies, which are as expected:
|
|
The estimation error is indirectly proportional to the number of optimized parameters.
|
|
However, even with {\em \optPerRegion{}} the maximal error is relatively high due to some locations that do
|
|
not fit the model at all. Looking at the optimization results for \mTXP{}, \mPLE{} and \mWAF{} supports
|
|
this finding. While the median for those values based on all optimized transmitters is totally sane
|
|
(\SI{-42}{\decibel{}m}, \SI{2.4}, \SI{-6.0}{\decibel}), the minimum and maximum values are clearly outside of the physically possible range.
|
|
|
|
The same holds for the estimated transmitter position when using {\em \optParamsPosEachAP{}}: The median
|
|
distance between estimated and real position is $\sim$\SI{8}{\meter} and the maximum $\sim$\SI{27}{\meter}.
|
|
For \SI{68}{\percent} of all installed transmitters, the estimated floor-number matched the real location.
|
|
|
|
\begin{figure}
|
|
\input{gfx/wifi_model_error_0_95.tex}
|
|
%\input{gfx/wifi_model_error_95_100.tex}
|
|
\label{fig:wifiModelError}
|
|
\caption{
|
|
Comparison between different optimization strategies by examining the error (in \decibel) at each reference measurement.
|
|
The higher the number of variable parameters, the better the model resembles real world conditions.
|
|
}
|
|
\end{figure}
|
|
|
|
% statds:
|
|
%TXP: cnt(34) min(-67.698959) max(4.299183) range(71.998146) med(-41.961170) avg(-41.659286) stdDev(17.742294)
|
|
%EXP: cnt(34) min(0.932817) max(4.699000) range(3.766183) med(2.380410) avg(2.546959) stdDev(1.074687)
|
|
%WAF: cnt(34) min(-27.764957) max(5.217187) range(32.982143) med(-5.921916) avg(-7.579522) stdDev(5.840527)
|
|
%Pos: cnt(34) min(3.032438) max(26.767128) range(23.734690) med(7.342710) avg(8.571227) stdDev(4.801449)
|
|
|
|
While {\em \optPerRegion{}} is able to overcome the indoor vs. outdoor issues depicted in
|
|
figure \ref{fig:wifiIndoorOutdoor} e.g. by using a separate bounding box just for the outdoor area,
|
|
it obviously requires a profound prior knowledge when selecting the individual regions for the sub-model.
|
|
%Such issues can only be fixed using more appropriate models that consider walls and other obstacles.
|
|
|
|
% das ist wohl zu viel
|
|
%\begin{figure}
|
|
% \centering
|
|
% \input{gfx/wifiOptApPosDifference.tex}
|
|
% \caption{zu viel, oder?}
|
|
%\end{figure}
|
|
|
|
|
|
% -------------------------------- number of fingerprints -------------------------------- %
|
|
|
|
As we try to minimize the system's setup time as much as possible, we need to determine
|
|
the amount of necessary reference measurements for the optimization to produce viable model parameters.
|
|
Depending on the number of to-be-optimized model parameters, more measurements are required.
|
|
This especially holds true for {\em \optPerRegion{}} where each region needs at least some measurements
|
|
to determine transmitter positions and parameters.
|
|
|
|
\begin{figure}[b]
|
|
\input{gfx/wifi_model_error_num_fingerprints_method_5_0_90.tex}
|
|
\input{gfx/wifi_model_error_num_fingerprints_method_5_90_100.tex}
|
|
\label{fig:wifiNumFingerprints}%
|
|
\caption{%
|
|
number of fingerprints
|
|
}%
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:wifiNumFingerprints} depicts the impact of reducing the number of fingerprints
|
|
for the {\em \optPerRegion{}} strategy. Only using 60 of the 121 fingerprints yields only a slightly
|
|
increasing model error and still provides good results. While using only \SI{25}{\percent} of the reference
|
|
measurements increases the error rapidly, \SI{75}{\percent} of all considered errors are still better
|
|
than using just empiric values without any reference measurements.
|
|
|
|
Additionally we examined the impact of skipping reference measurements for difficult locations
|
|
like aforementioned staircases, surrounded by concrete. While this slightly decreases the
|
|
estimation error for all other positions, the error within those locations is dramatically
|
|
increasing (see lower half of figure \ref{fig:wifiNumFingerprints}). It is thus highly recommended
|
|
to include such locations.
|
|
|
|
|
|
|
|
|
|
% -------------------------------- wifi walk error -------------------------------- %
|
|
|
|
\subsection{Location estimation error}
|
|
|
|
Using the optimized model setups and the measurements $\mRssiVec$ determined by scanning for nearby \docAPshort{}s,
|
|
we can directly perform a location estimation by rewriting \refeq{eq:wifiProb}:
|
|
|
|
\begin{equation}
|
|
p(\mPosVec \mid \mRssiVec) =
|
|
\frac{p(\mRssiVec \mid \mPosVec) p(\mPosVec)}{p(\mRssiVec)}
|
|
\approx p(\mRssiVec \mid \mPosVec),\enskip
|
|
p(\mPosVec) = p(\mRssiVec) = \text{const}
|
|
.
|
|
\label{eq:wifiBayes}
|
|
\end{equation}
|
|
|
|
The pedestrian's current location $\mPosVec^*$ given $\mRssiVec$ satisfies
|
|
|
|
\begin{equation}
|
|
\mPosVec^* = \argmax_{\mPosVec}
|
|
p(\mRssiVec \mid \mPosVec)
|
|
.
|
|
\label{eq:bestWiFiPos}
|
|
\end{equation}
|
|
|
|
The quality of the estimated location is determined by comparing the estimation
|
|
$\mPosVec^*$ with the pedestrian's ground truth at the time the scan $\mRssiVec$
|
|
has been received.
|
|
|
|
We therefore conducted 10 walks on 5 different paths within our building,
|
|
each of which is defined by connecting several marker points at well known positions
|
|
(see figure \ref{fig:allWalks}).
|
|
Whenever the pedestrian reached such a marker, the current time was recorded.
|
|
Due to constant walking speeds, the ground-truth for any timestamp can be approximated
|
|
using linear interpolation between adjacent markers.
|
|
|
|
% walked paths
|
|
\begin{figure}[t]
|
|
{
|
|
\centering
|
|
\input{gfx/all_walks.tex}
|
|
}
|
|
\label{fig:allWalks}
|
|
\caption{
|
|
Overview of all conducted paths.
|
|
Outdoor areas are marked in green.
|
|
}
|
|
\end{figure}
|
|
|
|
To estimate the performance of the prediction models, we compare the position estimation
|
|
for each \docWIFI{} measurement within the recorded paths (3756 \docAPshort{} scans in total)
|
|
against the corresponding ground-truth, which indicates the absolute 3D error in meter.
|
|
|
|
\begin{figure}[b]
|
|
\input{gfx/modelPerformance_meter.tex}
|
|
\label{fig:modelPerformance}
|
|
\caption{
|
|
Error between ground truth and estimation using \refeq{eq:bestWiFiPos} depending
|
|
on the underlying signal strength prediction model
|
|
}
|
|
\end{figure}
|
|
|
|
As can be seen in figure \ref{fig:modelPerformance}, the quality of the location estimation
|
|
directly scales with the quality of the signal strength prediction model.
|
|
However, depending on the model, the maximal estimation error might increase (see \optParamsPosEachAP{}).
|
|
%
|
|
This is either due to multimodalities, where more than one area is possible based on the recent
|
|
\docWIFI{} observation, or optimization yields an overadaption where the average signal
|
|
strength prediction error is small, but the maximum error is dramatically increased for some regions.
|
|
|
|
|
|
|
|
% -------------------------------- plots indicating walk issues -------------------------------- %
|
|
|
|
\begin{figure}[t]
|
|
\input{gfx/wifiMultimodality.tex}
|
|
\label{fig:wifiMultimodality}
|
|
\caption{
|
|
Location probability \refeq{eq:bestWiFiPos} for three scans. Higher color intensities are more likely.
|
|
Ideally, places near the ground truth (black) are highly highly probable (green).
|
|
Often, other locations are just as likely as the ground truth (blue),
|
|
or the location with the highest probability does not match at all (red).
|
|
}
|
|
\end{figure}
|
|
|
|
Figure \ref{fig:wifiMultimodality} depicts aforementioned issues of multimodal (blue) or wrong (red) location
|
|
estimations. Filtering (\refeq{eq:recursiveDensity}) thus is highly recommended as minor errors are compensated
|
|
using other sensors and/or a movement model that prevents the estimation from leaping within the building.
|
|
However, if wrong sensor values (red) are observed for longer time periods, even filtering will produce erroneous
|
|
results and might get stranded (density is trapped e.g. within a room),
|
|
as the movement model is constrained by the actual floorplan.
|
|
|
|
|
|
% -------------------------------- other distributions, unseen APs, etc -------------------------------- %
|
|
|
|
To reduce the amount such of misclassifications, where other locations within the building are
|
|
as likely as the pedestrians actual location, we examined various approaches.
|
|
Unfortunately, none of which provided a viable enhancement under all conditions for the performed walks.
|
|
|
|
The misclassification-rate is determined by counting the amount of (random) locations within
|
|
the building that produce a similar probability \refeq{eq:wifiProb} as the actual ground-truth
|
|
position.
|
|
|
|
One possibility to dissolve such an equal \docWIFI{}-likelihood between two (or more) locations is,
|
|
to not only consider the \docAPshort{}s seen by the Smartphone, but also the \docAPshort{}s not seen
|
|
by the Smartphone. This additional information can be used to rule out all locations where this
|
|
\docAP{} should be received (high signal strength from the prediction model).
|
|
% There might be an \docAP{} that should be visible at the other locations. However,
|
|
%as the Smartphone did not see this \docAPshort{} the other location can be ruled out.
|
|
While this works in theory, evaluations revealed several issues:
|
|
|
|
There is a chance that even a nearby \docAPshort{} is unseen during a scan due to packet collisions or
|
|
temporal effects within the surrounding. It thus might make sense to opt-out other locations
|
|
only, if at least two \docAPshort{}s are missing. On the other hand, this obviously requires (at least)
|
|
two \docAPshort{}s to actually be different between the two locations, and requires a lot of permanently
|
|
installed transmitters to work out.
|
|
|
|
Furthermore, this requires the signal strength prediction model to be fairly accurate. Within our testing
|
|
walks, several places are surrounded by concrete walls, which cause a harsh, local drop in signal strength.
|
|
The models used within this work will not accurately predict the signal strength for such locations.
|
|
%%Including \docAPshort{}s unseen by the Smartphone thus often increases the estimation error instead
|
|
%%of fixing the multimodality.
|
|
|
|
To sum up, while some situations, e.g. outdoors, could greatly be improved,
|
|
many other situations are deteriorated, especially when some transmitters are (temporarily)
|
|
attenuated by ambient conditions like concrete walls.
|
|
|
|
|
|
|
|
We therefore examined variations of the probability calculation from \refeq{eq:wifiProb}.
|
|
Removing the strongest/weakest \docAPshort{} from $\mRssiVec{}$ yielded similar results.
|
|
While some estimations were improved, the overall estimation error increased for our walks,
|
|
as there are many situations where only a handful \docAP{}s can be seen. Removing (valid)
|
|
information will highly increase the error for such situations.
|
|
|
|
Using a more strict exponential distribution for the model vs. scan comparison in \refeq{eq:wifiProb}
|
|
had a positive effect on the misclassification error for some of the walks, but slightly increased
|
|
the estimation error (see figure \ref{fig:normalVsExponential}) and thus produced negative side effects.
|
|
|
|
\begin{figure}
|
|
\input{gfx/wifiCompare_normalVsExp_cross.tex}
|
|
\input{gfx/wifiCompare_normalVsExp_meter.tex}
|
|
\label{fig:normalVsExponential}
|
|
\caption{
|
|
Comparison between normal- (black) and exponential-distribution (red) for \refeq{eq:wifiProb}.
|
|
While misclassifications are slightly reduced (upper chart),
|
|
the median error between ground-truth and estimation (lower chart) increases by
|
|
about \SI{1}{\meter}.
|
|
}
|
|
\end{figure}
|
|
|
|
|
|
|
|
|
|
\todo{
|
|
erwähnen??? sigma je nach signalstärke anpassen bringt leider auch nichts. wenn man das aber macht,
|
|
dann: fuer grosse signalstaerken ein grosses sigma! andersrum gehts nach hinten los!
|
|
}
|
|
|
|
|
|
|
|
|
|
% -------------------------------- final system -------------------------------- %
|
|
|
|
% REAL WALKS
|
|
\todo{obwohl das angepasste modell doch recht gut laeuft und der fehler recht klein wird, sind immernoch stellen dabei,
|
|
wo es einfach nicht gut passt, unguenstige mehrdeutigkeiten vorliegen, oder regionen einfach nicht passen wie sie sollten.
|
|
das liegt teils auch daran, dass die fingerprints drehend aufgenommen wurden und beim laufen nach hinten durch den
|
|
menschen abgeschottet wird. auch zeitlicher verzug kann ein problem darstellen.}
|
|
|
|
\todo{GPS ist leider kaum eine hilfe. entweder kein empfang wegen ueberdachung oder abschattung, oder
|
|
zu kurz draußen um einen guten gps-fix zu bekommen.}
|
|
|
|
\todo{
|
|
walk1 hat eine issue kurz bevor man zur tuer zum hoersaalgebaude reingeht
|
|
je nach resampling killt dieser wlan error evtl alle partikel!
|
|
}
|
|
|
|
\todo{
|
|
das bbox modell hat probleme an den uebergängen zwischen bboxes da dort teils starke spruenge sind
|
|
die nicht immer in der realität so auch vorliegen. z.B. z-wechsel machen teils probleme.
|
|
hier wäre ein kontinuierliches modell hilfreich bzw interpolation in randbereichen
|
|
}
|
|
|
|
\todo{
|
|
wenn ich beim fingerprinten einen AP an einer stelle NICHT gesehen habe,
|
|
ist das auch eine aussage für die model optimierung.. da kann dann sicher keine signatlstaerke > -90 an der stelle raus kommen
|
|
}
|
|
|
|
\todo{gps wird so schnell nicht warm, versagt denn auf dem hof als hilfestellung}
|
|
|
|
|
|
|
|
|
|
|
|
ware das grid-model nicht da, wuerde der outdoor teil richtig schlecht laufen,
|
|
weil das wlan hier absolut ungenau ist.. da die partikel aber aufgrund des vorherigen
|
|
walks schon recht dicht beisamen sind, kittet das das ganze sehr gut.
|
|
kann man testen, indem man z.B. weniger resampling macht und mehr alte partikel aufhebt.
|
|
geht sofort kaputt sobald man aus dem gebäude raus kommt
|
|
|
|
signalstaerke limitieren, wie : alles was im model oder scan < -90 ist, wird auf -90 abgeschnitten hilft
|
|
zwar an manchen stellen, im groben und ganzen führt es aber eher zu fehlern als zu verbesserungen.
|
|
zudem ist zu erwarten, dass diese zahl stark vom geraet/hardware abhaengt
|
|
|
|
jeweils beim weighting die niedrigste wifi probability weglassen [je nach particle also ein anderer AP]
|
|
bringt auch nicht immer was.. killt gelegentlich floor-changes. zudem stehen am ende nur sehr wenige
|
|
APs zur verfügung. da einen zu ignorieren, macht noch mehr kaputt
|
|
|
|
auch ein versuch wie werfe alle APs aus dem handy-scan weg, die kleiner -90 sind, birgt die selben risiken
|
|
es scheint wirklich am sinnvollsten, die scan-daten einfach 1:1 zu nehmen wie sie sind
|
|
|
|
|
|
kurz vor ende von path 2 will die estimation nicht in die cafeteria, weil ein paar particle
|
|
die treppe richtung h.1.5 hochgehen und durch das wlan sehr sehr hoch gewichtet werden.
|
|
die mittelwert-estimation versagt hier
|
|
|
|
|
|
\input{gfx/wifi-opt-error-hist-methods.tex}
|
|
|
|
\input{gfx/wifi-opt-error-hist-stair-outdoor.tex}
|
|
outdoor hat insgesamt nicht all zu viel einfluss, da die meisten APs
|
|
an den outdoor punkten kaum gesehen werden. auf einzelne APs kann
|
|
der einfluss jedoch recht groß sein, siehe den fingerprint plot von
|
|
dem einen ausgewählten AP
|
|
|
|
wenn noch zeit ist: wie aendert sich die model prediction wenn man z.B. nur die haelfte der referenzmessungen nimmt?
|
|
|
|
|
|
\todo{anfaenglich falsches heading ist gift, wegen rel. heading, weil sich dann alles verlaeuft. fix: anfaenglich große heading variation erlauben}
|
|
|
|
\todo{NICHT MEHR AKTUELL: abs-head ist in der observation besser, weil es beim resampling mehr bringt und dafuer srogt, dass die richtigen geloescht werden!}
|
|
|
|
\todo{ deutlich machen
|
|
wenn man nur die fingerprints des floors nimmt in dem gelaufen wird, ist alles gut
|
|
sobald man andere floors drueber/drunter dazu nimmt, ist es nicht mehr gnaz so gut, oder wird schlechter
|
|
das spricht dafuer dass das modell nicht gut passt
|
|
koennte man zeigen indem man den durchschnittlichen fehler je fingerprint plottet???
|
|
}
|