text:
\section*{References}}

\usepackage{fancyhdr}
\fancyhf{}
\renewcommand{\headrulewidth}{0pt}
\rfoot{\thepage}
\pagestyle{fancy}

\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}

\newenvironment{proof}[1][Proof]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{definition}[1][Definition]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{example}[1][Example]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{remark}[1][Remark]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{prop}[1][Proposition]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}

\newcommand{\specialcell}[2][c]{%
  \begin{tabular}[#1]{@{}c@{}}#2\end{tabular}}

\begin{document}

\begin{frontmatter}
\title{Feature Selection for Regression Problems Based on the Morisita Estimator of Intrinsic Dimension}
\author{Jean GOLAY, Michael LEUENBERGER and Mikhail KANEVSKI}
\address{Institute of Earth Surface Dynamics, Faculty of Geosciences and Environment, University of Lausanne, 1015 Lausanne, Switzerland. Email: jean.golay@unil.ch.}

\begin{abstract}
Data acquisition, storage and management have been improved, while the key factors of many phenomena are not well known. Consequently, irrelevant and redundant features artificially increase the size of datasets, which complicates learning tasks, such as regression. To address this problem, feature selection methods have been proposed. This paper introduces a new supervised filter based on the Morisita estimator of intrinsic dimension. It can identify relevant features and distinguish between redundant and irrelevant information. Besides, it offers a clear graphical representation of the results, and it can be easily implemented in different programming languages. Comprehensive numerical experiments are conducted using simulated datasets characterized by different levels of complexity, sample size and noise. The suggested algorithm is also successfully tested on a selection of real world applications and compared with RReliefF using extreme learning machine. In addition, a new measure of feature relevance is presented and discussed.
\end{abstract}

\begin{keyword}
Feature selection \sep Intrinsic dimension \sep Morisita index \sep Measure of relevance \sep Data mining
\end{keyword}
\end{frontmatter}

\section{Introduction}
In data mining, it is often not known a priori what features (or input variables \footnote{In this paper, the term ``feature'' is used as a synonym for ``input variable''.}) are truly necessary to capture the main characteristics of a studied phenomenon. This lack of knowledge implies that many of the considered features are irrelevant or redundant. They artificially increase the dimension $E$ of the Euclidean space $\mathbb{R}^E$ in which the data points are embedded ($E$ equals the number of input and output variables under consideration). This is a serious matter, since fast improvements in data acquisition, storage and management cause the number of redundant and irrelevant features to increase. As a consequence, the interpretation of the results becomes more complicated and, unless the sample size $N$ grows exponentially with $E$, the curse of dimensionality \cite{Bell61} may reduce the overall accuracy yielded by any learning algorithm. Besides, large $N$ and $E$ are also difficult to deal with because of computer performance limitations.

In regression and classification, these issues are often addressed by implementing supervised feature selection methods \cite{Guy03,Guy06,Ghe10,Zen15}. Such methods can be broadly subdivided into filter (e.g. RReliefF \cite{Robnik03}, mRMR \cite{Pen05} and CFS \cite{Hall00}), wrapper \cite{Koh97,Leu14} and embedded methods (e.g. the Lasso \cite{Tib96} and random forest \cite{Bre01}). Filters rank features, or subsets of features, according to a relevance measure independently of any predictive model, while wrappers use an evaluation criterion involving a learning machine. Both approaches can be used with search strategies, since an exhaustive exploration of the $2^{\# Feat.}-1$ models (all the possible combinations of features) is often computationally intractable. Greedy strategies \cite{Cot01,Col03}, such as Sequential Forward Selection (SFS) \cite{Whit71}, can be distinguished from stochastic ones (e.g. simulated annealing \cite{Mei06,Kirk83} and ant colony optimization \cite{Ta15,Do92}). Regarding the embedded methods, the feature selection is a by-product of a training procedure. It can be achieved by the addition of constraints in the cost function of a predictive model (e.g. the Lasso \cite{Tib96}), or it can be more specific to a given algorithm (e.g. random forest \cite{Bre01} and adaptive general regression neural networks \cite{Rob13}).

The present paper\footnote{The main idea of this paper was partly presented at the 23rd symposium on artificial neural networks, computational intelligence and machine learning (ESANN2015) \cite{Go15Esann}.} deals with a new SFS filter algorithm. It relies on the idea that, although data points are embedded in $E$-dimensional spaces, they often reside on lower $M$-dimensional manifolds \cite{Cama03,LeeVer07,Cama16}. The value $M$ ($\leq E$) is called Intrinsic Dimension (ID), and it can be estimated using the Morisita estimator of ID \cite{Go15} which is closely related to the fractal theory. The proposed filter algorithm is supervised, designed for regression problems and based on this new ID estimator. It also keeps the simplicity of the Fractal Dimension Reduction (FDR) algorithm introduced in \cite{Trai00}. Finally, the results show the ability of the new filter to capture non-linear relationships and to effectively identify both redundant and irrelevant information.

The paper is organized as follows. Section \ref{overview} reviews previous work on ID-based feature selection approaches. The Morisita estimator of ID is shortly presented in Section \ref{Mindex} (for the completeness of the paper). Section \ref{MBFR} introduces the Morisita-based filter, and Section \ref{Exp_art} is devoted to numerical experiments conducted on simulated data of varying complexity. In Section \ref{Exp_real}, real world applications from publicly accessible repositories are presented, and a comparison with a benchmark algorithm, RReliefF \cite{Robnik03}, is carried out using Extreme Learning Machine (ELM) \cite{Hua06}. Finally, conclusions are drawn in the last section with a special emphasis on future challenges and applications.

\section{Related Work}\label{overview}
The concept of ID can be extended to the more general case where the data ID may be a non-integer dimension $D$ \cite{Trai00,Ek11,LeeVer07}. The value $D$ is estimated by using fractal-based methods which have been presented in \cite{Cama16,LeeVer07,Go14} and successfully implemented in various fields, such as physics \cite{Grass833}, cosmology \cite{Bor93}, meteorology \cite{lov87} and pattern recognition \cite{Hua94,Xu15}. These methods rely on well-known fractal dimensions (e.g. the box-counting dimension \cite{Mand83,Ott93}, the correlation dimension \cite{Grass833} and R\'{e}nyi's dimensions of $q$th order \cite{Hent83}), and they can be used in feature selection \cite{Trai00,Mo12} and dimensionality reduction \cite{LeeVer07} to detect dependencies between variables (or features).

Traina et al. \cite{Trai00,Trai10} have opened up new prospects for the effective use of ID estimation in data mining by introducing the Fractal Dimension Reduction (FDR) algorithm. FDR executes an unsupervised procedure of feature selection aiming to remove from a dataset all the redundant variables. The fundamental idea is that fully redundant variables do not contribute to the value of the data ID.

This idea can be illustrated by sampling two uniformly distributed variables $V_1$ and $V_2$. If they are independent, which means that they are not redundant, one has that:\begin{equation}\label{Eq_concept}
ID(V_1,V_2)\approx ID(V_1)+ID(V_2)\approx 1+1=2
\end{equation}where $ID(\cdot)$ denotes the ID of a dataset. It indicates that both $V_1$ and $V_2$ contribute to increasing the value of $ID(V_1,V_2)$ by about $1$, which is, by construction, equal to the ID of each variable (i.e. $ID(V_1)$ and $ID(V_2)$). Conversely, the removal of either $V_1$ or $V_2$ would lead to a reduction in the data ID from about 2 (i.e. the dimension of the data space) to $1$ (i.e. the ID of a single variable) and information would be irreparably lost. In contrast, if $V_1$ and $V_2$ are fully redundant with each other (e.g. $V_2 = V_1$), one has that: \begin{equation}
ID(V_1,V_2)\approx ID(V_1)\approx ID(V_2)\approx 1
\end{equation}where the ID of the full dataset is approximately equal to the topological dimension of a smooth line. This means that the contribution of only one variable is enough to reach the value of $ID(V_1,V_2)$ and the remaining one can be disregarded without losing any information.

Based on these considerations, the FDR algorithm removes the redundant variables from a dataset by implementing a Sequential Backward Elimination (SBE) strategy \cite{Cot01}. Besides, it uses R\'{e}nyi's dimension of order $q=2$, $D_2$, for the ID estimation. Following the same principles, De Sousa et al. \cite{Sousa07} examined additional developments to FDR and presented a new algorithm for identifying subgroups of correlated variables.

FDR is designed to carry out unsupervised tasks, and it is not able to distinguish between variables that are relevant to a learning process and those that are irrelevant. The reason is that such variables can all contribute to the data ID. For instance, in Equation \ref{Eq_concept}, $V_1$ could be regarded as irrelevant to the learning of $V_2$, but it would be selected by FDR because it makes the data ID increase by about $1$. Consequently, different studies were carried out to adapt FDR to supervised learning. Lee et al. \cite{Lee06} suggested decoupling the relevance and redundancy analysis. Following the same idea, Pham et al. \cite{Pham09} used mutual information to identify irrelevant features and combined the results with those of FDR. Finally, Mo and Huang \cite{Mo12} developed an advanced algorithm to detect both redundant and irrelevant information in a single step. Their algorithm follows a SBE search strategy and relies on the correlation dimension, $df_{cor}$, for the estimation of the data ID.

The filter algorithm suggested in the present paper is designed in such a way that it combines the advantages of both FDR and Mo's algorithm: it can deal with non-linear dependencies, it does not rely on any user-defined threshold, it can discriminate between redundant and irrelevant information, and the results can be easily summarized in informative plots. Moreover, it can cope with high-dimensional datasets thanks to its SFS search strategy, and it uses the Morisita estimator of ID which was shown to yield comparable or better results than $D_2$ and $df_{cor}$ \cite{Go15}.

\section{The Morisita Estimator of Intrinsic Dimension}\label{Mindex}
The Morisita estimator of ID, $M_m$, has been recently introduced \cite{Go15}. It is a fractal-based ID estimator derived from the multipoint Morisita index $I_{m,\delta}$ \cite{Go14,Hul90} (named after Masaaki Morisita who proposed the first version of the index to study the spatial clustering of ecological data \cite{Mori59}). $I_{m,\delta}$ is computed by superimposing an $E$-dimensional grid of $Q$ quadrats of diagonal size $\delta$ onto the data points. It measures how many times more likely it is that $m$ ($m\geq 2$) randomly selected points will be from the same quadrat than it would be if all the $N$ points of the studied dataset were distributed at random (i.e. according to a random distribution generated from a Poisson process). The formula is the following:\begin{equation}
I_{m,\delta}=Q^{m-1}\frac{\sum_{i=1}^Q n_i(n_i-1)(n_i-2) \dotsm (n_i-m+1)}{N(N-1)(N-2) \dotsm (N-m+1)}
\end{equation}where $n_i$ is the number of points in the $i^{th}$ quadrat. For a fixed value of $m$, $I_{m,\delta}$ is calculated for a chosen scale range. If a dataset approximates a fractal behavior (i.e. is self-similar) within this range, the relationship of the plot relating $\log{(I_{m,\delta})}$ to $\log{(1/\delta)}$ is linear, and the slope of the regression line is defined as the Morisita slope $S_m$. Finally, $M_m$ is expressed as:\begin{equation}\label{Eq_mindid}M_m = E - \left( \frac{S_{m}}{m-1}\right).\end{equation}In practice, each variable is rescaled to the $[0,1]$ interval (so is the grid), and $\delta$ can be replaced with the quadrat edge length $\ell$, with $\ell^{-1}$ being simply the number of quadrats along each axis of the data space. Then a set of $R$ values of $\ell$ (or $\ell^{-1}$) is chosen so that it captures the linear part of the log-log plot. In the rest of this paper, only $M_{m=2}$ will be used, and it will be computed with an algorithm called Morisita INDex for Intrinsic Dimension estimation (MINDID) \cite{Go15} whose complexity is $\mathcal{O}(N*E*R)$.

\section{The Morisita-based Filter for Regression Problems}\label{MBFR}
The Morisita-Based Filter for Regression problems (MBFR) relies on three observations following from the work by Traina et al. \cite{Trai00}, De Sousa et al. \cite{Sousa07} and Mo and Huang \cite{Mo12}:\begin{enumerate}
\item Given an output variable $Y$ generated from $k$ relevant and non-redundant input variables $X_1,\ldots ,X_k$, one has that:
\begin{equation}
ID(X_1,\ldots ,X_k,Y)-ID(X_1,\ldots ,X_k)\approx 0
\end{equation}where $ID(\cdot)$ denotes the (possibly non-integer) ID of a dataset.
\item Given $i$ irrelevant input variables $I_1,\ldots ,I_i$ completely independent of Y, one has that:
\begin{equation}
ID(I_1,\ldots ,I_i,Y)-ID(I_1,\ldots ,I_i)\approx ID(Y)
\end{equation}
\item Given a randomly selected subset of $\left\lbrace X_1,\ldots ,X_k\right\rbrace$ of size $r$ with $1\leq r<k$ and $k\geq 2$, $j$ redundant input variables $J_1,\ldots ,J_j$ related to some or all $X_1,\ldots ,X_r$ and all the $i$ irrelevant input variables $I_1,\ldots ,I_i$, one has that:
\begin{equation}
\begin{aligned}
&ID(X_1,\ldots ,X_r,J_1,\ldots ,J_j,I_1,\ldots ,I_i,Y)\\
-&ID(X_1,\ldots ,X_r,J_1,\ldots ,J_j,I_1,\ldots ,I_i)\approx H
\end{aligned}\end{equation}where $H\in \left]0,ID(Y)\right[$ and $H$ decreases to $0$ as $r$ increases to $k$.
\end{enumerate}

\begin{algorithm}[t]
\caption{MBFR}\label{mbfr_algo}
\textbf{INPUT:}

A dataset $A$ with $E-1$ features $F_{1,\ldots ,E-1}$ and one output variable $Y$.

A vector $L$ of values $\ell^{-1}$.

An integer $C$ ($\leq E-1$) indicating the number of steps of the SFS to be performed.

Two empty vectors of length $C$: $SelF$ and $DissF$ for storing, respectively, the names of the selected features and the dissimilarity values.

An empty matrix $Z$ for storing the selected features.

\textbf{OUTPUT:} $SelF$ and $DissF$.
\begin{algorithmic}[1]
\STATE Rescale each feature and $Y$ to $[0,1]$.
\FOR{$i = 1 \ \TO \ C$}
\FOR{$j = 1 \ \TO \ (E-i)$}
\STATE $\widehat{Diss}(Z,F_j,Y)=M_2(Z,F_j,Y)-M_2(Z,F_j)$ (MINDID used with $L$)
\ENDFOR
\STATE Store in $SelF[i]$ the name of the $F_j$ yielding the lowest value of $\widehat{Diss}$.
\STATE Store this value of $\widehat{Diss}$ in $DissF[i]$.
\STATE Remove the corresponding $F_j$ from $A$ and add it into $Z$.
\ENDFOR\\
\end{algorithmic}
\end{algorithm}

The difference \begin{equation}\label{diss_equ}Diss(F,Y):=ID(F,Y)-ID(F)\end{equation}can thus be suggested as a way of measuring the dissimilarity (i.e the independence) between $Y$ and a set $F$ of features (e.g. $F=\lbrace X_1,X_2,J_1,I_1\rbrace$), among which only the relevant ones (i.e. the non-redundant features on which $Y$ depends) contribute to reducing the dissimilarity. Based on that idea, MBFR (see Algorithm \ref{mbfr_algo}) aims at retrieving the relevant features available in a dataset by sorting each subset of variables according to its dissimilarity with $Y$. MBFR implements a SFS search strategy and relies on the Morisita estimator of ID and the MINDID algorithm \cite{Go15} to estimate $Diss$:\begin{equation}\label{diss_equ_est}\widehat{Diss}(F,Y):=M_2(F,Y)-M_2(F).\end{equation}In terms of time complexity, the algorithm is linear on $N$ and $R$, but its bottleneck is the SFS search strategy which is quadratic on $E$. In spite of this limitation, the execution time of MBFR remains competitive as shown in Section \ref{Exp_real}. It can also be significantly reduced by setting $C$ (i.e. the number of steps of the SFS procedure to be performed) to a small value. For instance, if $Diss$ is likely to reach its minimum value after only a few SFS steps because of many redundant and irrelevant features, $C$ can be set to a value substantially lower than $E-1$.

For ease of comparison, the coefficient of dimensional relevance, $DR(F,Y)$, can be introduced. It is defined as:\begin{equation}\label{Eq_DR}
DR(F,Y):=1-\frac{Diss(F,Y)}{ID(Y)}=1-\frac{ID(F,Y)-ID(F)}{ID(Y)}
\end{equation}and it can be computed using the Morisita estimator of ID $M_2$. In the same way as $Diss(F,Y)$, $DR(F,Y)$ is able to capture both linear and non-linear relationships between an input and an output space. Besides, it lies between $0$ and $1$. If the target (or output) variable $Y$ can be completely explained by the considered features $F$, $DR(F,Y) = 1$. On the contrary, if all the available features are irrelevant, $DR(F,Y) = 0$, and in-between, the closer it is to $1$, the greater the predictive power of $F$.

\begin{figure}
\centering
\includegraphics[width=\linewidth]{Butterfly_visual_light.pdf}
\caption{(left) The functional relationship between the output variable $Y$ and the relevant features $X_1$ and $X_2$ of the butterfly dataset; (right) Shuffling of the output variable $Y$.}\label{Fig_butterVisual}
\end{figure}

\begin{table}[t]
\centering
\footnotesize
\begin{tabular}{cccc}
\hline
$j$& $\omega_{1,j}$ & $\omega_{2,j}$ & $\beta_j$ \\
\hline
\hline
$1$&  $0.6655$ & $0.8939$ & $1.3446$\\
$2$&  $1.2611$ & $-0.3512$ & $-0.0115$\\
$3$&  $0.3961$ & $-1.7827$ & $1.2770$\\
$4$&  $-1.7065$ & $-0.5297$ & $0.5962$\\
$5$&  $0.8807$ & $1.9574$ & $-0.8530$\\
$6$&  $1.8260$ & $0.7962$ & $-0.7290$\\
$7$&  $1.3400$ & $1.5001$ & $1.2339$ \\
$8$&  $1.2919$ & $-0.4462$ & $0.1186$\\
$9$&  $-1.3902$ & $1.6856$ & $0.5277$ \\
$10$& $0.0743$ & $1.5625$ & $-0.6952$ \\
\hline
\end{tabular}
\caption{Weights used in the construction of the butterfly dataset.}
\label{T6}
\end{table}

\begin{figure}[t!]
\centering
\includegraphics[width=\linewidth]{Scatterplot.pdf}
\caption{Scatterplot matrix of the butterfly dataset for one simulation.}\label{Fig_scplot}
\end{figure}

\section{Experimental Study Using Simulated Data}\label{Exp_art}
In this section, the MBFR algorithm is assessed by means of two simulated datasets (see Subsection \ref{sim_data}), and its overall performance is carefully examined through a set of questions around which the subsections are organized: \begin{itemize}[noitemsep]
\item Question 1: How does sample size affect MBFR (see Subsection \ref{samplesize})?
\item Question 2: How does the complexity of data manifolds affect MBFR (see Subsection \ref{manifold}) ?
\item Question 3: How can MBFR help distinguish between redundant and irrelevant information (see Subsection \ref{info})?
\item Question 4: How does MBFR respond to a (partial) lack of relevant information (see Subsection \ref{no_info})?
\item Question 5: How does MBFR respond to the presence of noise in data (see Subsection \ref{noise})?
\end{itemize}
Notice also that the R environment \cite{Rcran} was used to implement the MBFR algorithm and to carry out the experiments.

\subsection{Simulated Datasets}\label{sim_data}
Two simulated datasets were used: the butterfly and Friedman datasets.
\begin{enumerate}
\item The butterfly dataset\footnote{It can be downloaded from: https://sites.google.com/site/jeangolayresearch/.} (see Figure \ref{Fig_butterVisual}): An output variable $Y$ is generated from two uniformly distributed input variables $X_1,X_2 \in \left] -5,5\right[$ by using an Artificial Neural Network (ANN) consisting of one hidden layer of $10$ neurons. It can be expressed as:\begin{equation}\label{equ_butter}
Y = \left[ \sum_{j=1}^{10}\beta_j sig(X_1 \omega_{1,j} + X_2 \omega_{2,j})\right]  + \varepsilon
\end{equation}where $\omega_{1,j}$ and $\omega_{2,j}$ are the weights connecting the input variables to the $j^{th}$ neuron, $sig\left(x \right) : \mathbb{R}\rightarrow \mathbb{R}$ is a sigmoid transfer function, $\beta_j$ is the weight between the $j^{th}$ neuron and the output layer, and $\varepsilon$ is a Gaussian noise with zero mean and varying standard deviation (by default, it is set to $0.00$). The exact weights used in the construction of the dataset are given in Table \ref{T6}. Moreover, the addition of three redundant ($J$) and three irrelevant ($I$) variables is also made to complete the input space: $J_3=log_{10}{(X_1+5)}$, $J_4 = X_1^2-X_2^2$, $J_5 = X_1^4-X_2^4$, a uniformly distributed variable $I_6\in \left]-5,5\right[$, $I_7=log_{10}{(I_6+5)}$ and $I_{8}=I_6+I_7$. Finally, the butterfly dataset is generated by random sampling of $X_1$, $X_2$ and $I_6$. In this paper, different sample sizes were considered: $N=1000,2000,10000,20000$. Figure \ref{Fig_scplot} shows the scatterplot matrix of the full dataset for one simulation. The matrix highlights that the features were constructed so that the butterfly data not only contain linear relationships, but also a wide range of non-linear redundancies.
\item The Friedman dataset: this dataset uses a function suggested in \cite{Fri91} to test the ability of Multivariate Adaptive Regression Splines (MARS) models to uncover structures in data. The output $Y$ is given by:\begin{equation}\begin{aligned}Y = &10 \sin(\pi X_1 X_2)  +  20(X_3-0.5)^2  +\\
                                   &10 X_4  +  5 X_5  +  \varepsilon  \end{aligned}\end{equation}where $X_1$, $X_2$, $X_3$, $X_4$ and $X_5$ are i.i.d. variables following a uniform distribution $\textit{Unif}(0,1)$, and $\varepsilon$ is a Gaussian random noise with zero mean and unit variance. The input space is then completed by the addition of five irrelevant variables ($I$) following the same uniform distribution: $I_6$, $I_7$, $I_8$, $I_9$ and $I_{10} \sim \textit{Unif}(0,1)$. Finally, the Friedman dataset is produced by randomly sampling $N$ points from the inputs. In this paper, the sample size was set to $N=40000$ in accordance with the version of the dataset available on the Regression website \cite{Tor}.
\end{enumerate}

The butterfly and Friedman datasets are characterized by non-linear structures, and their input spaces contain extra variables (i.e. redundant and irrelevant variables) that can be removed without affecting the learning of the target $Y$. In the following subsections, MBFR will be subjected to a battery of tests to highlight its ability to select the relevant variables ($X$) and to remove the irrelevant ($I$) and redundant ($J$) ones. Additional experiments will consider shuffled data to examine the response of the algorithm to a complete absence of structure. In parallel, the variability of the results will be examined by means of Monte Carlo simulations: for each experiment, many simulations of the datasets will be generated by repeated random sampling of the input variables.

Notice also that the way the two datasets are constructed leads to the distinction between the data manifolds and the manifolds of the simulated phenomena. The former are built using all the variables (including the output variable), while the latter (referred to as the Friedman and butterfly manifolds) do not involve the irrelevant features.

Finally, from the perspective of MBFR, a dataset is fully characterized by the integer values of $\ell^{-1}$. For the butterfly and Friedman datasets, these values were respectively set to $\lbrace5,6,\ldots,20\rbrace$ and $\lbrace1,2,\ldots,6\rbrace$. The two sets were chosen, so that, within their bounds, the relationship between $\log{(I_{m=2,\ell^{-1}})}$ and $\log{(\ell^{-1})}$ was linear. Notice that the upper bound of the second set is lower than that of the first one. This partially follows from the fact that the Friedman dataset has the greatest ID causing the data points to be sparsely distributed inside the data space. As a consequence, beyond $\ell^{-1}=5$, the probability of drawing two points from the same cell is rather low, while it is possible to use values of $\ell^{-1}$ up to $20$ in the case of the butterfly dataset.

\begin{table}
\centering
\footnotesize
\begin{tabular}{cccc}
\hline
$N$    &  First Two Features (Occurrences) &  $mean(DR)$  & $sd(DR)$\\
\hline
\hline
1000     & $X_1$,$X_2$ ($99$); $X_2$,$X_1$ ($1$)  & $0.97$ &  $0.02$\\
2000     &           $X_1$,$X_2$ ($100$)          & $0.97$ &  $0.02$\\
10000    &           $X_1$,$X_2$ ($100$)          & $0.97$ &  $0.01$\\
\hline
\end{tabular}
\caption{The first two features selected by MBFR when applied successively to $100$ simulations of the butterfly dataset for different sample sizes. The mean values and the standard deviations of $DR$ are also provided.}
\label{T7}
\end{table}

\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{Nbr_points.pdf}
\caption{Results of the application of MBFR to 100 simulations of the butterfly data with (a) $N=1000$, (b) $N=2000$ and (c) $N=10000$. Notice that the names of the features were shortened to $X$, $J$ and $I$ if the way they were ordered by the SFS search strategy changed between the simulations.}\label{Fig_nbr_points}
\end{figure}

\subsection{Sample Size}\label{samplesize}
MBFR was applied to the butterfly dataset. Three sample sizes were successively considered ($N=1000$, $2000$, $10000$), and for each of them, $100$ simulations of the data were produced.

Table \ref{T7} gives the first two input variables selected by the algorithm and indicates the number of times they were selected first over the simulations. Regardless of the sample size, MBFR always identifies $X_1$ and $X_2$ as the most relevant features, although their order can be reversed for $N=1000$. Besides, the predictive power of these two features was assessed using the coefficient of dimensional relevance $DR$. Table \ref{T7} provides the mean values of $DR$ over the simulations as well as the corresponding standard deviations (sd). The means are close to $1$, which implies that $X_1$ and $X_2$ convey most of the information contained in the dataset, and this is in agreement with the data construction: $X_1$ and $X_2$ are sufficient to explain $Y$, whereas the other features are not necessary or even useless. Moreover, the SFS search strategy enables MBFR to select the most relevant features by exploring a rather low-dimensional space. Consequently, the variability of $DR$ remains roughly constant over the sample sizes, and the standard deviations do not exceed $0.02$.

To explore further the potential of the MBFR algorithm, a new series of $100$ simulations were generated. For ease of comparison with the next subsections, a constraint was imposed that for each simulation the redundant variables ($J$) had to be selected by MBFR before the irrelevant ones ($I$). The results are plotted in Figure \ref{Fig_nbr_points}.

The red dots indicate the mean dissimilarity values that are computed, over the simulations, by adding to the input space the features appearing on the horizontal axis. In addition, the red bars are the corresponding standard deviations. The features are progressively selected from left to right according to the SFS search strategy of MBFR, and the names of the redundant and irrelevant features were shortened to the letters $J$ and $I$ because they happened to switch position between the simulations. For the same reasons, $X_1$ and $X_2$ were replaced with the letter $X$ for $N=1000$. Furthermore, in each plot, two additional values are provided: the mean ID estimate of the target variable $Y$ (i.e. $mean\left(M_2\left(Y \right)\right)$) and the mean of the minimum dissimilarity (i.e. $mean\left(min\left(\widehat{Diss} \right)\right)$). The standard deviations of the two values are indicated using the black stripes.

For each sample size, $X_1$ and $X_2$ are easily identified as the two relevant features, since they contribute to reducing the dissimilarity from $M_2(Y)$ to about $0$ and a clear cut-off point is visible. However, as the number of points is reduced, the variability of the dissimilarity estimates increases. It does not question the potential of the algorithm for feature selection, but it emphasizes two aspects of its implementation: (1) the progressive increase in the variability as more features are added, and (2) the departure from $0$ of the mean dissimilarity estimates after the addition of the second relevant feature (see Figure \ref{Fig_nbr_points}). These two aspects will be addressed in the next subsection.

\subsection{Complexity of data manifolds}\label{manifold}
Lower sample sizes highlight that the variability of the dissimilarity values progressively increase as more features are picked out (see Figure \ref{Fig_nbr_points}). This response of MBFR is partly due to the presence of the relevant and irrelevant features which amplifies the data ID during the SFS procedure. But it is also related to the non-linear constructions of these features that affect the ID estimates by altering the point clustering on the data manifold.

\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{Manifold.pdf}
\caption{Results of each term of Equation \ref{diss_equ_est} for (a) the original butterfly dataset ($N=1000$), for (b) a modified version in which the non-linear dependencies between the features were replaced with pure linear ones ($N=1000$), and for (c) a modified version in which $Y$ is replaced with a zero constant value ($N=1000$). Notice that the names of the features were shortened to $X$, $J$ and $I$ if the way they were ordered by the SFS search strategy changed between the simulations.}\label{Fig_Manifold}
\end{figure}

The upper panels of Figure \ref{Fig_Manifold} illustrates the impact of the non-linear dependencies on MBFR. The left-hand panel displays the results of each term of Equation \ref{diss_equ_est} for $100$ simulations of the original butterfly dataset. It can be clearly seen that the non-linearly constructed redundant and irrelevant features influence the ID estimates when they are added to the previously selected features. In contrast, the right-hand panel shows what happens when only pure linear dependencies are considered (i.e. $J_1$, $J_2$ and $J_3$ were replaced with $X_1$, $I_7$ and $I_8$ were replaced with $I_6$): the mean values and the standard deviations of the ID estimates are modified exclusively by the input variables bringing new information (either useful or useless), and the increase in the variability of the dissimilarity values is no longer progressive. It simply corresponds to $X_1$, $X_2$ and to the addition the first irrelevant feature.

There is still one aspect of the results of Figure \ref{Fig_nbr_points} which has not been fully accounted for yet: after the addition of $X_1$ and $X_2$, the dissimilarity values should be equal to $0$, but the estimates are slightly higher. Likewise, the mean values of $DR$ in Table \ref{T7} should not be lower than $1$. The observed deviations are due to the gap between $mean(M_2(F,Y))$ and $mean(M_2(F))$ that is clearly visible in the upper panels of Figure \ref{Fig_Manifold} after the addition of the first irrelevant feature. However, if the target variable $Y$ is replaced with a constant value, the gap vanishes and the dissimilarity estimates equal $0$. The last panel of Figure \ref{Fig_Manifold} shows the outcome of MBFR for such a simplified version of the butterfly manifold (again, $100$ simulations were used, and the non-linear dependencies between the input variables were replaced with pure linear ones for ease of comparison with the upper panels). This result shows that the shape and the orientation of the data manifold, along with the non-linear construction of $Y$, are key factors to explain the gap between the mean ID estimates of the original dataset. The importance of these factors might be partly related to the quadrats (i.e. the hyper-boxes) of the MINDID algorithm which cannot fit perfectly complex point patterns.

In conclusion, the complexity of the data manifolds (i.e. their shapes, their orientations, the non-linear dependencies between the features and the non-linear constructions of the output variables) affects the results by altering the terms of Equation \ref{diss_equ_est}. However, it does not prevent MBFR from identifying the relevant features.

\subsection{Redundant and Irrelevant Information}\label{info}
The MBFR algorithm aims to detect the features which are useless (i.e. irrelevant) or not necessary (i.e. redundant) to a regression problem. In addition, it is also able to help distinguish between the two types of inputs, and more precisely, between redundant and irrelevant information.

For instance, in Figure \ref{Fig_Manifold}, the first irrelevant feature causes the mean ID estimates to increase by about $1$ (i.e. by about the value of $M_2(I)$), and if it was removed, the second one would have the same effect. In contrast, the redundant features have a much smaller impact. It is even hardly noticeable for the relatively low ID values, as highlighted by the dashed ellipse in the left-hand panel. Consequently, the inputs of the butterfly data can be classified as either redundant or irrelevant according to their impacts on the ID estimates.

In real-world applications, a feature (e.g. $F_1$) rejected by MBFR could contain both redundant and irrelevant information. Nevertheless, the exact amounts of the two types of information could still be quantified by using the terms of Equation \ref{diss_equ_est}. For instance, if $F_1$ was partly redundant and partly irrelevant, it would cause an increase in the data ID which would be both higher than $0$ (fully redundant) and lower than the value of $M_2(F_1)$ (fully irrelevant). $F_1$ would also contain more irrelevant information if the increase was closer to $M_2(F_1)$ than to $0$.

In conclusion, the MBFR algorithm can help distinguish between redundant and irrelevant information by means of the ID estimates on which it relies.

\subsection{Lack of Information}\label{no_info}
This subsection investigates the behaviour of the MBFR algorithm when the relevant information is completely or partially missing.

The top-left panel of Figure \ref{Fig_info} displays the results of MBFR achieved for $100$ simulations of the Friedman dataset. The algorithm distinctly detects the five relevant features and offers a clear cut-off point. The computations were rerun after the removal of $X_5$ and the results are given in the top-right panel. This time, the value of $\min(\widehat{Diss})$ is higher, and the difference accounts for the amount of information of $X_5$. This last experiment shows that the MBFR algorithm is also able to detect and quantify the absence of relevant features.

\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{Information.pdf}
\caption{Results of MBFR for (a) the complete Friedman dataset and (b) after the removal of $X_5$. In (c) and (d), MBFR was applied to the butterfly dataset after the shuffling of the target variable $Y$ for, respectively, $N=10000$ and $N=20000$. Notice that the names of the features were shortened to $X$, $J$ and $I$ because the way they were ordered by the SFS search strategy changed between the simulations.}\label{Fig_info}
\end{figure}

It is also worth exploring how MBFR responds to a complete absence of structure between an input and an output space. A second numerical experiment was set up to that end. The butterfly dataset was used, and $100$ simulations were generated for $N=10000$ and $N=20000$. The target variable $Y$ of each simulation was then shuffled to destroy the dependencies between the input and output spaces. The right-hand panel of Figure \ref{Fig_butterVisual} illustrates the impact of the shuffling procedure on the functional relationship between $Y$, $X_1$ and $X_2$. Finally, MBFR was applied to each simulation, and the results are displayed in the bottom panels of Figure \ref{Fig_info}. As expected, the values of $\widehat{Diss}$ stay close to $M_2(Y)$, which indicates that no features carry relevant information with regard to $Y$. The remaining gap between the mean values of $\widehat{Diss}$ and $M_2(Y)$ follows from the complexity of the data manifold discussed in Subsection \ref{manifold}, and it is more pronounced for the lower sample size, as expected from Subsection \ref{samplesize}.

Notice also that for comparison purposes, the simulations used in the lower panels of Figure \ref{Fig_info} were restricted to those for which the relevant features were selected first, followed successively by the redundant and irrelevant ones. Without this restriction, the final rankings were unpredictable, and the irrelevant features could also be picked first. This is consistent with the shuffling procedure which makes all the inputs irrelevant.

In conclusion, the MBFR algorithm provides a way to detect and quantify a lack of relevant information. In many cases, a part of that information is not truly missing, but simply corrupted with noise, and the next subsection investigates this issue.

\subsection{Additional Noise}\label{noise}
MBFR should also be able to provide reliable results when the data points are near a manifold instead of being exactly on it. This aspect was investigated by using noisy versions of the butterfly dataset. The target variable $Y$ was corrupted with a Gaussian noise (see $\varepsilon$ in Equation \ref{equ_butter}) characterized by a zero mean and a varying standard deviation (sd) ranging from $0\%$ to $100\%$ of the original standard deviation of $Y$ (the mean standard deviation of $Y$ over $100$ simulations is $0.52$). In total, seven noise thresholds were considered (i.e. $0\%$, $10\%$, $20\%$, $25\%$, $50\%$, $75\%$ and $100\%$), and for each of them the butterfly dataset was generated $100$ times with $N=10000$. The MBFR algorithm was then applied to each simulation, and the first two selected features were recorded at each run. The results are given in Table \ref{T1}.

The first two features selected by MBFR are given in the middle column, along with the number of times they were selected first over the simulations (in brackets). In addition, the minimum value of $\widehat{Diss}$, over all the simulations, is indicated in the last column. The results show that the first two features can switch position when the noise threshold is higher than $20\%$ and that $J_3$ is often substituted for $X_1$. Nevertheless, even for the greatest noise thresholds, the irrelevant features are never selected first, which means that the MBFR algorithm is still able to recognize the data manifold. Of course, the minimum dissimilarity value is higher once the noise has been added. This was to be expected according to Subsection \ref{no_info}, since the noise component partially masks the information that a variable may carry.

\begin{table}
\centering
\footnotesize
\begin{tabular}{ccc}
\hline
Noise Threshold    &  First Two Features (Occurrences) &  $\min(\widehat{Diss})$  \\
\hline
\hline
$0\%$              & $X_1$,$X_2$ ($100$)                                       & $0.02$\\
$10\%$             & $X_1$,$X_2$ ($100$)                                       & $0.24$\\
$20\%$             & $X_2$,$J_3$ ($2$) ; $X_1$,$X_2$ ($4$); $X_2$,$X_1$ ($94$) & $0.46$\\
$25\%$             & $X_2$,$J_3$ ($41$); $X_2$,$X_1$ ($59$)                    & $0.55$\\
$50\%$             & $X_2$,$X_1$ ($2$) ; $X_2$,$J_3$ ($98$)                    & $0.72$\\
$75\%$             & $X_2$,$X_1$ ($10$); $X_2$,$J_3$ ($90$)                    & $0.81$\\
$100\%$            & $X_2$,$X_1$ ($10$); $X_2$,$J_3$ ($90$)                    & $0.85$\\
\hline
\end{tabular}
\caption{The first two features selected by MBFR when applied successively to $100$ simulations of the butterfly dataset for different noise thresholds. The minimum value of $\widehat{Diss}$ over the simulations is also provided.}
\label{T1}
\end{table}

\subsection{Concluding Remarks}
The variability of the results of MBFR increases as the sample size is reduced. Besides, the use of small sample sizes highlights the impact of the manifold complexity on MBFR: MBFR is influenced by the shapes and the orientations of the data manifolds, as well as by the non-linear dependencies between the features and the non-linear constructions of the output variables. Despite this, the numerical experiments performed in this section show that MBFR is able to effectively fulfilled its goal. In other words, it can identify the relevant features and provide a clear cut-off point indicating the number of features to be retained, even in the presence of noise. MBFR also allows the user to make a distinction between redundant and irrelevant information. This is achieved by the analysis of the ID estimates involved in the implementation of the algorithm. And finally, the dissimilarity values computed from the ID estimates can help detect and quantify a possible lack of relevant information.

\section{Experimental Study Using Real Data}\label{Exp_real}
In this section, the MBFR algorithm is applied to real-world case studies from the UCI machine learning repository \cite{Lich} and the Regression website \cite{Tor}. The results are discussed with a special emphasis on the parameter $\ell^{-1}$, the coefficient of dimensional relevance $DR$, and the ability of MBFR to distinguish between redundant and irrelevant information. Finally, a comparison with a renowned filter, RReliefF \cite{Robnik03}, is conducted by using Extreme Learning Machine (ELM) \cite{Hua06}.

\subsection{Data}\label{subsec_real_data}
Six datasets from the UCI machine learning repository \cite{Lich} and the Regression website \cite{Tor} were used in the experiments:\begin{enumerate}[noitemsep]
\item Abalone: the goal is to predict the age of marine snails (the abalones) from physical measurements. This dataset originally contains 4177 instances and 8 features. Among the instances, two outliers were detected and removed (instances 1418 and 2052), and only the physical measurements were considered. Consequently, the resulting dataset consists of 4175 instances characterized by 7 features.
\item Ailerons: this dataset contains 13750 instances and 40 continuous features describing the status of a F16 aircraft. The goal is to predict the control actions on the ailerons. Some issues affect the last 16 features which can be treated as either nominal or continuous. Since the decision may have an influence on the results of RReliefF, these input variables were not considered, and the dataset used in the experiments consists of 13750 instances and 24 features.
\item Boston Housing: The objective is to predict the housing prices in areas of Boston. The dataset contains 506 instances and 13 features.
\item CompAct: The goal is to predict the portion of time (\%) during which a set of CPUs run in user mode. The prediction is performed using a collection of computer system activity measures. The dataset contains 8192 instances and 21 features.
\item Parkinson's Telemonitoring: One of the objectives is to predict the motor score of the Unified Parkinson's Disease Rating Scale (UPDRS) from 16 biomedical voice measures. Telemonitoring devices were used to automatically capture speech signals in 42 patients' homes, and 5875 voice recordings were made available. One of the voice measures was removed, since it provides the same value for all the recordings, and the final dataset used in the experiments consists of 5875 instances and 15 features.
\item CT slice: the task is to predict the relative location of Computer Tomography (CT) slices on the axial axis of the human body. The prediction is carried out using features extracted from CT images. The dataset originally contains 53500 instances described by 385 features. But 63 instances are replicated several times, and 5 features provide a constant value. Consequently, the data preprocessing resulted in a slightly modified dataset consisting of 53436 instances and 380 features.
\end{enumerate}

\subsection{Feature Selection with MBFR}
The MBFR algorithm was applied to the real world datasets described in the previous subsection. For each of them, the set of values of the parameter $\ell^{-1}$ was chosen as follows:\begin{enumerate}[noitemsep]
\item The plot relating $\ln(I_{m=2,\ell^{-1}})$ to $\ln(\ell^{-1})$ was computed for the full dataset (including all the instances, all the features and the target variable) with $\ell^{-1} \in \lbrace 1,2,\ldots,130\rbrace$.
\item The upper and lower bounds of the set were given by the extent of the linear part of the plot. For most of the datasets, the upper bound turned out to be simply the maximum value of $\ell^{-1}$ ensuring the presence of two points in, at least, one quadrat.
\item If the upper bound was lower than 30, every integer value within the bounds was retained. But, if it was equal to or higher than 30, only the integer values following a geometric progression with ratio $2$ were used, and the bounds were modified accordingly. This allows the MBFR algorithm to run faster by reducing the value of $R$ (see Section \ref{Mindex}).
\end{enumerate}

The resulting sets of values of the parameter $\ell^{-1}$ are given in Table \ref{T3}. Although it might seem better to change the values for each feature combination, it turned out not to be necessary. Once a set had been built by following the described procedure, it was used throughout the feature selection process. The CompAct dataset was the only exception: the log-log plots of several single features were characterized by two distinct linear parts. The steepest one was retained, since it led to a higher value of $DR$. Finally, Table \ref{T3} also gives the values $M_2$ for the whole datasets (including the target variables). These values suggest that the dimensions of the spaces in which the data points truly reside could be smaller than that of the original data spaces. In other words, the datasets could contain redundant information that MBFR might uncover.

\begin{table}
\centering
\footnotesize
\begin{tabular}{ccccccc}
\hline
Datasets &  Parameter $\ell^{-1}\in$            &  $M_2$  &\# F.  &  $\widehat{Diss}$ &   $DR$\\
\hline
\hline
Abalone  & $\lbrace 4,8,16,32,64\rbrace$        & $3.66$  &$3 (7)$     &     $0.36$       & $0.46$ \\
Ailerons & $\lbrace 5,6,\ldots,24,25\rbrace$    & $5.34$  &$7 (24)$    &     $0.22$       & $0.71$ \\
Housing  & $\lbrace 2,3,\ldots,18,19\rbrace$    & $3.30$  &$8 (13)$    &     $0.14$       & $0.84$ \\
CompAct  & $\lbrace 1,2,\ldots,9,10\rbrace$     & $2.16$  &$6 (21)$    &     $0.03$       & $0.94$ \\
Parkinson& $\lbrace 1,2,4,8,16,32,64\rbrace$    & $4.30$  &$8 (15)$    &     $0.58$       & $0.31$ \\
CT slice & $\lbrace 1,2,4,8,16,32,64,128\rbrace$& $3.09$  &$22 (380)$  &  $\approx 0.00$  & $\approx 1.00$ \\
\hline
\end{tabular}
\caption{Parameters and Results of the application of MBFR to the real world datasets. The values in brackets in the $4^{th}$ column are the total numbers of features in the datasets, and ``\# F.'' stands for ``number of selected features''. Besides, $\widehat{Diss}$ and $DR$ were computed by considering only the selected features, while $M_2$ is given for the whole datasets (including the target variables).}
\label{T3}
\end{table}

\begin{figure}
\centering
\includegraphics[width=\linewidth]{RealDataMBFR.pdf}
\caption{Results of MBFR: (a) Abalone (b) Ailerons (c) Boston Housing (d) CompAct, (e) Parkinson's Telemonitoring, (f) CT slice. The selected features are indicated in bold red font, and ``Target'' refers to the output (or target) variable of a dataset.}\label{RealData_MBFR}
\end{figure}

The MBFR algorithm was applied to each dataset with the parameter values of Table \ref{T3}. The results are displayed in Figure \ref{RealData_MBFR}. In each panel, a relatively clear cut-off point allows the user to identify the features to be selected. Moreover, Table \ref{T3} summarizes the results by providing the dissimilarity estimates (i.e. $\widehat{Diss}$) and the values of $DR$ that were computed by considering only the selected features. This overview shows that MBFR leads to a significant reduction in the number of features and that the chosen datasets cover a wide range of situations. For instance, in the Parkinson dataset, 8 features out of 15 are detected as relevant, and they account for about a third of the information contained in the target variable. In contrast, in the CT slice dataset, 359 features are considered redundant or irrelevant, and the relevant ones fully explain the relative location of the CT slices (i.e the target variable).

In conclusion, this subsection highlights the effectiveness of the MBFR algorithm for feature selection in real world applications. In the next subsection, the results will be validated by means of a comparison with a benchmark algorithm called RReliefF.

\subsection{Comparison with RReliefF Using Extreme Learning Machine}
In this subsection, the performance of MBFR is compared with that of RReliefF \cite{Robnik96,Robnik97,Robnik03} using a comprehensive evaluation procedure based on Extreme learning Machine (ELM) \cite{Hua06,GHua15}. A brief introduction to ELM and the Relief family of algorithms is also provided for a good understanding of the results.

The Relief family comprises three main algorithms for feature selection: Relief \cite{Kir92}, ReliefF \cite{Kon94,Kon97} and RReliefF \cite{Robnik96,Robnik97,Robnik03}. They all consist in attributing scores to the features according to how well their values can distinguish between instances that are close to one another in the data space. Relief achieves this goal for two-class classification problems. It randomly selects an instance and searches for its two nearest neighbours: one from the same class and one from the other class. After that, it updates the scores of the features depending on their values for the randomly selected instance and the two neighbours. The operation is repeated $m_{RF}$ times, and a final score lower than $0$ indicates that a feature might be irrelevant. This threshold, determining whether a feature should be retained, is one of the major advantages of the technique. Following a similar procedure, the ReliefF algorithm is an extension which can deal with multiclass problems and with incomplete and noisy data.

RReliefF (i.e. Regressional ReliefF) is an adaptation of Relief and ReliefF to regression. In regression, the exact knowledge of whether two instances belong to the same class or not cannot be used. RReliefF replaces it with a probability that the predicted values of two instances will be different. Moreover, it computes the final score of each feature by taking into account $k_{RF}$ neighbours. Among these neighbours, the closest ones should have a greater influence, and a kernel of parameter $\sigma_{RF}$ can be used to assign a weight to each of them.

The algorithms of the Relief family have often been used as benchmarks \cite{Hall00,Sousa07,Vi16}. In this research, RReliefF was applied to the real world datasets of Subsection \ref{subsec_real_data}, with the exception of the CT slice data that contain too many instances. The R package ``CORElearn'' \cite{Core15} was used with the evaluation heuristic ``RReliefFexpRank'' and its default parameters: $m_{RF}=N$ ($N$ is the number of instances in the datasets), $k_{RF}=70$ and $\sigma_{RF}=20$. These parameters were tested and turned out to be suitable for each dataset. Besides, RReliefF was run with two relevance thresholds: $0.00$ and $0.01$ as suggested in \cite{Hall00}. Any feature with a final score less than the specified threshold was considered irrelevant.

Table \ref{T5} compares the ability of RReliefF and MBFR to reduce the dimensionality of the datasets. In all but one case, MBFR performs more feature selection than RReliefF with the relevance threshold of $0.00$. The differences are less pronounced with the relevance threshold of $0.01$ for which the two algorithms achieve comparable results on three datasets. Nevertheless, they still provide distinct outcomes for the CompAct and Parkinson data, and they never select the exact same features. Consequently, the comparison requires a way of assessing the amount of information contained in the selected features. This is the reason why ELM is used in this paper.

ELM is a single layer feed-forward neural network which can achieve the same accuracy as the well-known Multilayer Perceptrons (MLP) \cite{Ros58,Ros62,Wer74,Rum86}, while being much faster. Its main specificity lies in the weights connecting the inputs to the hidden layer. They are randomly generated and never updated, and then the weights between the hidden layer and the outputs are learnt in a single step. In this way, the number $\tilde{N}$ of hidden nodes is the only hyperparameter of ELM, making its implementation rather straightforward. However, an activation function is also required and a sigmoid function was used in this paper.

ELM has been applied successfully in a wide range of case studies \cite{Bar13,Leu14,Me15}, and its high efficiency makes it well-suited to carry out comparisons between feature selection techniques. It is also worth mentioning that RReliefF and ELM have already been combined to effectively improve learning performance in temperature forecasting \cite{Mat13}. In this paper, ELM was used to evaluate the predictive power of the subsets of features selected by MBFR and RReliefF. This evaluation was achieved according to a procedure which was partly presented in \cite{Ja97,Reu03} to prevent overfitting \cite{Li16}. It can be subdivided into 5 steps:\begin{enumerate}[noitemsep]
\item $20\%$ of the $N$ instances are randomly assigned to a test set, and the remaining $80\%$ are passed on to Step 2. The same split is used for all the subsets of features being compared.
\item The data are projected into the $[0,1]$ interval, and the coefficients of the projection are recorded.
\item \label{step_cv} The value of the hyperparameter $\tilde{N}$ is selected by performing 10-fold cross-validation. For each value of $\tilde{N}\in\lbrace1,\ldots,350\rbrace$, 1 fold is iteratively allocated to a set of validation, and the remaining 9 folds are assigned to a training set. For each of the 10 iterations, an ELM model is fit on the training observations, and the Mean Square Error (MSE) is computed using the validation set. Then the 10 MSEs are averaged to provide an estimate of the true error, and their standard deviation is recorded. Finally, the value of $\tilde{N}$ resulting in the lowest error estimate is retained for Step 4, unless the corresponding variability is too high. In that case, $\tilde{N}$ is manually tuned to find a good compromise between the mean and the variance of the error.
\item Using all the instances involved in the cross-validation procedure (i.e. $80\%$ of the original data), a new model is trained with the value of $\tilde{N}$ from Step 3. Then a prediction is made for the instances of the test set (i.e. $20\%$ of the original data) after they have been projected to the $[0,1]$ interval using the coefficient of Step 2. The operation (training and prediction) is repeated 100 times to account for the variability of the weights connecting the inputs to the hidden nodes. The final prediction for each instance is computed by averaging the 100 values and by rescaling the results to the original output range (using coefficients of Step 2). Finally, the relative mean squared error \cite{Robnik96} is calculated on the test set. It is defined as follows:\begin{equation}\label{Eq_RE}
RE_{tst}=\frac{\sum_{i=1}^{N_{tst}}(y_i-\hat{y}_i)^2}{\sum_{i=1}^{N_{tst}}(y_i-\bar{y}_{tst})^2}
\end{equation}where $N_{tst}$ is the number of instances in the test set, $y_i$ is the measurement of the output variable for the $i^{th}$ instance, $\hat{y}_i$ is the corresponding prediction, and $\bar{y}_{tst}$ is the mean of the output variable computed on the test set. Lower values of $RE_{tst}$ are better and a value higher than $1$ indicates that the tested model performs worse than the mean.
\item Steps 1 to 4 are repeated 20 times to account for the randomness in the data splits of step 1. After that, the mean and standard deviation of the 20 values of $RE_{tst}$ are calculated and are used to assess the predictive power of the selected features.
\end{enumerate}

For each dataset, four sets of features were passed on to the evaluation procedure: the set selected by MBFR, the two sets selected by RReliefF (one for each relevance threshold) and a benchmark set selected by a technique called ELM\_{SFS}. ELM\_{SFS} is a simple wrapper approach combining ELM and the same SFS search strategy as MBFR. It works as follows: at each step of the search process, the predictive power of each set of features is assessed using the same cross-validation as in Step \ref{step_cv} of the evaluation procedure; and finally, the set returning the lowest MSE over the entire SFS is selected.

Table \ref{T5} presents the results of the evaluation procedure for each feature selection technique. The mean values of $RE_{tst}$ over the 20 iterations are provided, along with the corresponding standard deviations. The sets of features selected by MBFR provide comparable or better accuracies (i.e. lower values of $RE_{tst}$) than those resulting from RReliefF. This is true even when MBFR performs more feature selection. Besides, although neither RReliefF nor MBFR improves the performance of ELM, only MBFR is able to maintain or stay close to (i.e. no more than one standard deviation away from) the mean values of $RE_{tst}$ achieved by ELM\_{SFS} for each dataset.

\begin{table}
\centering
\scriptsize
\begin{tabular}{ccccccccc}
\cline{2-9}
&\multicolumn{2}{c}{RReliefF 0.00}&\multicolumn{2}{c}{RReliefF 0.01}&\multicolumn{2}{c}{MBFR}&\multicolumn{2}{c}{ELM\_{SFS}}\\
\hline
Datasets &\# F.&   $RE_{tst}$   & \# F.&    $RE_{tst}$    & \# F. &   $RE_{tst}$ & \# F.    &  $RE_{tst}$     \\
\hline
\hline
Abalone  & $7^{\ast}$& {\boldmath$0.43$($0.02$)}  &$1$ &$0.57$($0.02$)            &$3$ & \underline{$0.46$($0.03$)} &$5$ & {\boldmath$0.43$($0.02$)}\\
Ailerons &$24^{\ast}$& {\boldmath$0.15$($0.01$)}  &$8$ &$0.24$($0.01$)            &$7$ & {\boldmath$0.15$($0.01$)} &$9$& {\boldmath$0.15$($0.01$)}\\
Housing  &$10$&  $0.17$($0.06$) &$8$ &$0.20$($0.06$) &$8$ & \underline{$0.16$($0.06$)} &$9$& {\boldmath$0.13$($0.05$)}\\
CompAct  &$21^{\ast}$& {\boldmath$0.02$($0.00$)}  &$20$ &{\boldmath$0.02$($0.00$)} &$6$ & {\boldmath$0.02$($0.00$)} &$11$& {\boldmath $0.02$($0.00$)}\\
Parkinson& $2$&  $0.90$($0.02$) &$1$ &$0.98$($0.01$) &$8$ & \underline{$0.81$($0.02$)} &$6$& {\boldmath$0.79$($0.03$)}\\
\hline
\end{tabular}
\caption{Comparison between RReliefF and MBFR based on the number of selected features (i.e. \# F.) and the relative mean squared error $RE_{tst}$. The table provides the mean values and the standard deviations (in brackets) of $RE_{tst}$ over the 20 iterations of the evaluation procedure. And the number of selected features is marked with an asterisk if the feature selection process did not lead to a reduction in the dimensionality of the data. Moreover, the best results are indicated in bold script (the lower, the better) and the second best results are underlined.}
\label{T5}
\end{table}

In terms of computing time, MBFR turned out to be competitive with RReliefF, since the two algorithms run in less than 30 seconds (s) on all datasets, with the exception of Ailerons. On the Ailerons dataset, MBFR was slower than RReliefF. It performed feature selection in about 220 s, while only 56 s were necessary for RReliefF. This significant difference is mainly due to the fact that Ailerons is the dataset requiring the largest number of values of the parameter $\ell^{-1}$. Despite this, MBFR is extremely fast for an algorithm following a SFS search strategy. By comparison, ELM\_{SFS} took more than 100 hours to complete the full feature selection procedure, while leading to the same mean error values as MBFR. Notice that all the numerical experiments were carried out using an Intel Core i7-2600 CPU @ 3.40 GHz along with 16.0 GB of RAM under Windows 7, and the value of $C$ was set to $E-1$ (i.e. the default value) for each dataset.

Another interesting point is the relationship between $RE_{tst}$ and the coefficient of dimensional relevance $DR$. The correlation between the two measures was computed with the mean values of $RE_{tst}$ resulting from MBFR and the values of $DR$ given in Table \ref{T3}. It turned out that Pearson's coefficient was equal to $-0.96$, which tends to confirm that $DR$ is a promising measure of feature relevance.

\section{Conclusion}\label{End}
This paper presents a new algorithm for supervised feature selection, namely the Morisita-Based Filter for Regression problems (MBFR). As its name suggests, it is designed for regression problems, and it relies on the recently introduced Morisita estimator of Intrinsic Dimension (ID). Comprehensive numerical experiments were carried out using two simulated datasets: the well-known Friedman dataset and the butterfly dataset which was specifically designed for the needs of this research. Different sample sizes, noise levels and non-linear dependencies were tested, and the variability of the results was examined by means of Monte Carlo simulations. MBFR was shown to be an effective tool for reducing the dimensionality of large datasets of varying complexity. Besides, the ability of the algorithm to distinguish between redundant and irrelevant information was presented and successfully tested.

MBFR was applied to real world datasets from publicly accessible repositories. An innovative methodology was implemented to conduct a comparison with a benchmark algorithm called RReliefF. MBFR resulted in better or comparable performance according to the accuracy achieved by Extreme Learning Machine (ELM). This was true even when MBFR retained fewer features than RReliefF.

A new coefficient of relevance was introduced, namely the coefficient of dimensional relevance $DR$. It was estimated by using MBFR, and its reliability was evaluated by means of ELM. $DR$ is exclusively based on the ID concept, it is easily interpretable, and it can be applied to high-dimensional datasets.

Finally, this paper shows that ID-based methods have the potential to improve the performance of existing machine learning algorithms. In addition to the presented work, they can also contribute to the development of new powerful tools to conduct fundamental tasks, such as classification, clustering and pattern detection.

\section{Acknowledgements}
The authors are grateful to the anonymous reviewers for their helpful and constructive comments that contributed to improving the paper. They also would like to thank Mohamed Laib and Zhivko Taushanov for many fruitful discussions about machine learning and statistics.

\bibliographystyle{elsarticle-num}

redpajama_set_name: RedPajamaArXiv


Diffuser Specialist is a nationwide master distributor and manufacturer of ceiling light covers, lenses, light diffusers, and plastic lighting accessories. We stock a wide range of products from simple prismatic acrylic sheet to the latest in energy efficient retrofitting technology – and to put it in laymen's terms, that means we have a perfect light cover for all of your needs, functional & aesthetic.
Our friendly and knowledgeable staff can help you find exactly what you need and, if we don't stock it, we can manufacture it to your exact specification.
We are centrally located in the continental U.S. and have shipping warehouses in both San Antonio and Houston, Texas that allow us to quickly reach our customers across the United States. Since we are located in a major shipping hub, we can also ship to our international customers quickly and inexpensively.
As an already successful representative for Republic Conduit, Don Griffin was always looking for a new line to which he could apply his sales expertise. Don was immersed in the electrical industry and noticed a need for replacement lenses, ceiling light covers and lay-ins of all kinds. Diffuser Specialist was started in 1982 by Mary Ann and Don Griffin with the philosophy that it is always cheaper to replace the lens than replace the entire fixture. With very loyal customers, Diffuser Specialist quickly grew into South Texas' primary source of diffusing materials and accessories.
The 90's brought rapid growth as Diffuser Specialist expanded their product lines. As the business grew, Mary Ann found herself taking a larger role in the company and became CEO. Under her guidance, Diffuser Specialist branched out through the rest of Texas with a second location in Houston. This new location allowed for quicker shipping times to national and international customers. Our large warehouses and family owned hospitality make us the number one choice for all your plastic diffuser and electrical supply needs.
Philosophy: why replace the whole fixture when all you need is a lens?
In the 90's: Growth & prosperity allow Diffuser Specialist to quickly become South Texas' primary source for ceiling light covers.
Mary Ann gains a larger role in the company and becomes CEO.
Diffuser Specialist expands and opens new locations which allow for efficient shipping to national and international customers.
Diffuser Specialist is family owned & operated, even today.
Why replace the whole fixture when all you need is a lens?
Get in Touch …and order only what you need!


20 Gb is the maximum for partition C?
Discussion in 'Acronis Disk Director Suite' started by Beekersi, Jul 28, 2008.
I just purchased Disk Director 10 and an upgrade from True Image 8 to 11. I hoped that Disk Director would be able to resize two partitions for me quickly.
I have nearly 4 your old Dell Inspiron 510m notebook running Windows XP Pro. My D partition (NTFS) was 27.3 Gb and my primary C partition (NTFS) was 20 Gb. I want D to shrink 15 Gb and C to increase 15 Gb.
Disk Director allowed me to configure the shrink and increase at the same time. The notebook rebooted once and did its thing. The result was a D partition of 12.3 Gb as desired, and a C-partition that remained unchanged. No error messages.
I now have 15Gb of unallocated space sitting between C and D (see attachment). Whatever I try, I keep getting "The current size of the selected partition is the maximum possible"
I tried automatic and manual mode, create an new partition and merge it with C, boot from CD and doing the same thing without Windows. No result.
What can I do? I don't see a reason why I can't increase C with 15 Gb.
Is there any jumper at the hdd to limit space to 20GB per partition?
Have you checked the C: partition for any errors? DD won't resize a partition with errors or bad sectors.
Do you have a current backup image of the C: partition?
MrMorse: I don't think there's jumper forcing this limit. I never opened my laptop and I'm quite sure that it was delivered to me with a single partition of about 60 Gb (4 years ago).
MudCrab: I just checked for errors and bad sectors using DDS. After that, I was able to shrink C, but not to increase it. I continued trying and created a partition in the 15 Gb unallocated space. I created it as large as possible. However, some unallocated space of 7.8 Mb remained between C and this new partition of 14.99 Gb! Really strange... I created a 7.8 Mb partition as well. Then I used the "increase free space" function on C, taking space from the new 14.99 partition. Taking space form the 7.8 Mb partition was not allowed. I commited the action and the PC rebooted several times. The result should have been a C partition of 34.99 Gb, but this is the actual result: C still 20 Gb, 14.98 Gb unallocated, a 5.4 Mb partition and a 13.8 Mb partition ( in that order). It really drives me crazy! DDS did something totally different then what I asked for.
I'm considering to wipe the whole disk, create a new 35 Gb primary partition and restore the Acronis True Image I made this morning. Should that work? Restoring an image of a 20 Gb partition (probably with some bad sectors) on a 35 Gb partition? I would rather solve it in a different way.
What kind of partitions are there?
However, some unallocated space of 7.8 Mb remained between C and this new partition of 14.99 Gb!
This is most likely due to partition sector alignment.
When you tried the DD CD, did you also try using the Safe Mode version of DD?
I'm considering to wipe the whole disk, create a new 35 Gb primary partition and restore the Acronis True Image I made this morning. Should that work? Restoring an image of a 20 Gb partition (probably with some bad sectors) on a 35 Gb partition?
My guess would be that the Extended Partition Container is starting directly after the Primary partition (before the unallocated space). Since DD doesn't show this, it's invisible. Perhaps it's blocking DD from expanding the Primary partition, though I'd think it would just move it.
Do you have a screenshot of what Windows Disk Management shows for the drive?
MrMorse: the partitions are listed in the image attached to my first post, and the new situation can be seen in the picture attached to this post.
I decided to take the risk and create True Images of C and D, delete the C and D partition, create the partitions again with the desired size and restore the images. I was partly succesful.
Even the unallocted space I was looking at now was splitted in a 20 Gb part and a 27 Gb part. Creating a partition greater than 27 Gb was not allowed! I decided that I should be grateful if I would succeed creating a C partition in the 27 Gb unallocated space. At least a 7 Gb increase!
My first try to restore the image of C in the new 27 Gb partition resulted in a warning from True Image: "The selected partition contains the following error: Run list corrupted. Please note that you cannot change the file system and size of this partition. It is recommended that you check this partition after restoration with your operating system disk tools".
I did not restore it and first ran chkdsk /f /r on C and on D. I noticed that it was deleting corrupted attribute records (whatever that may be) on C. Then I created new True Images. I restored the image of C on the new 27 Gb partition. Succesful this time.
So now I have a 27 Gb C-partition and 20 Gb D partition. It works fine, although I would prefer C to be 35 Gb and D 12 Gb. I anyone has any idea what the error means and how the 20 Gb limited can be exceeded, I would be glad to hear about it.
DELL-Restore-partition (FAT16), Primary partition, Logical partition, an unknown FAT32 partition and a Acronis-secureZone partition.
1. create validated images from "C" and "D".
3. destroy all partitions of the 60GB hdd !
Without doing some manual disk editing, you may not be able to get around the problem. Even that may not work.
I agree with MrMorse that the easiest way is probably to create a backup image and then restore the partitions in the order and sizes you want them.
However, I would first create an Entire Disk Image backup (check the Disk # box). You can include the SZ in the backup if you want (it's empty so it won't take up any space).
Remove all partitions from the hard drive so it's all unallocated space.


Parts & Support for Central Vac Int.
Hoses for Central Vac Int.
Accessories for Central Vac Int.
Kits for Central Vac Int.
No matter how old or new your Central Vac Int. Central Vacuum system is, we can help! Call us at 1-800-997-2278.
Using your Central Vac Int. vac is proven to increase indoor air quality and reduce allergies!It's a smart investment.
How to Maintain Your Central Vac Int.
Your Central Vac Int. Central Vacuum is an integral part of a clean and healthy home. We can help you match the proper products for your cleaning needs, budget, and Central Vac Int. Vacuum Model. The foundational parts of a Central Vac Int. Central Vacuum are universal, such as the installation and wall inlet valves, making Central Vac Int. central vacuum repairs easy. Therefore, all hoses*, kits and accessories sold here are compatible with Central Vac Int. Central Vacuums. Some of the most popular parts for repairing Central Vac Int. Vacuums are given below or choose a category above.
The central vacuum parts, products, and accessories above are compatible with all Central Vac Int. central vacuum cleaner models produced over the years, including the models listed below and those that are not. If your home has no hoses or accessories, we suggest a complete attachment kit. The kits here will work perfectly with the system and there is information to help you make an informed decision. If you have a Central Vac Int. vacuum system but need help with repairing it, we have many products and lots of information that should work for you. In rare circumstances, non-universal hoses or inlets were used with the system, but there is always a solution for using universal parts available. Just give us a call for more information.
If you can't find what you need, please contact us by phone or through the Central Vac Int. Questions and Answers, and one of our helpful technical customer service representatives will point you in the right direction. We also have a large network of independent dealers that can assist you with a Central Vac Int. central vac repair, if needed.
I am extremely pleased with the product, the order process, the customer service, and the follow up.
*Note: Central Vac Int. Vacuum is a registered trademark of Central Vac International, not M.D. Manufacturing, Inc. Products sold by M.D. Manufacturing, Inc. are not from Central Vac Int. Vacuum but are designed to complement a Central Vac Int. Central Vacuum. Therefore, parts and products carry warranties from MD Manufacturing and not Central Vac Int. Vacuum and thus may alter the warranty provided by Central Vac Int. Vacuum.


"OS Interiors", The name for commercial interior designing and decoration services was established and managed by highly experienced professionals of the same field. Its team comprises of innovative, energetic, hard working, skillful members. It offers full range of designing and technical services covering all aspect of commercial interior design and decoration services to its clients.
We are not about imposing ideas. We believe in sharing concepts and thoughts, helping client to develop their vision to accomplish their design requirements.
We excel at collecting the concepts, thoughts and tastes of our clients and subsequently extend their imagination with our creative talent to achieve compatible, comfortable, visually exciting and timeless interiors.
Our motto"Building Trust is our Business" echoed our philosophy of providing outstanding services, building concrete ties & maintaining high business ethical standards with our clients.
"OS Interiors", The name for commercial interior designing and decoration services was established and managed by highly experienced professionals of the same field. Its team comprises of innovative, energetic, hard working, skillful members.


The Holy Trinity of tropical drinks.
Nearly all tropical cocktail recipes are some spin on these three ingredients.
The Daiquiri is the category's classic standard consisting of - that's right - rum, sugar and lime.
You're invited to experience Reno's newest contemporary cocktail bar with a decidedly tropical flavor.
Limited Street Parking on Virginia Street (Southbound).
Parking available on Mary Street and directly across Virginia from RSL.
for the most recent project information.
Copyright 2018-2019 Rum Sugar Lime LLC | All Rights Reserved.


Salter Electronic Bathroom Scales Instructions is free HD wallpaper. This wallpaper was upload at May 04, 2018 upload by admin in bathroom ideas.
So, if you'd like to obtain all these amazing images regarding (Salter Electronic Bathroom Scales Instructions), click save icon to download the photos to your personal pc. There're available for obtain, if you like and wish to own it, simply click save badge in the article, and it will be instantly down loaded in your notebook computer. At last if you want to secure new and latest photo related with (Salter Electronic Bathroom Scales Instructions), please follow us on google plus or book mark this site, we attempt our best to provide regular update with all new and fresh shots. We do hope you love keeping right here. For many up-dates and latest news about (Salter Electronic Bathroom Scales Instructions) pictures, please kindly follow us on twitter, path, Instagram and google plus, or you mark this page on bookmark area, We attempt to provide you with up-date periodically with fresh and new graphics, like your searching, and find the ideal for you.
Salter Electronic Bathroom Scales Instructions in your computer by clicking resolution image in Download by size:. Don't forget to rate and comment if you interest with this wallpaper.
Related Posts of "Salter Electronic Bathroom Scales Instructions"


text:
Are Narduzzi, Lyke Seeing Eye-to-Eye?
If you had told me two months ago that Pat Narduzzi and Heather Lyke weren't going to be seeing eye to eye, I'd have said you were crazy. After all, Narduzzi was on the search committee that eventually recommended the hire of Lyke to replace Scott Barnes.
However, that line of thinking may not be too far out of line.
Multiple sources have told Pittsburgh Sports Now that over the last couple of weeks, the relationship between Narduzzi and Lyke has become strained.
What's the source of this divide between Pitt's head of athletics and football coach?
Heinz FIeld. — DAVID HAGUE
The biggest issue right now is the subject of tarping the upper deck of Heinz Field. For a number of reasons, the subject matter has upset Narduzzi.
Lyke publicly raised the possibility last week without consulting or getting Narduzzi's input, which didn't go over well. Considering that he's the highest-profile coach and person it will impact, a conversation was probably warranted.
Secondly, the idea was brought up publically without much if any discussion within the athletic department. How much would this cost? How much of Heinz Field would be tarped off? How would this financially effect the Pitt season ticket holders?
Finally, Narduzzi, who is not in favor of tarping off seats, believes it would negatively impact his recruiting and let Lyke and others know that other coaches have already started using it against Pitt.
Tarping off seats at Heinz Field isn't the only source of friction.
Getting more money for assistants is a common practice in college sports. Unless you're at a powerhouse program like Alabama, LSU, etc., assistants aren't paid in the same tax bracket as the head coach. For that reason, the head coach appreciates the hours and efforts of his staff and tries to get them more money.
Pat Narduzzi, September 2, 2017. — DAVID HAGUE
Narduzzi is trying to do just that and it's being met with resistance from Lyke and Pitt Chancellor Patrick Gallagher.
Speaking of assistants, when a team doesn't win in college or professionally, it's common for changes to be made on the teams staff.
I can tell you that Narduzzi doesn't want to make any changes with his assistants. On Thursday, he came out in total support of offensive coordinator Shawn Watson.
Shawn Watson and Pat Narduzzi. — DAVID HAGUE
Was that just a coincidence? What will happen if Narduzzi is asked to make changes when he doesn't want to?
Keep in mind that college coaching staffs are expanding to 10 next year so there is going to be a ton of movement amongst assistants this offseason. Staying static might be hard, even if Pitt wants to.
The point of all this isn't to suggest Narduzzi is leaving Pitt. It's just to point out that like at a lot of athletic offices and work places, employees and bosses don't always agree and this relationship between coach and AD is one to watch as we approach the football offseason.
Related Topics:featured, Heather Lyke, Heinz Field, Heinz Field Tarp, Pat Narduzzi, Pitt football
Five Takeaways From Pitt-Virginia Tech Game
Pitt Football Practice Sights and Sounds: Week 13

redpajama_set_name: RedPajamaCommonCrawl


As a brand development firm, one of the most valuable services we offer might surprise you. It's objectivity.
So often, an organization's intuition on their brand is colored by deep engagement in their work. Those last couple words sound like a good thing, don't they?
But whether you are a sole proprietor, or a CEO of a global brand, the minute you dive into your work and market, your objectivity is compromised. You begin to see things in a different light through different filters (both positive and negative). Your busyness, or lack thereof, taints the "truth". This skewed vision now begins to impact your decision-making and your brand.
It could result in an unrealistic view of your loyalty status (either negative or positive) and in that case, you would be connecting with your marketplace under false pretences, which would be recognized quickly and have immediate ramifications. Alternatively, your view on your communications/advertising could be skewed because your in-house creative department (which you spearhead) followed your direction to the "T".
While no one is absolutely unbiased (we all bring our life experiences to the table), there are those of us who are keenly aware of our potential for bias and have put training and processes in place to be that valuable, unbiased opinion. This, mixed with a talented and experienced marketing pool, and combined with the client's strong knowledge of their work and marketplace creates an atmosphere of massive potential.
That is my biased opinion. What's yours?


As expected Sony came up with 2 new smartphones at IFA 2015 event. As like last year IFA, Sony unveiled Xperia Z5 & Z5 compact to the world market which are super successors to Xperia Z3 & Z3 Compact. Comparing to the previous smartphones lineup in Z' series, Sony wasn't showing any desired change on the phone from the users' perception, let's see if there's any change in the new flagships lineup.
We've been hearing a lot about Xperia Z5 features and the recent rumors leaked that the new flagship might feature a 23MP camera and a fingerprint sensor. Let us find out if it is true or not.
Assuring a strong agreement to the leak, Sony made a change to the camera and added a fingerprint sensor (most of the new smartphones are acquiring this feature). Sony equipped a 23MP rear camera which has customised Sony Exmor RS sensor. With its hybrid autofocus feature, the objects can be quickly captured without any delay. As like other smartphones in the Z series lineup you can capture stunning visuals even in low-light too ( thanks to the next generation Sony Image Sensor). The front-camera is as usual equipped with 5MP which is apt for the selfies and video calls.
Sony heard us 🙂 The design of the new smartphone is what we've been waiting for. Although it resembles the old 'Z' series smartphones the new fingerprint sensor on the power button made it unique. A frosted glass on the rear side gives a premium look for the phone.
Of course, it is a water-resistant phone just like the other smartphones lineup in Z series with IP68 certification and as expected the smartphone is powered by a Qualcomm's Snapdragon 810 Soc, Adreno 430 GPU and runs on Android v5.1 Lollipop. It has 3GB of RAM just like the Xperia Z3+ and has 32GB of internal memory. The memory is expandable up to 200GB using an external microSD card.
One interesting thing we heard at the event is the battery life. Sony officially stated that the new flagship will last up to 2 days which seems to be a pretty good move comparing to the low battery life Samsung Phones. It comes with 2900mAh non-removable battery & Just like the Xperia Z3's quick charging capability, the new phone is also featured with the same spec. It can power up the smartphone to run for 2 days in 45 minutes.
A 5.2-inch FHD 1080p display with a screen resolution of 1920 x 1080 pixels is what the display made of! There's no need to say how exquisite is the display which divinely shows the true meaning of colors in a beautiful way. The Xperia Z5 is available in white, Graphite Black, Gold and Green colors. The pricing details are yet to be revealed and we hope it won't reach a higher price tag like the new Samsung flagships released recently.
As like every other phone release from Sony, there's a compact version for this one too. Sony Xperia Z5 compact also features similar specifications with few changes. It has 2GB of RAM with the same 32GB internal memory but depending on the type of market it may come with the 16GB model too. The battery capacity is 2700mAh which is kinda low compared to the Xperia Z5. As the name suggests it is a compact version for the new flagship, the display is of 4.6-inches. Remaining specifications are almost similar except the weight and the dimensions.
They even revealed the world's first 4K display smartphone which is a premium version of Xperia Z5- Xperia Z5 Premium. It seems that Sony is slowly picking up in the race and might gonna crush the other phones soon. Even Nokia is also gearing up for a smartphone release next year which seems to be like the old days.
As of now, Sony hadn't hinted any clue regarding the pricing but the flagships will be available from October onwards.


We wish you a Happy New Year 2018. May the new year be less stormy, both concerning weather and politics, and may it bring health and happiness to you and your loved ones.
This coming year, the Ibn Rushd Fund will become 20 years old; we are thinking of celebrating and will notify you in due time on how and whereabouts.
The last year was packed with activities from lectures to philosophical diwans and other events big and small from the first month, and was completed on December 1 with the awarding ceremony to AMAN - Coalition for Coalition for Accountability and Integrity.
For all those among you who where unable to join us on the day, we have prepared videos of the talks and performances to watch and share with your friends.
If you like our work, please support us by becoming a member, or donate money to support our work.
Link to membership page Ibn Rushd Fund.
Musical Interludes with Zaina Dalla, vocals; George Saade, percussion; Gaitha Alshaar, Oud.


text:
Ukraine president rejects PM's resignation, gives second chance
Zelensky's understanding of economics. The former lawyer and deputy head of the prime minister's office became Ukraine's youngest-ever head of government at the age of 35 when he took on the rol...
Khamenei to lead Friday prayers for 1st time since 2012
At a rare Friday address, Iranian religious leader Ayatollah Ali Khamenei greeted the rocket attacks that damaged two U.S. bases in Iraq last week and wounded almost a dozen USA servants. He said I...
Clinton prosecutor Ken Starr to defend Trump in impeachment
The trial is expected to last at least two weeks and could feature testimony by close Trump advisers and former advisers who have yet to speak out on allegations that Trump withheld military aid and a...
Federal Bureau of Investigation arrests fugitive Canadian reservist after tapping into encrypted neo-Nazi chatroom
Lemley previously served as Cavalry Scout in the United States Army, and Mathews, a Canadian citizen illegally in the USA, was a combat engineer with the Canadian Army Reserve as of August 2019. Sever...
California woman found alive after 6 days stuck in snow-covered vehicle
About noon Wednesday, a sheriff's helicopter was flying over the Butte Meadows area when a sergeant spotted a vehicle below, covered in snow, about three miles from the Bambi Inn. James is conscio...
Michael Avenatti's bail revoked; trial in Nike case delayed
Gardephe summoned prosecutors and defense attorneys to a phone conference Wednesday and said Avenatti's arrest "has essentially thrown the trial into chaos". The arrest of Avenatti marks a new low f...
FAA: Delta didn't inform us about California fuel dump ahead of time
According to recorded radio communications, air traffic control asked the Delta crew if they wanted to return to LAX immediately or linger over the ocean "to hold and burn fuel". "They may have decide...
School expels 15-year-old for rainbow birthday cake, sweater
Kenney has started attended a public school, and although Alford says it is at times overwhelming, it is also likely for the best. The woman said that her daughter had been put on "probation" by t...
Rockets Fired Into Israel, Military Says 2 of Them Intercepted
The Israeli military responded in the evening by targeting Hamas installations, it said. Thursday's balloon launch appeared to reflect Hamas' unhappiness with the slow pace of the ceasefire efforts,...
Pope Francis appoints woman to senior Vatican diplomatic role
For the first time in history, the Pope has appointed a woman as manager in the State Secretariat of the Vatican - with Francesca Di Giovanni, a 27-year-old lawyer in the Catholic Church. "I would lik...
Secret mission saves Australia's 'dinosaur trees' from bushfires
Fire retardant, irrigation techniques and an army of conservation experts have helped save the wild prehistoric Wollemi Pine during a secret mission amid the Gospers Mountain megablaze. 1365 homes ha...
Southwest Airlines push Boeing 737 Max flight cancellations into June
Key lawmakers have said they may try to stop the FAA from letting Boeing do some inspections and safety analysis on its own planes. The biggest negative (-183) was in the 737 category, due to the th...
United Nations envoy slams Lebanon's leaders as protests rock Beirut
Meanwhile, banks have imposed informal capital controls, limiting withdrawal of dollars and foreign transfers in the country. The hours-long clashes also left 47 policemen injured, security forces sa...
EU Commission unveils €1trn green finance deal
The president of the European Commission (EC), Ursula von der Leyen, said: "The Green Deal comes with important investment needs, which we will turn into investment opportunities". In a document see...
Japan political star Shinjiro Koizumi has baby boy, his 1st child
He hopes to set an example for other fathers as the nation deals with a crisis of work-life balance, and declining birth rates. Koizumi said that society's attitude toward childcare needs to change....
Schumer says he believes Iran war powers resolution will pass Senate
Kaine named four Republican senators - Rand Paul , Mike Lee , Todd Young and Susan Collins - who he said will join the 47 Democrats voting in favor in the 100-member chamber. "We have to figur...
Trudeau: US partly to blame for plane being shot down
Saeed Maroof, the captain of Iran's national volleyball team, also wrote on Instagram: "I wish I could be hopeful that this was the last scene of the show of deceit and lack of wisdom of these inc...
Trump Wants to Reprogram $7.2B DoD Funds for Border Wall
The Pentagon authorized $3.6 billion in military construction funds from deferred projects overseas and planned projects in the US. The pending transfer, if not blocked by Congress or the courts, ...
'OK, Boomer' makes a Supreme Court appearance in age case
In case you're wondering whether the phrase "OK, boomer" is over yet, the chief justice of the United States has used it in a sentence. The attorney was hesitant to say so, only stating that the proce...
Prosecutors seek almost 5 years in prison for ex-Congressman
USA attorneys in the Southern District of NY filed their sentencing submission Monday, arguing Collins should receive a sentence on the high-end of his 46-to-57-month sentencing guidelines. Cameron Co...
MAB seeks clarity from aviation authorities on 737 MAX airworthiness
The "phase one" trade deal signed by the US and China Wednesday is expected to unblock an informal halt to Chinese orders for Boeing jets that had begun to affect the manufacturer's long-term producti...
US Slaps Sanctions on Lawmakers in Venezuela over Control of Parliament
The U.S. has announced fresh sanctions on Venezuelan officials after dissident opposition legislator Luis Parra had been sworn in as the new president of Venezuela's opposition-led National Assembly...
Taiwan elections do not change facts: China
In a thinly veiled threat, Ma cited the "increasing voices" within China expecting Beijing to step up its efforts to protect the "one China principle" through a process of "reunification through milit...
Cambodia begins treason trial against opposition leader despite global outcry
The video cited as evidence by the court was recorded in 2013 and shows Kem Sokha discussing a strategy to win power at the ballot box with the help of U.S. Rights group have called the trial politica...
Tourists to be deported over alleged damage, defecation at Machu Picchu
Since 1911 Machu Picchu has gone from obscurity to over a million annual visitors. The tourists were found by the park rangers and police on Sunday. A representative for Peru's Ministry of Culture...
Man asks court for 'Trial by Combat' with wife's attorney in divorce
He even specified the types of swords that could be used in the battle in the court filing. In response to Ostrom's trial by combat motion, Hudson filed a resistance that also corrected Ostrom's spe...
Donald Trump's former security advisor Michael Flynn seeks to withdraw guilty plea
Flynn ("Mr. Flynn") hereby moves to withdraw his plea because of the government's bad faith, vindictiveness, and breach of the plea agreement. The judge, Emmet Sullivan , ordered Mr Flynn to appear...
Virgin Islands suit alleges decades-long abuse by Epstein
Epstein carried out "an expansive scheme of human trafficking and sexual abuse of young women" and concealed it with an elaborate scheme of entities, said George, who took office in April. Those victi...
Human body parts found in north Dublin
Gardai have launched an investigation following the discovery of human body parts in north Dublin. The Dublin City Coroner has been notified, and the services of the state pathologist have been requ...
Russian prime minister submits resignation to President Vladimir Putin
The unexpected resignation means Russian Federation will get a new prime minister. Notably, the Russian Constitution doesn't allow anyone to serve as the country's president for more than two co...
Port Clinton teenager found dead in neighboring house
The abandoned home that Dilly's body was located in was across the street from his home. His maroon puffy coat and glasses, which he was wearing when he was last seen, were found on the floor near the...
At least 17 killed by extreme cold snap in Afghanistan
Meanwhile in western Pakistan, heavy snowfall in southwestern Balochistan destroyed several houses in the mountainous region, killing 17 people. As per the details, at least six persons of a family,...
Chemical alert issued after massive explosion in Tarragona, Spain
The fire was now contained, the spokesman said, adding that two people suffered severe burns, another victim sustained less severe burns and three had light injuries. Local trains between Tarragona an...
Terrifying moment as Australian firefighters drive through flames
Many stood in shallow water to escape the flames . Some of the worst loses were suffered in Conjola Park, where 89 homes were destroyed, and Malua Bay, where 40 homes were lost. Prime Minister Scott ...
Citizenship Act: Satya Nadella criticises new law, says, 'It's just bad'
Through a statement from Microsoft India's official Twitter handle, Nadella said that being shaped by Indian heritage and in view of his immigrant experience in the United States, he hopes to see an I...


Experience true Africa with open air dining and campfires under a crystal clear mountain sky.
The Gorillas you will track in Bwindi will belong either one of two 'habituated' family groups, the Mubare Group or the Habinyanja Group. For up to five years, each of these groups have undergone an extremely delicate process that has gradually made them used to the presence of humans, and allowed a few privileged visitors to interact with them briefly in the wild.
The Gorillas are by no means tame. They are completely wild animals, which even now tolerate human presence for an hour a day at most. Experienced guides will accompany you on your tracking, many of who have been involved in the habituation process themselves. These guides will brief you in detail on your arrival on the various aspects of 'Gorilla etiquette', but the information contained in this set of guidelines is to help you arrive for your trek well prepared and ready to enjoy this unique opportunity to the full.
Gorilla tracking is a year-round activity, with no season as such. The rain forest is moist, and it rains very often in Bwindi, even in the dry season. The trek commences every morning from the park headquarters at 0830 hours. There is a daily maximum of 6 visitors to each Gorilla group, and each group is accompanied by a guide and by porters who will carry your shoulder pack. The Gorillas cover large distances overnight, and they are never constantly in one area. The guides will use their knowledge of the Gorillas' habits and information from the previous day to locate the group's whereabouts.
Because of this, the time taken to track the Gorillas varies enormously, from as little as half an hour to as much as 9 hours before one returns to camp. The terrain is extremely difficult, which steep slopes (often steeper than a flight of stairs) covered in dense vegetation that gives the park its name. In addition, the altitude of 5200 feet and more means participants do need to be physically fit to enjoy the track. Once the Gorillas are located, your group will be allowed a maximum of one hour with them. After this, you will return to the park headquarters and your camp.
Only persons over the age of 15 are allowed to track the Gorillas.
The Gorilla Forest Camp offers a bar and dining cottage overlooking the rain forest, as well as a raised natural platform for open air dining. At night, a traditional African campfire is lit from where a lecture might be given by a visiting primatologist, the Gorilla briefings are undertaken or guests simply admire the myriad brilliance of Bwindi's crystal clear mountain sky.
In cases of light aircraft, scheduled flights or private charters, luggage is limited to 12.5 kilos in one soft bag.
Only 600 of the endangered Mountain Gorilla remain worldwide, and Bwindi National Park is home to just over half of them. Gorilla Forest Camp is situated here.
Gorilla Forest Camp in Bwindi Park is the ideal base for tracking of the endangered Mountain Gorilla, and is home to the rare Forest Elephant.