You will be shown an excerpt from a computer science scientific research paper, followed by a question about the paper. Please answer the question. Do not include any text in your response other than the answer.

Context: Pre-training with an auxiliary task is a common technique for deep neural networks.Indeed, it is the de facto standard step in natural language processing and computer vision to learn a powerful backbone networks such as BERT devlin2018bert  and ResNet he2016deep  leveraging large datasets such as BooksCorpus zhu2015aligning , English Wikipedia, and ImageNet deng2009imagenet .The models trained on the auxiliary task are often beneficial for the primary (target) task of interest.Despite the success of pre-training, few approaches have been generalized to graph-structured data due to their fundamental challenges.First, graph structure (e.g., the number of nodes/edges, and diameter) and its meaning can significantly differ between domains. So the model trained on an auxiliary task can harm generalization on the primary task, i.e., negative transfer pan2009survey .Also, many graph neural networks are transductive approaches. This often makes transfer learning between datasets inherently infeasible.So, pre-training on the target dataset has been proposed using auxiliary tasks: graph kernel  navarin2018pre , graph reconstruction zhang2020graph , and attribute masking  hu2020strategies . These assume that the auxiliary tasks for pre-training are carefully selected with substantial domain knowledge and expertise in graph characteristics to assist the primary task.Since most graph neural networks operate on homogeneous graphs, which have a single type of nodes and edges, the previous pre-training/auxiliary tasksare not specifically designed for heterogeneous graphs, which have multiple types of nodes and edges.Heterogeneous graphs commonly occur in real-world applications, for instance, a music dataset has multiple types of nodes (e.g., user, song, artist) and multiple types of relations (e.g., user-artist, song-film, song-instrument). Our framework SELAR is learning to learn a primary task with multiple auxiliary tasks to assist the primary task.This can be formally written asmin𝐰,Θ⁡𝔼⁢[ℒp⁢r⁢(𝐰∗⁢(Θ))](x,y)∼Dp⁢r⁢ s.t. ⁢𝐰∗⁢(Θ)=arg⁡min𝐰⁡𝔼⁢[ℒp⁢r+a⁢u⁢(𝐰;Θ)](x,y)∼Dp⁢r+a⁢u,\displaystyle\min_{\mathbf{w},\Theta}\;\;\underset{(x,y)\sim D^{pr}\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;}{\text{\large$\mathbb{E}$}\;\;\left[\;\;\mathcal{L}^{pr}(\mathbf{w}^{\ast}(\Theta))\;\;\right]}\;\;\text{ s.t. }\;\;\mathbf{w}^{\ast}(\Theta)=\operatorname*{\arg\!\min}_{\mathbf{w}}\underset{(x,y)\sim D^{pr+au}\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;\;}{\;\;\mathbb{E}\;\;\left[\;\;\mathcal{L}^{pr+au}(\mathbf{w};\Theta)\;\;\right]},roman_min start_POSTSUBSCRIPT bold_w , roman_Θ end_POSTSUBSCRIPT start_UNDERACCENT ( italic_x , italic_y ) ∼ italic_D start_POSTSUPERSCRIPT italic_p italic_r end_POSTSUPERSCRIPT end_UNDERACCENT start_ARG blackboard_E [ caligraphic_L start_POSTSUPERSCRIPT italic_p italic_r end_POSTSUPERSCRIPT ( bold_w start_POSTSUPERSCRIPT ∗ end_POSTSUPERSCRIPT ( roman_Θ ) ) ] end_ARG s.t. bold_w start_POSTSUPERSCRIPT ∗ end_POSTSUPERSCRIPT ( roman_Θ ) = start_OPERATOR roman_arg roman_min end_OPERATOR start_POSTSUBSCRIPT bold_w end_POSTSUBSCRIPT start_UNDERACCENT ( italic_x , italic_y ) ∼ italic_D start_POSTSUPERSCRIPT italic_p italic_r + italic_a italic_u end_POSTSUPERSCRIPT end_UNDERACCENT start_ARG blackboard_E [ caligraphic_L start_POSTSUPERSCRIPT italic_p italic_r + italic_a italic_u end_POSTSUPERSCRIPT ( bold_w ; roman_Θ ) ] end_ARG ,(2)where \mathcal{L}^{pr}(\cdot) is the primary task loss function to evaluate the trained model f(x;\mathbf{w}^{\ast}(\Theta)) on meta-data (a validation for meta-learning han2018coteaching ) D^{pr} and \mathcal{L}^{pr+au} is the loss function to train a model on training data D^{pr+au} with the primary and auxiliary tasks. To avoid cluttered notation, f, x, and y are omitted. Each task \mathcal{T}_{t} has N_{t} samples and \mathcal{T}_{0} and \{\mathcal{T}_{t}\}_{t=1}^{T} denote the primary and auxiliary tasks respectively.The proposed formulation in Eq. (2) learns how to assist the primary task by optimizing \Theta via meta-learning. The nested optimization problem given \Theta is a regular training with properly adjusted loss functions to balance the primary and auxiliary tasks. The formulation can be more specifically written as\displaystyle\min_{\mathbf{w},\Theta}∑i=1M01M0ℓ0(yi(0,m⁢e⁢t⁢a),f(xi(0,m⁢e⁢t⁢a);𝐰∗(Θ))\displaystyle\sum_{i=1}^{M_{0}}\frac{1}{M_{0}}\ell^{0}(y_{i}^{(0,meta)},f(x_{i}^{(0,meta)};\mathbf{w}^{\ast}(\Theta))∑ start_POSTSUBSCRIPT italic_i = 1 end_POSTSUBSCRIPT start_POSTSUPERSCRIPT italic_M start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT end_POSTSUPERSCRIPT divide start_ARG 1 end_ARG start_ARG italic_M start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT end_ARG roman_ℓ start_POSTSUPERSCRIPT 0 end_POSTSUPERSCRIPT ( italic_y start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT ( 0 , italic_m italic_e italic_t italic_a ) end_POSTSUPERSCRIPT , italic_f ( italic_x start_POSTSUBSCRIPT italic_i end_POSTSUBSCRIPT start_POSTSUPERSCRIPT ( 0 , italic_m italic_e italic_t italic_a ) end_POSTSUPERSCRIPT ; bold_w start_POSTSUPERSCRIPT ∗ end_POSTSUPERSCRIPT ( roman_Θ ) )(3)s.t.\displaystyle\mathbf{w}^{\ast}(\Theta)=\operatorname*{\arg\!\min}_{\mathbf{w}}\sum_{t=0}^{T}\sum_{i=1}^{N_{t}}\frac{1}{N_{t}}\mathcal{V}(\xi^{(t,train)}_{i};\Theta)\ell^{t}(y_{i}^{(t,train)},f^{t}(x_{i}^{(t,train)};\mathbf{w})),(4)where \ell^{t} and f^{t} denote the loss function and the model for task t. We overload \ell^{t} with its function value, i.e., \ell^{t}=\ell^{t}(y_{i}^{(t,train)},f^{t}(x_{i}^{(t,train)};\mathbf{w})). \xi^{(t,train)}_{i} is the embedding vector of i_{th} sample for task t. It is the concatenation of one-hot representation of task types, the label of the sample (positive/negative), and its loss value, i.e., \xi^{(t,train)}_{i}=\left[\ell^{t};e_{t};y_{i}^{(t,train)}\right]\in\textbf{R}^{T+2}.To derive our learning algorithm,we first shorten the objective function in Eq. (3) and Eq. (4) as \mathcal{L}^{pr}(\mathbf{w}^{\ast}(\Theta)) and \mathcal{L}^{pr+au}(\mathbf{w};\Theta).This is equivalent to Eq. (2) without expectation.Then, our formulation is given as\min_{\mathbf{w},\Theta}\mathcal{L}^{pr}(\mathbf{w}^{\ast}(\Theta))\;\;\text{ s.t. }\mathbf{w}^{\ast}(\Theta)=\operatorname*{\arg\!\min}_{\mathbf{w}}\mathcal{L}^{pr+au}(\mathbf{w};\Theta),(5)To circumvent the difficulty of the bi-level optimization, as previous works MAML ; han2018coteaching  in meta-learning we approximate it with the updated parameters \hat{\mathbf{w}} using the gradient descent update as\displaystyle\mathbf{w}^{\ast}(\Theta)\approx\hat{\mathbf{w}}^{k}(\Theta^{k})=\mathbf{w}^{k}-\alpha\nabla_{\mathbf{w}}\mathcal{L}^{pr+au}(\mathbf{w}^{k};\Theta^{k}),(6)where \alpha is the learning rate for \mathbf{w}.We do not numerically evaluate \hat{\mathbf{w}}^{k}(\Theta) instead we plug the computational graph of \hat{\mathbf{w}}^{k} in \mathcal{L}^{pr}(\mathbf{w}^{\ast}(\Theta)) to optimize \Theta.Let \nabla_{\Theta}\mathcal{L}^{pr}(\mathbf{w}^{\ast}(\Theta^{k})) be the gradient evaluated at \Theta^{k}.Then updating parameters \Theta is given as\displaystyle\Theta^{k+1}=\Theta^{k}-\beta\nabla_{\Theta}\mathcal{L}^{pr}(\hat{\mathbf{w}}^{k}(\Theta^{k})),(7)where \beta is the learning rate for \Theta. This update allows softly selecting useful auxiliary tasks (meta-paths) and balance them with the primary task to improve the performance of the primary task. Without balancing tasks with the weighting function \mathcal{V}(\cdot;\Theta), auxiliary tasks can dominate training and degrade the performance of the primary task.

Question: Why does negative transfer occur when learning with auxiliary tasks?
Negative transfer happens when the learning of an auxiliary task negatively impacts the performance of the primary task. In the case of graph-based tasks, it can happen because the graph structure, such as the number of nodes, edges, and diameter, can be vastly different between domains. This causes confusion for the model, resulting in poor generalization of the primary task.