text:
News Biology 14 November 2018
Krill another day: Antarctic crustaceans more resilient than thought
Adult krill appear unharmed by upticks in ocean acidification. Nick Carne reports.
One in a million: an Antarctic krill.
cunfek/Getty Images
Antarctic krill may be more resilient in the face of climate change than we have assumed.
New Australian research suggests the small crustaceans, which belong to several species and comprise the order Euphausiacea, are largely unaffected by even significant levels of ocean acidification.
In a laboratory study conducted by scientists at Australia's University of Tasmania, they were able to survive, grow, store fat, mature and maintain respiration rates when exposed for up to a year to acidification levels that can be expected this century and beyond.
The caveats are that the research focussed only on adults, so other life stages may be more vulnerable, and that the persistence of krill will depend on how they respond to acidification in synergy with other stressors, such as ocean warming and decreases in sea ice extent.
Nevertheless, it's an important finding. Krill are the primary prey of many Antarctic marine mammals and seabirds, and thus a key link in the Antarctic food chain. One species, Euphausia superba, comprises an estimated 379 million tonnes of biomass in the region.
Previous short-term studies have indicated that they may be more vulnerable to ocean acidification than crustaceans from lower latitudes.
As international researchers noted in a major 2012 paper looking at the impact of climate change on Antarctic krill, we need "a better understanding of the resilience and the genetic plasticity of krill life stages".
The new study was carried out in the university's Institute for Marine and Antarctic Studies and reported in a paper published in the journal Communications Biology.
"We reared adult krill in laboratory tanks for 46 weeks in seawater with a range of pH levels, including those in the present day, levels predicted within 100 to 300 years, and up to an extreme level," says lead author Jess Ericson.
"We measured a suite of physiological and biochemical variables to investigate how future ocean acidification may affect the survival, size, lipid stores, reproduction, metabolism and extracellular fluid of krill.
"Our results showed that their physiological processes were largely unaffected by pH levels that they are expected to encounter over the coming century."
Increasing anthropogenic carbon dioxide emissions are causing ocean acidification, as absorption of atmospheric carbon dioxide in seawater alters ocean chemistry.
Ocean acidification increases mortality and negatively affects physiological functioning in some marine invertebrates, causing decreased mineralisation or dissolution of calcium carbonate shells, decreased or delayed growth, increased mortality and delayed reproduction or abnormalities in offspring.
A US study in 2011 argued that changes in the abundance of krill were responsible for increases and decreases in penguin numbers in the Antarctic.
In the same year, another group of Tasmanian researchers reported that both environmental change and human activity were having an impact on krill populations, with increased interest in products derived from krill.
Explore #krill #marine life #marine biology #Antarctica #ecosystems
Nick Carne is editor of Cosmos digital and editorial manager for The Royal Institution of Australia.
https://www.int-res.com/articles/feature/m458p001.pdf
http://www.pnas.org/content/108/18/7625.short
https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-2979.2011.00406.x
Eye colour more complex than we thought
Eye colour is controlled by many genes, according to a new Australian study that challenges the textbook example of blue eye colour as a simple 'recessive trait'.
Comets are bigger and more numerous than we thought
There are more comets in the outer reaches of the solar system than astronomers previously believed
Galaxy clusters much more common than thought
Data mining exercise reveals a whole new class of astronomical structure.
Australia has more large, scary spiders than previously thought


I thought it would be great fun and a help to my blog to share old content, but not just any old content. Each week I will share an old blog post from a previous year, any year, BUT it has to be from the current week (for example, something I've posted around April 21st, from any previous year) I will go in, edit the post if needed, add a pinnable image if I don't already have one, and share it on Facebook.
Oh my god! I remember this game I love it! It was fun playing it with y'all. I miss helping you review things with the children. Most of them were great.


Find a galaxy full of LEGO® Star Wars(TM) ideas to build--from activities and art, games and challenges, to practical makes and your very own inventions!
This book features 200 imaginative play and building ideas, from LEGO projects that take just a few minutes and a handful of bricks, to builds to keep you occupied for hours. Set up a game of LEGO Star Wars skittles, create a pencil pot shaped like your favorite Sith or Jedi, build a fleet of tiny TIEs, design Darth Vader's family tree, and customize a podracer. Challenge a friend to take on a rathtar, mix up your minifigures, stage your favorite movie battle scene with LEGO StarWars bricks--and much, much more! LEGO® Star Wars(TM) Ideas Book is packed with a family friendly range of activities to inspire every LEGO Star Wars fan. What will you build?
LEGO, the LEGO logo, the Minifigure and the Brick and Knob configurations are trademarks of the LEGO Group.
All rights reserved. ©2017 The LEGO Group. Manufactured by Dorling Kindersley, Strand 80, London, WC2R 0RL under licence from the LEGO Group. © & TM 2017 LUCASFILM LTD. All rights reserved. Used under authorization.
Elizabeth Dowsett is the author of LEGO Star Wars: Revenge of the Sith, LEGO Star Wars: Attack of the Clones, and more.


Algebra is a branch of mathematics where we study the representation of abstract concepts in letter or symbol form, and these forms are called pronumerals.
It involves grouping and collecting like pro numerals to do the four basic operations in mathematics, viz. addition, subtraction, multiplication and division. We can solve equations, inequations, and quadratic polynomials through different methods – computation method and graphical methods.
It also involves looking at patterns and relationships between different variables; analysing the patterns to find a rule which these variables follow.
Now, what does Algebra mean?
Algebra is a representation and simplification of algebraic expressions, index notations, and their linear relationships. Knowledge of algebra will enable you to make complex calculations in the field of science and commerce.
Algebra has wide applications in diverse fields such as engineering, medical science, computer programming. Knowledge of problem solving using matrices, logarithms, functions and graphical representation of physical phenomena are all important and useful in these fields.


thinks sucky female artiste now making a roadrunner cloud away from her own "Everybody Draw Mohammed Day" idea.
"I made a cartoon that went viral and I am not going with it. Many other folks have used my cartoon to start sites, etc. Please go to them as I am a private person who draws stuff," she writes.
It went viral, however, because she was the one who passed it around.
Then why did she send the cartoon to the media in the first place? "Because I'm an idiot," Norris replied.
"This particular cartoon of a 'poster' seems to have struck a gigantic nerve, something I was totally unprepared for," she said.
I think Muslims should punish her for getting Mohammed's name wrong.
She doesn't appear to be alone. The creator of a Facebook page dedicated to the day has bowed out as well.
Apparently the posts weren't "welcoming" enough, as on Sunday morning he announced his departure from the cause.
"I am aghast that so many people are posting deeply offensive pictures of the Prophet," he writes.
How do I know he's a liberal, besides his wimpiness and stupidity, that is?
Edith Efron: where has she been all my life?


text:
"""Tests for rnn_layers."""

import types
from absl.testing import parameterized
import lingvo.compat as tf
from lingvo.core import attention
from lingvo.core import base_layer
from lingvo.core import cluster_factory
from lingvo.core import py_utils
from lingvo.core import rnn_cell
from lingvo.core import rnn_layers
from lingvo.core import test_utils
import numpy as np

from tensorflow.python.ops import inplace_ops

FLAGS = tf.flags.FLAGS

class TimestepAccumulator(base_layer.Accumulator):
  """Simple accumulator for counting timesteps."""

  def DefaultValue(self):
    return tf.convert_to_tensor(0.0)

  def Increment(self):
    self.SetValue(self.GetValue() + 1.0)

def AddTimestepAccumulator(layer):
  orig_fprop = layer.FProp

  def WrappedFProp(*args, **kwargs):
    layer.accumulators.ts_count.Increment()
    return orig_fprop(*args, **kwargs)

  layer.FProp = WrappedFProp

  layer.RegisterAccumulator('ts_count', TimestepAccumulator())

class LayersTestBase(test_utils.TestCase):

  def _testStackedFRNNHelper(self,
                             cls,
                             dtype,
                             trailing_pad_len=0,
                             keep_prob=1.0,
                             bi_directional=False,
                             input_dim=-1,
                             output_dim=-1):
    batch = 3
    dims = 16
    slen = 10 + trailing_pad_len
    num_layers = 4

    params = rnn_cell.LSTMCellSimple.Params()
    params.name = 'lstm'
    params.output_nonlinearity = True
    params.dtype = dtype
    params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
    params.vn.global_vn = True
    params.vn.per_step_vn = False
    params.vn.seed = 2938482
    params.vn.scale = 0.1
    # StackedFRNNParallel executes cells in a different order, causing the
    # deterministic random to not match, so we set deterministic = False for
    # comparing.
    params.vn.deterministic = False
    params.num_input_nodes = dims
    params.num_output_nodes = dims // 2 if bi_directional else dims

    sfrnn_params = cls.Params()
    sfrnn_params.name = 'sfrnn'
    sfrnn_params.dtype = dtype
    sfrnn_params.random_seed = 123456
    sfrnn_params.cell_tpl = params
    sfrnn_params.num_layers = num_layers
    sfrnn_params.skip_start = 2
    sfrnn_params.dropout.keep_prob = keep_prob
    sfrnn_params.num_input_nodes = input_dim
    sfrnn_params.num_output_nodes = output_dim

    np.random.seed(12345)
    input_dim = input_dim if input_dim > 0 else dims
    output_dim = output_dim if output_dim > 0 else dims
    inputs = np.random.uniform(size=(slen, batch, input_dim))
    paddings = np.zeros([slen, batch, 1])
    if trailing_pad_len > 0:
      paddings[-trailing_pad_len:, :] = 1.0
      paddings[-trailing_pad_len - 3:-trailing_pad_len - 1, :] = 1.0

    with self.session(use_gpu=True, graph=tf.Graph()):
      sfrnn = sfrnn_params.Instantiate()
      self.evaluate(tf.global_variables_initializer())

      if bi_directional:
        sfrnn_outputs = sfrnn.FPropFullSequence(sfrnn.theta,
                                                tf.constant(inputs, dtype),
                                                tf.constant(paddings, dtype))
        sfrnn_outputs = py_utils.HasShape(sfrnn_outputs,
                                          [slen, batch, output_dim])
        return self.evaluate(sfrnn_outputs)
      else:
        sfrnn_outputs, sfrnn_final = sfrnn.FPropDefaultTheta(
            tf.constant(inputs, dtype), tf.constant(paddings, dtype))
        sfrnn_outputs = py_utils.HasShape(sfrnn_outputs,
                                          [slen, batch, output_dim])
        return self.evaluate([sfrnn_outputs, sfrnn_final])

  def _testStackedFRNNGradHelper(self, cls, bi_directional=False):
    trailing_pad_len = 2
    dtype = tf.float64
    batch = 3
    dims = 16
    slen = 10 + trailing_pad_len
    num_layers = 4
    with self.session(use_gpu=True, graph=tf.Graph()) as sess:
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'lstm'
      params.output_nonlinearity = True
      params.dtype = dtype
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.vn.global_vn = False
      params.vn.per_step_vn = False
      params.num_input_nodes = dims
      params.num_output_nodes = dims // 2 if bi_directional else dims

      sfrnn_params = cls.Params()
      sfrnn_params.name = 'sfrnn'
      sfrnn_params.dtype = dtype
      sfrnn_params.cell_tpl = params
      sfrnn_params.num_layers = num_layers
      sfrnn_params.skip_start = 2
      with tf.name_scope('sfrnn'):
        sfrnn = sfrnn_params.Instantiate()

      np.random.seed(12345)
      inputs = tf.constant(np.random.uniform(size=(slen, batch, dims)), dtype)
      paddings = np.zeros([slen, batch, 1])
      paddings[-trailing_pad_len:, :] = 1.0
      paddings[-trailing_pad_len - 3:-trailing_pad_len - 1, :] = 1.0
      paddings = tf.constant(paddings, dtype)

      if bi_directional:
        sfrnn_outputs = sfrnn.FPropDefaultTheta(inputs, paddings)
        loss = tf.reduce_sum(sfrnn_outputs)
      else:
        sfrnn_outputs, sfrnn_final = sfrnn.FPropDefaultTheta(inputs, paddings)
        loss = tf.reduce_sum(sfrnn_outputs)
        for fin in sfrnn_final.rnn:
          loss += tf.reduce_sum(fin.m) + tf.reduce_sum(fin.c)
      xs = sfrnn.vars.Flatten() + [inputs]
      dxs = tf.gradients(loss, xs)

      # Compares the sym grad against the numeric grads.
      self.evaluate(tf.global_variables_initializer())
      grad_step = 17
      sym_grads = self.evaluate(dxs)
      sym_grads = [test_utils.PickEveryN(_, grad_step) for _ in sym_grads]
      num_grads = [
          test_utils.PickEveryN(
              test_utils.ComputeNumericGradient(
                  sess, loss, v, delta=1e-4, step=grad_step), grad_step)
          for v in xs
      ]
      for (sym, num) in zip(sym_grads, num_grads):
        self.assertFalse(np.any(np.isnan(sym)))
        self.assertFalse(np.any(np.isnan(num)))
        print('max = ', np.max(np.abs(sym)))
        self.assertAllClose(sym, num)

class LayersTest(LayersTestBase, parameterized.TestCase):

  def testIdentitySeqLayer(self):
    with self.session(use_gpu=False):
      rnn_params = rnn_layers.IdentitySeqLayer.Params()
      rnn_params.name = 'no_op'
      rnn = rnn_params.Instantiate()

      np.random.seed(12345)
      inputs_sequence = []
      paddings_sequence = []
      for _ in range(5):
        inputs_sequence.append(
            tf.constant(np.random.uniform(size=(3, 2)), tf.float32))
        paddings_sequence.append(tf.zeros([3, 1]))

      paddings_sequence[-1] = tf.constant([[1.0], [1.0], [1.0]])
      paddings_sequence[-2] = tf.constant([[1.0], [1.0], [1.0]])

      inputs, paddings = tf.stack(inputs_sequence), tf.stack(paddings_sequence)
      outputs = rnn.FPropFullSequence(rnn.theta, inputs, paddings)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())

      inputs_v, outputs_v = self.evaluate([inputs, outputs])
      self.assertAllEqual(inputs_v, outputs_v)

  def testRNN(self):
    with self.session(use_gpu=False):
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.num_input_nodes = 2
      params.num_output_nodes = 2

      rnn_params = rnn_layers.RNN.Params()
      rnn_params.name = 'rnn'
      rnn_params.vn.global_vn = True
      rnn_params.vn.per_step_vn = False
      rnn_params.vn.seed = 2938482
      rnn_params.vn.scale = 0.1
      rnn_params.cell = params
      rnn_params.sequence_length = 10
      rnn = rnn_layers.RNN(rnn_params)

      np.random.seed(12345)
      inputs_sequence = []
      paddings_sequence = []
      for _ in range(rnn_params.sequence_length):
        inputs_sequence.append(
            tf.constant(np.random.uniform(size=(3, 2)), tf.float32))
        paddings_sequence.append(tf.zeros([3, 1]))

      paddings_sequence[-1] = tf.constant([[1.0], [1.0], [1.0]])
      paddings_sequence[-2] = tf.constant([[1.0], [1.0], [1.0]])

      inputs, paddings = tf.stack(inputs_sequence), tf.stack(paddings_sequence)
      outputs, final = rnn.FPropDefaultTheta(inputs, paddings)

      outputs *= paddings
      sum_outputs = tf.reduce_sum(outputs, axis=0)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())

      actual = self.evaluate(py_utils.NestedMap(sum=sum_outputs, **final))

      sum_expected = [[-0.366546, 0.788136], [-0.372402, 0.54518],
                      [-0.367016, 0.674535]]
      m_expected = [[-0.18327278, 0.39406797], [-0.18620083, 0.27259],
                    [-0.18350805, 0.33726758]]
      c_expected = [[-0.46244526, 0.8282993], [-0.30769998, 0.7895613],
                    [-0.31077483, 0.8967643]]
      self.assertAllClose(sum_expected, actual.sum)
      self.assertAllClose(m_expected, actual.m)
      self.assertAllClose(c_expected, actual.c)

  def testRNNGradientChecker(self):
    with self.session(use_gpu=False) as sess:
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.num_input_nodes = 2
      params.num_output_nodes = 2

      rnn_params = rnn_layers.RNN.Params()
      rnn_params.name = 'rnn'
      rnn_params.dtype = tf.float64
      rnn_params.vn.global_vn = False
      rnn_params.vn.per_step_vn = False
      rnn_params.cell = params
      rnn_params.sequence_length = 10
      rnn = rnn_layers.RNN(rnn_params)

      np.random.seed(12345)
      inputs_sequence = []
      paddings_sequence = []
      for _ in range(rnn_params.sequence_length):
        inputs_sequence.append(
            tf.constant(np.random.uniform(size=(3, 2)), tf.float64))
        paddings_sequence.append(tf.zeros([3, 1], dtype=tf.float64))

      paddings_sequence[-1] = tf.constant(
          [[1.0], [1.0], [1.0]], dtype=tf.float64)
      paddings_sequence[-2] = tf.constant(
          [[1.0], [1.0], [1.0]], dtype=tf.float64)

      inputs, paddings = tf.stack(inputs_sequence), tf.stack(paddings_sequence)
      outputs, final = rnn.FPropDefaultTheta(inputs, paddings)
      outputs *= paddings
      sum_outputs = tf.reduce_sum(outputs, axis=0)

      loss = tf.reduce_sum(sum_outputs) + tf.reduce_sum(final.m + final.c)
      all_vars = py_utils.Flatten(rnn.vars)
      assert len(all_vars) == 2

      grads = tf.gradients(loss, all_vars)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())

      symbolic_grads = [gd.eval() for gd in grads]
      numerical_grads = []
      for v in all_vars:
        numerical_grads.append(test_utils.ComputeNumericGradient(sess, loss, v))
      for x, y in zip(symbolic_grads, numerical_grads):
        self.assertAllClose(x, y)

  def testRNNReversed(self):
    """Test an RNN layer with reverse=true.

    This should yield the same output as feeding reversed input into
    the same RNN with reverse=false (except the output is in reversed order).
    """

    timesteps = 10
    padding_steps = 2
    batch_size = 2
    depth = 3
    with self.session(use_gpu=True):
      lstm_params = rnn_cell.LSTMCellSimple.Params()
      lstm_params.output_nonlinearity = True
      lstm_params.num_input_nodes = depth
      lstm_params.num_output_nodes = depth

      rnn_params = rnn_layers.RNN.Params()
      rnn_params.vn.global_vn = False
      rnn_params.vn.per_step_vn = False
      rnn_params.cell = lstm_params
      rnn_params.sequence_length = timesteps

      fwd_rnn_params = rnn_params.Copy()
      fwd_rnn_params.name = 'fwd'
      fwd_rnn_params.cell.name = 'fwd_lstm'
      fwd_rnn = rnn_layers.RNN(fwd_rnn_params)

      bak_rnn_params = rnn_params.Copy()
      bak_rnn_params.name = 'bak'
      bak_rnn_params.cell.name = 'bak_lstm'
      bak_rnn_params.reverse = True
      bak_rnn = rnn_layers.RNN(bak_rnn_params)

      # Create 8 timesteps of random input, 2 timesteps of zeros, and paddings
      # to match.
      fwd_inputs = tf.constant(
          np.concatenate(
              (np.random.uniform(
                  size=(timesteps - padding_steps, batch_size, depth)),
               np.zeros(shape=(padding_steps, batch_size, depth))),
              axis=0), tf.float32)
      fwd_paddings = tf.concat(
          (tf.zeros(shape=(timesteps - padding_steps, batch_size, depth)),
           tf.ones(shape=(padding_steps, batch_size, depth))),
          axis=0)
      bak_inputs = tf.reverse(fwd_inputs, [0])
      bak_paddings = tf.reverse(fwd_paddings, [0])

      # Run the forward rnn with reversed inputs
      reversed_outputs, _ = fwd_rnn.FProp(fwd_rnn.theta, bak_inputs,
                                          bak_paddings)
      reversed_outputs = tf.reverse(reversed_outputs, [0])

      # Run the backward rnn with forward inputs.  Note we reuse the fwd_rnn
      # theta so the results should match
      bak_outputs, _ = bak_rnn.FProp(fwd_rnn.theta, fwd_inputs, fwd_paddings)

      self.evaluate(tf.global_variables_initializer())
      actual_reversed_outputs, actual_bak_outputs = self.evaluate(
          [reversed_outputs, bak_outputs])
      self.assertAllClose(actual_reversed_outputs, actual_bak_outputs)

  def testRNNWithConvLSTMCell(self):
    with self.session(use_gpu=False):
      params = rnn_cell.ConvLSTMCell.Params()
      params.name = 'conv_lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.inputs_shape = [None, 4, 2, 3]
      params.cell_shape = [None, 4, 2, 2]
      params.filter_shape = [3, 2]

      rnn_params = rnn_layers.RNN.Params()
      rnn_params.name = 'rnn'
      rnn_params.vn.global_vn = True
      rnn_params.vn.per_step_vn = False
      rnn_params.vn.seed = 2938482
      rnn_params.vn.scale = 0.1
      rnn_params.cell = params
      rnn_params.sequence_length = 10
      rnn = rnn_layers.RNN(rnn_params)

      np.random.seed(12345)
      inputs_sequence = []
      paddings_sequence = []
      for _ in range(rnn_params.sequence_length):
        inputs_sequence.append(
            tf.constant(np.random.uniform(size=(3, 4, 2, 3)), tf.float32))
        paddings_sequence.append(tf.zeros([3, 1]))

      paddings_sequence[-1] = tf.constant([[1.0], [1.0], [1.0]])
      paddings_sequence[-2] = tf.constant([[1.0], [1.0], [1.0]])

      inputs, paddings = tf.stack(inputs_sequence), tf.stack(paddings_sequence)
      outputs, final = rnn.FPropDefaultTheta(inputs, paddings)
      outputs *= tf.reshape(1.0 - paddings,
                            [rnn_params.sequence_length, -1, 1, 1, 1])

      sum_outputs = tf.reduce_sum(tf.reduce_sum(outputs, axis=0), [1, 2, 3])
      sum_final_m = tf.reduce_sum(final.m, [1, 2, 3])
      sum_final_c = tf.reduce_sum(final.c, [1, 2, 3])

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      actual = self.evaluate(
          py_utils.NestedMap(sum=sum_outputs, m=sum_final_m, c=sum_final_c))

      print('sum_outputs', np.array_repr(actual.sum))
      print('final_m', np.array_repr(actual.m))
      print('final_c', np.array_repr(actual.c))

      sum_expected = [2.999277, 2.291916, -0.528728]
      m_expected = [0.52095217, 0.59586865, 0.27472413]
      c_expected = [2.4646373, 2.2752855, 1.9296129]
      self.assertAllClose(sum_expected, actual.sum)
      self.assertAllClose(m_expected, actual.m)
      self.assertAllClose(c_expected, actual.c)

  def testFRNNWithConvLSTMCell(self):
    params = rnn_cell.ConvLSTMCell.Params()
    params.name = 'conv_lstm'
    params.output_nonlinearity = True
    params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
    params.inputs_shape = [None, 4, 2, 3]
    params.cell_shape = [None, 4, 2, 2]
    params.filter_shape = [3, 2]

    rnn_params = rnn_layers.RNN.Params()
    rnn_params.name = 'rnn'
    rnn_params.vn.global_vn = True
    rnn_params.vn.per_step_vn = False
    rnn_params.vn.seed = 2938482
    rnn_params.vn.scale = 0.1
    rnn_params.cell = params
    rnn_params.sequence_length = 10

    frnn_params = rnn_layers.FRNN.Params()
    frnn_params.name = 'frnn'
    frnn_params.cell = params
    frnn_params.vn = rnn_params.vn

    np.random.seed(12345)
    inputs_sequence = []
    paddings_sequence = []
    for _ in range(rnn_params.sequence_length):
      inputs_sequence.append(
          np.random.uniform(size=(3, 4, 2, 3)).astype(np.float32))
      paddings_sequence.append(np.zeros([3, 1], dtype=np.float32))
    paddings_sequence[-1] = [[1.0], [1.0], [1.0]]
    paddings_sequence[-2] = [[1.0], [1.0], [1.0]]

    with self.session(graph=tf.Graph(), use_gpu=True):
      rnn = rnn_params.Instantiate()
      self.evaluate(tf.global_variables_initializer())

      inputs, paddings = tf.stack(inputs_sequence), tf.stack(paddings_sequence)
      outputs, final = rnn.FPropDefaultTheta(inputs, paddings)
      outputs *= tf.reshape(1.0 - paddings,
                            [rnn_params.sequence_length, -1, 1, 1, 1])
      rnn_outputs_v, rnn_final_v = self.evaluate([outputs, final])

    with self.session(graph=tf.Graph(), use_gpu=True):
      frnn = frnn_params.Instantiate()
      self.evaluate(tf.global_variables_initializer())

      frnn_outputs, frnn_final = frnn.FPropDefaultTheta(
          tf.stack(inputs_sequence), tf.stack(paddings_sequence))
      paddings = tf.stack(paddings_sequence)
      frnn_outputs *= tf.reshape(1.0 - paddings,
                                 tf.concat([tf.shape(paddings), [1, 1]], 0))
      frnn_outputs_v, frnn_final_v = self.evaluate([frnn_outputs, frnn_final])

    self.assertAllClose(rnn_outputs_v, frnn_outputs_v)
    self.assertAllClose(rnn_final_v.m, frnn_final_v.m)
    self.assertAllClose(rnn_final_v.c, frnn_final_v.c)

  def testRNNWithConvLSTMCellGradientChecker(self):
    with self.session(use_gpu=True) as sess:
      params = rnn_cell.ConvLSTMCell.Params()
      params.name = 'conv_lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.inputs_shape = [None, 4, 2, 3]
      params.cell_shape = [None, 4, 2, 2]
      params.filter_shape = [3, 2]

      rnn_params = rnn_layers.RNN.Params()
      rnn_params.name = 'rnn'
      rnn_params.vn.global_vn = False
      rnn_params.vn.per_step_vn = False
      rnn_params.cell = params
      rnn_params.sequence_length = 10
      rnn = rnn_layers.RNN(rnn_params)

      np.random.seed(12345)
      inputs_sequence = []
      paddings_sequence = []
      for _ in range(rnn_params.sequence_length):
        inputs_sequence.append(
            tf.constant(np.random.uniform(size=(3, 4, 2, 3)), tf.float32))
        paddings_sequence.append(tf.zeros([3, 1]))

      paddings_sequence[-1] = tf.constant([[1.0], [1.0], [1.0]])
      paddings_sequence[-2] = tf.constant([[1.0], [1.0], [1.0]])

      inputs, paddings = tf.stack(inputs_sequence), tf.stack(paddings_sequence)
      outputs, final = rnn.FPropDefaultTheta(inputs, paddings)
      outputs *= tf.reshape(1.0 - paddings,
                            [rnn_params.sequence_length, -1, 1, 1, 1])
      loss = tf.reduce_sum(tf.reduce_sum(
          outputs, axis=0)) + tf.reduce_sum(final.m + final.c)
      all_vars = tf.trainable_variables()
      assert len(all_vars) == 2

      grads = tf.gradients(loss, all_vars)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())

      symbolic_grads = [gd.eval() for gd in grads]
      numerical_grads = []
      for v in all_vars:
        numerical_grads.append(test_utils.ComputeNumericGradient(sess, loss, v))
      for x, y in zip(symbolic_grads, numerical_grads):
        self.assertAllClose(x, y, rtol=0.1, atol=0.1)

  def _testFRNNWithConvLSTMCellGradientChecker(self):
    with self.session(use_gpu=True) as sess:
      params = rnn_cell.ConvLSTMCell.Params()
      params.name = 'conv_lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.inputs_shape = [None, 4, 2, 3]
      params.cell_shape = [None, 4, 2, 2]
      params.filter_shape = [3, 2]

      frnn_params = rnn_layers.FRNN.Params()
      frnn_params.name = 'rnn'
      frnn_params.vn.global_vn = False
      frnn_params.vn.per_step_vn = False
      frnn_params.cell = params
      frnn = rnn_layers.FRNN(frnn_params)

      np.random.seed(12345)
      inputs_sequence = tf.constant(
          np.random.uniform(size=(10, 3, 4, 2, 3)), tf.float32)
      paddings = inplace_ops.inplace_update(
          tf.zeros([10, 3, 1]), 1, [[1.0], [0.0], [1.0]])

      outputs, final = frnn.FPropDefaultTheta(inputs_sequence, paddings)
      outputs *= tf.reshape(paddings, [10, 3, 1, 1, 1])
      loss = tf.reduce_sum(outputs) + tf.reduce_sum(final.m + final.c)
      all_vars = tf.trainable_variables()
      assert len(all_vars) == 2

      grads = tf.gradients(loss, all_vars)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())

      symbolic_grads = [gd.eval() for gd in grads]
      numerical_grads = []
      for v in all_vars:
        numerical_grads.append(test_utils.ComputeNumericGradient(sess, loss, v))
      for x, y in zip(symbolic_grads, numerical_grads):
        self.assertAllClose(x, y, rtol=0.1, atol=0.1)

  def testFRNNWithConvLSTMCellGradientChecker(self):
    self._testFRNNWithConvLSTMCellGradientChecker()

  def testFRNNWithLSTMCellSimpleGradientChecker(self):
    with self.session(use_gpu=True) as sess:
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'conv_lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.deterministic = True
      params.zo_prob = 0.25
      params.num_input_nodes = 4
      params.num_output_nodes = 6
      params.dtype = tf.float64

      frnn_params = rnn_layers.FRNN.Params()
      frnn_params.dtype = tf.float64
      frnn_params.name = 'rnn'
      frnn_params.cell = params
      old_enable_asserts = FLAGS.enable_asserts
      FLAGS.enable_asserts = False
      frnn = rnn_layers.FRNN(frnn_params)
      FLAGS.enable_asserts = old_enable_asserts

      np.random.seed(12345)
      inputs_sequence = tf.constant(
          np.random.uniform(size=(10, 3, 4)), tf.float64)
      paddings = inplace_ops.inplace_update(
          tf.zeros([10, 3, 1], tf.float64), 1, [[1.0], [0.0], [1.0]])

      outputs, _ = frnn.FPropDefaultTheta(inputs_sequence, paddings)
      outputs *= (1.0 - tf.reshape(paddings, [10, 3, 1]))
      loss = tf.reduce_sum(outputs)

      all_vars = tf.trainable_variables()
      assert len(all_vars) == 2

      grads = tf.gradients(loss, all_vars)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())

      symbolic_grads = [gd.eval() for gd in grads]
      numerical_grads = []
      for v in all_vars:
        numerical_grads.append(test_utils.ComputeNumericGradient(sess, loss, v))
      for x, y in zip(symbolic_grads, numerical_grads):
        self.assertAllClose(x, y, rtol=0.00001, atol=0.00001)

  @parameterized.parameters((True, True, True), (True, False, True),
                            (True, False, False), (False, True, True),
                            (False, False, True), (False, False, False))
  def testFRNN(self, inline, per_step_vn, deterministic):
    config = py_utils.SessionConfig(inline=inline)
    dtype = tf.float32
    batch = 3
    dims = 16
    slen = 10

    params = rnn_cell.LSTMCellSimple.Params()
    params.name = 'lstm'
    params.output_nonlinearity = True
    params.dtype = dtype
    params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
    params.vn.global_vn = True
    params.vn.per_step_vn = per_step_vn
    params.vn.seed = 2938482
    params.vn.scale = 0.1
    params.vn.deterministic = deterministic
    params.num_input_nodes = dims
    params.num_output_nodes = dims

    frnn_params = rnn_layers.FRNN.Params()
    frnn_params.name = 'frnn'
    frnn_params.dtype = dtype
    frnn_params.cell = params

    rnn_params = rnn_layers.RNN.Params()
    rnn_params.name = 'rnn'
    rnn_params.dtype = dtype
    rnn_params.sequence_length = slen
    rnn_params.cell = params

    np.random.seed(12345)
    inputs = np.random.uniform(size=(slen, batch, dims)).astype(np.float32)
    paddings = np.zeros([slen, batch, 1], dtype=np.float32)
    paddings[-3:-1, :] = 1.0

    with self.session(graph=tf.Graph(), use_gpu=True, config=config):
      frnn = frnn_params.Instantiate()
      self.evaluate(tf.global_variables_initializer())
      AddTimestepAccumulator(frnn.cell)
      frnn_out, frnn_final = self.evaluate(
          frnn.FPropDefaultTheta(tf.constant(inputs), tf.constant(paddings)))

    with self.session(graph=tf.Graph(), use_gpu=True, config=config):
      rnn = rnn_params.Instantiate()
      self.evaluate(tf.global_variables_initializer())
      rnn_out, rnn_final = self.evaluate(
          rnn.FPropDefaultTheta(tf.unstack(inputs), tf.unstack(paddings)))

    self.assertAllClose(frnn_out, rnn_out)
    self.assertAllClose(frnn_final.m, rnn_final.m)
    self.assertAllClose(frnn_final.c, rnn_final.c)

  def _testFRNNGradHelper(self, config):
    dtype = tf.float64  # More stable using float64.
    batch = 3
    dims = 16
    slen = 10
    with self.session(use_gpu=True, config=config) as sess:
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'lstm'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(0.02, 429891685)
      params.vn.global_vn = False
      params.vn.per_step_vn = False
      params.num_input_nodes = dims
      params.num_output_nodes = dims
      params.dtype = dtype

      frnn_params = rnn_layers.FRNN.Params()
      frnn_params.name = 'frnn'
      frnn_params.dtype = dtype
      frnn_params.cell = params
      frnn = rnn_layers.FRNN(frnn_params)
      AddTimestepAccumulator(frnn.cell)
      w, b = frnn.theta.cell.wm, frnn.theta.cell.b

      np.random.seed(12345)
      inputs = tf.constant(
          np.random.uniform(-0.02, 0.02, size=(slen, batch, dims)), dtype)
      paddings = np.zeros([slen, batch, 1])
      paddings[-3:-1, :] = 1.0
      paddings = tf.constant(paddings, dtype)
      frnn_outputs, frnn_final = frnn.FPropDefaultTheta(inputs, paddings)
      loss = tf.reduce_sum(frnn_outputs) + tf.reduce_sum(
          frnn_final.m) + tf.reduce_sum(frnn_final.c)
      dw, db, dinputs = tf.gradients(loss, [w, b, inputs])

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      grad_step = 7
      sym_grads = self.evaluate([db, dw, dinputs])
      sym_grads = [test_utils.PickEveryN(_, grad_step) for _ in sym_grads]
      num_grads = [
          test_utils.PickEveryN(
              test_utils.ComputeNumericGradient(
                  sess, loss, v, delta=1e-4, step=grad_step), grad_step)
          for v in [b, w, inputs]
      ]
      for (sym, num) in zip(sym_grads, num_grads):
        self.assertFalse(np.any(np.isnan(sym)))
        self.assertFalse(np.any(np.isnan(num)))
        print('max = ', np.max(np.abs(sym)))
        self.assertAllClose(sym, num)

  def testFRNNGradNoInline(self):
    self._testFRNNGradHelper(py_utils.SessionConfig(inline=False))

  def testFRNNGradInline(self):
    self._testFRNNGradHelper(py_utils.SessionConfig(inline=True))

  def testStackedFRNNDropout(self):
    v1_out, _ = self._testStackedFRNNHelper(
        rnn_layers.StackedFRNNLayerByLayer,
        tf.float32,
        trailing_pad_len=0,
        keep_prob=0.5)
    if tf.test.is_gpu_available():
      rtol = 1e-5
    else:
      rtol = 1e-6
    self.assertAllClose([175.9741], [np.sum(v1_out * v1_out)], rtol=rtol)

  def testStackedFRNNInputOutputDims(self):
    v1_out, _ = self._testStackedFRNNHelper(
        rnn_layers.StackedFRNNLayerByLayer,
        tf.float32,
        trailing_pad_len=0,
        keep_prob=0.5,
        input_dim=5,
        output_dim=7)
    if tf.test.is_gpu_available():
      rtol = 1e-5
    else:
      rtol = 1e-6
    self.assertAllClose([32.74327], [np.sum(v1_out * v1_out)], rtol=rtol)

  def testStackedFRNNLayerByLayerGrad(self):
    self._testStackedFRNNGradHelper(rnn_layers.StackedFRNNLayerByLayer)

  def testStackedFRNNPackedInput(self):
    tf.random.set_seed(123456)
    batch = 2
    dims = 4
    slen = 5
    num_layers = 4
    dtype = tf.float32
    with self.session() as sess:
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'lstm'
      params.output_nonlinearity = True
      params.dtype = dtype
      params.params_init = py_utils.WeightInit.Uniform(1.24, 429891685)
      params.vn.global_vn = False
      params.vn.per_step_vn = False
      params.num_input_nodes = dims
      params.num_output_nodes = dims

      sfrnn_params = rnn_layers.StackedFRNNLayerByLayer.Params()
      sfrnn_params.name = 'sfrnn'
      sfrnn_params.dtype = dtype
      sfrnn_params.random_seed = 123456
      sfrnn_params.cell_tpl = params
      sfrnn_params.num_layers = num_layers
      sfrnn_params.skip_start = 2
      sfrnn_params.num_input_nodes = dims
      sfrnn_params.num_output_nodes = dims
      sfrnn_params.packed_input = True
      with tf.name_scope('sfrnn'):
        sfrnn = sfrnn_params.Instantiate()

      np.random.seed(12345)
      inputs = tf.constant(np.random.uniform(size=(slen, batch, dims)), dtype)
      paddings = tf.constant(np.zeros([slen, batch, 1]), dtype)
      segment_id = tf.constant(np.ones([slen, batch, 1]), dtype)

      self.evaluate(tf.global_variables_initializer())

      # Output with unpacked inputs.
      sfrnn_outputs, _ = sfrnn.FPropDefaultTheta(
          inputs, paddings, segment_id=segment_id)
      sfrnn_outputs = py_utils.HasShape(sfrnn_outputs, [slen, batch, dims])

      def _Pack(x):
        # [batch, slen, ...].
        x = tf.transpose(x, [1, 0, 2])
        # [batch * slen, 1, ...].
        return tf.reshape(x, [batch * slen, 1, -1])

      # Output with packed inputs.
      packed_inputs = _Pack(inputs)
      packed_paddings = _Pack(paddings)
      packed_segment_id = _Pack(tf.cumsum(segment_id, axis=1))
      packed_outputs, _ = sfrnn.FPropDefaultTheta(
          packed_inputs, packed_paddings, segment_id=packed_segment_id)
      packed_outputs = tf.reshape(packed_outputs, [batch, slen, dims])
      packed_outputs = tf.transpose(packed_outputs, [1, 0, 2])

      # Check that the outputs are equal.
      sfrnn_outputs, packed_outputs = sess.run([sfrnn_outputs, packed_outputs])
      self.assertAllClose(sfrnn_outputs, packed_outputs)

  def testStackedBiFRNNDropout(self):
    v1_out = self._testStackedFRNNHelper(
        rnn_layers.StackedBiFRNNLayerByLayer,
        tf.float32,
        trailing_pad_len=0,
        keep_prob=0.5,
        bi_directional=True)
    if tf.test.is_gpu_available():
      rtol = 1e-5
    else:
      rtol = 1e-6
    self.assertAllClose([305.77435], [np.sum(v1_out * v1_out)], rtol=rtol)

  def testStackedBiFRNNInputOutputDims(self):
    v1_out = self._testStackedFRNNHelper(
        rnn_layers.StackedBiFRNNLayerByLayer,
        tf.float32,
        trailing_pad_len=0,
        keep_prob=0.5,
        bi_directional=True,
        input_dim=5,
        output_dim=8)
    if tf.test.is_gpu_available():
      rtol = 1e-5
    else:
      rtol = 1e-6
    self.assertAllClose([8.116007], [np.sum(v1_out * v1_out)], rtol=rtol)

  def testStackedBiFRNNLayerByLayerGrad(self):
    self._testStackedFRNNGradHelper(
        rnn_layers.StackedBiFRNNLayerByLayer, bi_directional=True)

  def _testBidirectionalFRNNHelper(self,
                                   trailing_pad_len=0,
                                   cluster_params=None):
    batch = 3
    dims = 16
    slen = 10 + trailing_pad_len

    params = rnn_cell.LSTMCellSimple.Params()
    params.name = 'lstm_forward'
    params.output_nonlinearity = True
    params.params_init = py_utils.WeightInit.Uniform(0.02, 429891685)
    params.vn.global_vn = True
    params.vn.per_step_vn = False
    params.vn.seed = 2938482
    params.vn.scale = 0.1
    params.num_input_nodes = dims
    params.num_output_nodes = dims
    lstm_forward = params.Copy()
    params.name = 'lstm_backward'
    params.params_init = py_utils.WeightInit.Uniform(0.02, 83820209838)
    lstm_backward = params.Copy()

    frnn_params = rnn_layers.BidirectionalFRNN.Params()
    frnn_params.name = 'bifrnn'
    frnn_params.fwd = lstm_forward.Copy()
    frnn_params.bak = lstm_backward.Copy()

    rnn_params = rnn_layers.BidirectionalRNN.Params()
    rnn_params.name = 'rnn'
    rnn_params.fwd = lstm_forward.Copy()
    rnn_params.bak = lstm_backward.Copy()
    rnn_params.sequence_length = slen

    np.random.seed(12345)
    inputs = np.random.uniform(size=(slen, batch, dims)).astype(np.float32)
    paddings = np.zeros([slen, batch, 1], dtype=np.float32)
    paddings[-trailing_pad_len:, :] = 1.0
    paddings[-trailing_pad_len - 3:-trailing_pad_len - 1, :] = 1.0

    with cluster_factory.Cluster(
        cluster_params) if cluster_params else cluster_factory.Current():
      with self.session(
          graph=tf.Graph(),
          use_gpu=True,
          config=tf.config_pb2.ConfigProto(allow_soft_placement=True)):
        frnn = frnn_params.Instantiate()
        self.evaluate(tf.global_variables_initializer())
        frnn_outputs = self.evaluate(
            frnn.FPropDefaultTheta(tf.constant(inputs), tf.constant(paddings)))

      with self.session(
          graph=tf.Graph(),
          use_gpu=True,
          config=tf.config_pb2.ConfigProto(allow_soft_placement=True)):
        rnn = rnn_params.Instantiate()
        self.evaluate(tf.global_variables_initializer())
        rnn_outputs = self.evaluate(
            rnn.FPropDefaultTheta(tf.unstack(inputs), tf.unstack(paddings)))

      rnn_outputs_val, frnn_outputs_val = [
          x[:-trailing_pad_len] for x in [rnn_outputs, frnn_outputs]
      ]
      self.assertAllClose(rnn_outputs_val, frnn_outputs_val)

  def testBidirectionalFRNN(self):
    self._testBidirectionalFRNNHelper()

  def testBidirectionalFRNNTrailingPadding(self):
    self._testBidirectionalFRNNHelper(trailing_pad_len=2)

  def testBidirectionalFRNNSplit(self):
    cluster_params = cluster_factory.Current().params.Copy()
    cluster_params.worker.Set(
        gpus_per_replica=2, devices_per_split=2, name='/job:localhost')
    self._testBidirectionalFRNNHelper(cluster_params=cluster_params)

  def testBidirectionalFRNNGrad(self):
    dtype = tf.float64  # More stable using float64.
    batch = 3
    dims = 16
    slen = 10
    with self.session(use_gpu=True) as sess:
      params = rnn_cell.LSTMCellSimple.Params()
      params.name = 'lstm_forward'
      params.output_nonlinearity = True
      params.params_init = py_utils.WeightInit.Uniform(0.02, 429891685)
      params.vn.global_vn = False
      params.vn.per_step_vn = False
      params.dtype = dtype
      params.num_input_nodes = dims
      params.num_output_nodes = dims
      lstm_forward = params.Copy()
      params.name = 'lstm_backward'
      params.params_init = py_utils.WeightInit.Uniform(0.02, 83820209838)
      params.dtype = dtype
      lstm_backward = params.Copy()

      frnn_params = rnn_layers.BidirectionalFRNN.Params()
      frnn_params.name = 'bifrnn'
      frnn_params.dtype = dtype
      frnn_params.fwd = lstm_forward.Copy()
      frnn_params.bak = lstm_backward.Copy()
      frnn = rnn_layers.BidirectionalFRNN(frnn_params)
      w0, b0 = (frnn.theta.fwd_rnn.cell.wm, frnn.theta.fwd_rnn.cell.b)
      w1, b1 = (frnn.theta.bak_rnn.cell.wm, frnn.theta.bak_rnn.cell.b)

      np.random.seed(12345)
      inputs = tf.constant(np.random.uniform(size=(slen, batch, dims)), dtype)
      paddings = np.zeros([slen, batch, 1])
      paddings[-3:-1, :] = 1.0
      paddings = tf.constant(paddings, dtype)
      frnn_outputs = frnn.FPropDefaultTheta(inputs, paddings)
      loss = tf.reduce_sum(frnn_outputs)

      dw0, db0, dw1, db1, dinputs = tf.gradients(loss, [w0, b0, w1, b1, inputs])

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      grad_step = 13
      sym_grads = self.evaluate([dw0, db0, dw1, db1, dinputs])
      sym_grads = [test_utils.PickEveryN(_, grad_step) for _ in sym_grads]
      num_grads = [
          test_utils.PickEveryN(
              test_utils.ComputeNumericGradient(
                  sess, loss, v, delta=1e-4, step=grad_step), grad_step)
          for v in [w0, b0, w1, b1, inputs]
      ]
      for (sym, num) in zip(sym_grads, num_grads):
        self.assertFalse(np.any(np.isnan(sym)))
        self.assertFalse(np.any(np.isnan(num)))
        print('max = ', np.max(np.abs(sym)))
        self.assertAllClose(sym, num)

  def _MultiSourceFRNNWithAttentionInputs(self,
                                          single_source=False,
                                          single_source_length=True,
                                          dtype=tf.float32):
    np.random.seed(12345)
    if single_source:
      src_names = ['en']
      slens = [10]
      sdepths = [4]
    elif single_source_length:
      src_names = ['en1', 'en2', 'de']
      slens = [11, 10, 9]
      sdepths = [4, 4, 4]
    else:
      src_names = ['en1', 'en2', 'de']
      slens = [11, 10, 9]
      sdepths = [4, 4, 3]
    sbatch = 3
    tlen = 7
    tbatch = 6
    dims = 4

    src_encs = py_utils.NestedMap()
    src_paddings = py_utils.NestedMap()
    for sdepth, slen, sname in zip(sdepths, slens, src_names):
      src_encs[sname] = tf.constant(
          np.random.uniform(size=[slen, sbatch, sdepth]), dtype)
      src_paddings[sname] = tf.constant(np.zeros([slen, sbatch]), dtype)
    inputs = tf.constant(np.random.uniform(size=(tlen, tbatch, dims)), dtype)
    paddings = tf.constant(np.zeros([tlen, tbatch, 1]), dtype)
    return (src_encs, src_paddings, inputs, paddings)

  def _MultiSourceFRNNWithAttentionParams(self,
                                          single_source=False,
                                          single_source_length=True,
                                          dtype=tf.float32):
    dims = 4
    alt_depth = 3
    if single_source:
      src_names = ['en']
    else:
      src_names = ['en1', 'en2', 'de']

    p = rnn_cell.LSTMCellSimple.Params()
    p.name = 'lstm'
    p.dtype = dtype
    p.output_nonlinearity = True
    p.params_init = py_utils.WeightInit.Uniform(0.02, 429891685)
    p.vn.global_vn = False
    p.vn.per_step_vn = False
    p.num_input_nodes = dims * 2
    p.num_output_nodes = dims
    lstm_params = p

    p = attention.AdditiveAttention.Params()
    p.name = 'atten'
    p.dtype = dtype
    p.params_init = py_utils.WeightInit.Gaussian(0.1, 12345)
    p.source_dim = dims
    p.query_dim = dims
    p.hidden_dim = dims
    p.vn.global_vn = False
    p.vn.per_step_vn = False
    attention_tpl = p

    p = attention.MergerLayer.Params()
    p.name = 'merger'
    p.dtype = dtype
    p.merger_op = ('mean' if single_source else 'atten')
    p.source_dim = dims
    p.query_dim = dims
    p.hidden_dim = dims
    merger_tpl = p

    p = rnn_layers.MultiSourceFRNNWithAttention.Params()
    p.name = 'msrc_frnn_with_atten'
    p.dtype = dtype
    p.cell = lstm_params
    p.attention_tpl = attention_tpl
    p.atten_merger = merger_tpl
    p.source_names = src_names

    if not single_source_length:
      de_atten = attention_tpl.Copy()
      de_atten.source_dim = alt_depth
      p.source_name_to_attention_params = {'de': de_atten}
      merger_tpl.pre_proj_input_dims = [dims, dims, alt_depth]
      merger_tpl.pre_proj_output_dims = [dims, dims, dims]
      merger_tpl.proj_tpl.batch_norm = False
      merger_tpl.proj_tpl.weight_norm = True

    return p

  def testMultiSourceFRNNWithAttention(self):
    with self.session(use_gpu=True):
      p = self._MultiSourceFRNNWithAttentionParams()
      msrc_frnn = p.Instantiate()

      (src_encs, src_paddings, inputs,
       paddings) = self._MultiSourceFRNNWithAttentionInputs()
      a, m = msrc_frnn.FPropDefaultTheta(src_encs, src_paddings, inputs,
                                         paddings)
      msrc_frnn_out = tf.concat([a, m], 2)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      ys = self.evaluate([msrc_frnn_out])[0]
      self.assertEqual(ys.shape, (7, 6, 8))
      print(np.sum(ys, axis=(1, 2)), np.sum(ys, axis=(0, 1)),
            np.sum(ys, axis=(0, 2)))
      # pyformat: disable
      # pylint: disable=bad-whitespace
      self.assertAllClose(
          np.sum(ys, axis=(1, 2)), [
              11.87568951,  11.8436203 ,  11.80368233,  11.80167198,
              11.82034779,  11.80246162,  11.80818748
          ])
      self.assertAllClose(
          np.sum(ys, axis=(0, 1)), [
              21.41802788,  20.86244965,  21.48164749,  19.95701981,
              -0.54706949,   0.07046284,  -0.50449395,   0.0176318
          ])
      self.assertAllClose(
          np.sum(ys, axis=(0, 2)), [
              13.29822254,  14.01552773,  14.04851151,  13.28098106,
              14.05391502,  14.0585041
          ])
      # pyformat: enable
      # pylint: enable=bad-whitespace

  def testMultiSourceFRNNWithAttentionMultiDepth(self):
    with self.session(use_gpu=True):
      p = self._MultiSourceFRNNWithAttentionParams(single_source_length=False)
      msrc_frnn = p.Instantiate()

      (src_encs, src_paddings, inputs, paddings
      ) = self._MultiSourceFRNNWithAttentionInputs(single_source_length=False)
      a, m = msrc_frnn.FPropDefaultTheta(src_encs, src_paddings, inputs,
                                         paddings)
      msrc_frnn_out = tf.concat([a, m], 2)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      ys = self.evaluate([msrc_frnn_out])[0]
      self.assertEqual(ys.shape, (7, 6, 8))
      print(np.sum(ys, axis=(1, 2)), np.sum(ys, axis=(0, 1)),
            np.sum(ys, axis=(0, 2)))
      # pyformat: disable
      # pylint: disable=bad-whitespace
      self.assertAllClose(
          np.sum(ys, axis=(1, 2)), [
              5.976197,  5.932313,  5.917447,  5.907898,  5.907385,  5.90272 ,
              5.890248
          ])
      self.assertAllClose(
          np.sum(ys, axis=(0, 1)), [
              2.635296e+01,   3.177989e+00,   1.024462e+01,   2.403777e+00,
              -4.908564e-01,   1.006475e-01,  -3.303704e-01,  -2.455414e-02
          ])
      self.assertAllClose(
          np.sum(ys, axis=(0, 2)), [
              6.610287,  6.657996,  7.452699,  6.626875,  6.60216 ,  7.484191
          ])
      # pyformat: enable
      # pylint: enable=bad-whitespace

  def testMultiSourceFRNNWithAttentionSingleSource(self, dtype=tf.float32):
    with self.session(
        use_gpu=True, config=py_utils.SessionConfig(inline=False)):
      p = self._MultiSourceFRNNWithAttentionParams(
          single_source=True, dtype=dtype)
      frnn = p.Instantiate()

      (src_encs, src_paddings, inputs,
       paddings) = self._MultiSourceFRNNWithAttentionInputs(
           single_source=True, dtype=dtype)

      a, m = frnn.FPropDefaultTheta(src_encs, src_paddings, inputs, paddings)
      frnn_out = tf.concat([a, m], 2)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      ys, = self.evaluate([frnn_out])
      self.assertEqual(ys.shape, (7, 6, 8))
      print(np.sum(ys, axis=(1, 2)), np.sum(ys, axis=(0, 1)),
            np.sum(ys, axis=(0, 2)))

      # These values are identical with FRNNWithAttention.
      expected_sum12 = [
          13.07380962, 13.03321552, 12.99956226, 13.00612164, 13.01202011,
          12.99347878, 12.98680687
      ]
      expected_sum01 = [
          2.41238327e+01, 2.11899853e+01, 2.45926647e+01, 2.22827835e+01,
          -5.62886238e-01, 2.42760777e-02, -5.79716980e-01, 3.40666063e-02
      ]
      expected_sum02 = [
          12.74695969, 16.13114548, 16.66101837, 12.74922562, 16.16581345,
          16.65085411
      ]

      self.assertAllClose(np.sum(ys, axis=(1, 2)), expected_sum12)
      self.assertAllClose(np.sum(ys, axis=(0, 1)), expected_sum01)
      self.assertAllClose(np.sum(ys, axis=(0, 2)), expected_sum02)

  def testMultiSourceFRNNWithAttentionGradSingleSource(self, dtype=tf.float64):
    with self.session(
        use_gpu=True, config=py_utils.SessionConfig(inline=False)) as sess:

      p = self._MultiSourceFRNNWithAttentionParams(
          single_source=True, dtype=dtype)
      frnn = p.Instantiate()

      (src_encs, src_paddings, inputs,
       paddings) = self._MultiSourceFRNNWithAttentionInputs(
           single_source=True, dtype=dtype)

      # Fetch all the parameters.
      w0, b0 = (frnn.theta.cell.wm, frnn.theta.cell.b)
      att0h, att0q, att0s = (frnn.theta.attentions[0].hidden_var,
                             frnn.theta.attentions[0].query_var,
                             frnn.theta.attentions[0].source_var)

      out, _ = frnn.FPropDefaultTheta(src_encs, src_paddings, inputs, paddings)
      loss = tf.reduce_sum(out)

      parameters = [w0, b0, inputs, att0h, att0q, att0s]
      grads = tf.gradients(loss, parameters)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      sym_grads = self.evaluate(grads)
      num_grads = [
          test_utils.ComputeNumericGradient(sess, loss, v, delta=1e-5)
          for v in parameters
      ]
      for i, (sym, num) in enumerate(zip(sym_grads, num_grads)):
        print([
            i, sym.shape, num.shape,
            np.max(np.abs(sym)),
            np.max(np.abs(sym - num)),
            np.max(np.abs(sym - num) / np.abs(sym))
        ])

      def Compare(name, sym, num, rtol=1e-5):
        print(['name = ', name])
        self.assertFalse(np.any(np.isnan(sym)))
        self.assertFalse(np.any(np.isnan(num)))
        self.assertAllClose(sym, num, rtol=rtol, atol=1e-8)

      for i, (sym, num) in enumerate(zip(sym_grads, num_grads)):
        Compare(parameters[i].name, sym, num)

  def testMultiSourceFRNNWithAttentionGrad(self, dtype=tf.float64):
    with self.session(
        use_gpu=True, config=py_utils.SessionConfig(inline=False)) as sess:

      p = self._MultiSourceFRNNWithAttentionParams(dtype=dtype)
      frnn = p.Instantiate()

      # Fetch all the parameters.
      w0, b0 = (frnn.theta.cell.wm, frnn.theta.cell.b)
      mh, mq, ms = (frnn.theta.atten_merger.atten.hidden_var,
                    frnn.theta.atten_merger.atten.query_var,
                    frnn.theta.atten_merger.atten.source_var)
      att0h, att0q, att0s = (frnn.theta.attentions[0].hidden_var,
                             frnn.theta.attentions[0].query_var,
                             frnn.theta.attentions[0].source_var)
      att1h, att1q, att1s = (frnn.theta.attentions[1].hidden_var,
                             frnn.theta.attentions[1].query_var,
                             frnn.theta.attentions[1].source_var)
      att2h, att2q, att2s = (frnn.theta.attentions[2].hidden_var,
                             frnn.theta.attentions[2].query_var,
                             frnn.theta.attentions[2].source_var)

      (src_encs, src_paddings, inputs,
       paddings) = self._MultiSourceFRNNWithAttentionInputs(dtype=dtype)

      out, _ = frnn.FPropDefaultTheta(src_encs, src_paddings, inputs, paddings)
      loss = tf.reduce_sum(out)

      parameters = [
          w0, b0, inputs, mh, mq, ms, att0h, att0q, att0s, att1h, att1q, att1s,
          att2h, att2q, att2s
      ]
      grads = tf.gradients(loss, parameters)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      sym_grads = self.evaluate(grads)
      num_grads = [
          test_utils.ComputeNumericGradient(sess, loss, v, delta=1e-5)
          for v in parameters
      ]
      for i, (sym, num) in enumerate(zip(sym_grads, num_grads)):
        print([
            i, sym.shape, num.shape,
            np.max(np.abs(sym)),
            np.max(np.abs(sym - num)),
            np.max(np.abs(sym - num) / np.abs(sym))
        ])

      def Compare(name, sym, num, rtol=1e-5):
        print(['name = ', name])
        self.assertFalse(np.any(np.isnan(sym)))
        self.assertFalse(np.any(np.isnan(num)))
        self.assertAllClose(sym, num, rtol=rtol, atol=1e-8)

      for i, (sym, num) in enumerate(zip(sym_grads, num_grads)):
        Compare(parameters[i].name, sym, num)

  def testMultiSourceFRNNWithAttentionGradMultiDepth(self, dtype=tf.float64):
    with self.session(
        use_gpu=True, config=py_utils.SessionConfig(inline=False)) as sess:

      p = self._MultiSourceFRNNWithAttentionParams(
          single_source_length=False, dtype=dtype)
      frnn = p.Instantiate()

      # Fetch all the parameters.
      w0, b0 = (frnn.theta.cell.wm, frnn.theta.cell.b)
      mh, mq, ms, mw0, mw1, mw2 = (frnn.theta.atten_merger.atten.hidden_var,
                                   frnn.theta.atten_merger.atten.query_var,
                                   frnn.theta.atten_merger.atten.source_var,
                                   frnn.theta.atten_merger.pre_proj[0].w,
                                   frnn.theta.atten_merger.pre_proj[1].w,
                                   frnn.theta.atten_merger.pre_proj[2].w)
      att0h, att0q, att0s = (frnn.theta.attentions[0].hidden_var,
                             frnn.theta.attentions[0].query_var,
                             frnn.theta.attentions[0].source_var)
      att1h, att1q, att1s = (frnn.theta.attentions[1].hidden_var,
                             frnn.theta.attentions[1].query_var,
                             frnn.theta.attentions[1].source_var)
      att2h, att2q, att2s = (frnn.theta.attentions[2].hidden_var,
                             frnn.theta.attentions[2].query_var,
                             frnn.theta.attentions[2].source_var)

      (src_encs, src_paddings, inputs,
       paddings) = self._MultiSourceFRNNWithAttentionInputs(
           single_source_length=False, dtype=dtype)

      out, _ = frnn.FPropDefaultTheta(src_encs, src_paddings, inputs, paddings)
      loss = tf.reduce_sum(out)

      parameters = [
          w0, b0, inputs, mh, mq, ms, att0h, att0q, att0s, att1h, att1q, att1s,
          att2h, att2q, att2s, mw0, mw1, mw2
      ]
      grads = tf.gradients(loss, parameters)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      sym_grads = self.evaluate(grads)
      num_grads = [
          test_utils.ComputeNumericGradient(sess, loss, v, delta=1e-5)
          for v in parameters
      ]
      for i, (sym, num) in enumerate(zip(sym_grads, num_grads)):
        print([
            i, sym.shape, num.shape,
            np.max(np.abs(sym)),
            np.max(np.abs(sym - num)),
            np.max(np.abs(sym - num) / np.abs(sym))
        ])

      def Compare(name, sym, num, rtol=1e-5):
        print(['name = ', name])
        self.assertFalse(np.any(np.isnan(sym)))
        self.assertFalse(np.any(np.isnan(num)))
        self.assertAllClose(sym, num, rtol=rtol, atol=1e-8)

      for i, (sym, num) in enumerate(zip(sym_grads, num_grads)):
        Compare(parameters[i].name, sym, num)

  def _CreateFRNNWithAttentionParams(self,
                                     dtype,
                                     dims,
                                     slen,
                                     sbatch,
                                     tlen,
                                     tbatch,
                                     input_prev_atten_ctx=True,
                                     output_prev_atten_ctx=False):
    # Create RNN Layer.
    p = rnn_cell.LSTMCellSimple.Params()
    p.name = 'lstm'
    p.dtype = dtype
    p.output_nonlinearity = True
    p.params_init = py_utils.WeightInit.Uniform(0.02, 429891685)
    p.vn.global_vn = False
    p.vn.per_step_vn = False
    p.num_input_nodes = dims * 2 if input_prev_atten_ctx else dims
    p.num_output_nodes = dims
    lstm_params = p

    # Create Attention Layer.
    p = attention.AdditiveAttention.Params()
    p.name = 'atten'
    p.dtype = dtype
    p.params_init = py_utils.WeightInit.Gaussian(0.1, 12345)
    p.source_dim = dims
    p.query_dim = dims
    p.hidden_dim = dims
    p.vn.global_vn = False
    p.vn.per_step_vn = False
    atten = p

    p = rnn_layers.FRNNWithAttention.Params()
    p.name = 'frnn_with_atten'
    p.dtype = dtype
    p.cell = lstm_params
    p.attention = atten
    p.input_prev_atten_ctx = input_prev_atten_ctx
    p.output_prev_atten_ctx = output_prev_atten_ctx
    return p

  @parameterized.parameters((False, False), (False, True), (True, False),
                            (True, True))
  def testFRNNWithAttentionSeparateSourceContextIdenticalToSourceEnc(
      self, input_prev_atten_ctx, output_prev_atten_ctx):
    dtype = tf.float32
    dims = 4
    slen = 10
    sbatch = 3
    tlen = 7
    tbatch = 6

    with self.session(use_gpu=True, config=py_utils.SessionConfig(inline=True)):
      np.random.seed(12345)
      p = self._CreateFRNNWithAttentionParams(
          dtype=dtype,
          dims=dims,
          slen=slen,
          sbatch=sbatch,
          tlen=tlen,
          tbatch=tbatch,
          input_prev_atten_ctx=input_prev_atten_ctx,
          output_prev_atten_ctx=output_prev_atten_ctx)

      frnn = p.Instantiate()

      src_encs = tf.constant(
          np.random.uniform(size=[slen, sbatch, dims]), dtype)
      src_paddings = tf.constant(np.zeros([slen, sbatch]), dtype)

      inputs = tf.constant(np.random.uniform(size=(tlen, tbatch, dims)), dtype)
      paddings = tf.constant(np.zeros([tlen, tbatch, 1]), dtype)

      # Run without specifying source context vectors.
      atten_ctx, rnn_out, atten_prob, _ = frnn.FPropDefaultTheta(
          src_encs, src_paddings, inputs, paddings)
      frnn_out = tf.concat([atten_ctx, rnn_out, atten_prob], 2)

      # Run after providing separate source context vectors set to the src_encs
      # should provide the same answer.
      (atten_ctx_src_ctx, rnn_out_src_ctx, atten_prob_src_ctx,
       _) = frnn.FPropDefaultTheta(
           src_encs, src_paddings, inputs, paddings, src_contexts=src_encs)
      frnn_out_src_ctx = tf.concat(
          [atten_ctx_src_ctx, rnn_out_src_ctx, atten_prob_src_ctx], 2)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      frnn_out_v, frnn_out_src_ctx_v = self.evaluate(
          [frnn_out, frnn_out_src_ctx])

      # Expected last dimensions for atten_ctx_src_ctx, rnn_out_src_ctx,
      # atten_prob_src_ctx are respectively, (dims, dims, slen).
      self.assertEqual(frnn_out_v.shape, (tlen, tbatch, 2 * dims + slen))
      self.assertEqual(frnn_out_src_ctx_v.shape, frnn_out_v.shape)

      self.assertAllClose(frnn_out_v, frnn_out_src_ctx_v)

  @parameterized.parameters((False, False), (False, True), (True, False),
                            (True, True))
  def testFRNNWithAttentionSeparateSourceContextDifferentFromSourceEnc(
      self, input_prev_atten_ctx, output_prev_atten_ctx):
    dtype = tf.float32
    dims = 4
    slen = 10
    sbatch = 3
    tlen = 7
    tbatch = 6

    with self.session(use_gpu=True, config=py_utils.SessionConfig(inline=True)):
      np.random.seed(12345)
      p = self._CreateFRNNWithAttentionParams(
          dtype=dtype,
          dims=dims,
          slen=slen,
          sbatch=sbatch,
          tlen=tlen,
          tbatch=tbatch,
          input_prev_atten_ctx=input_prev_atten_ctx,
          output_prev_atten_ctx=output_prev_atten_ctx)

      frnn = p.Instantiate()

      src_encs = tf.constant(
          np.random.uniform(size=[slen, sbatch, dims]), dtype)
      src_paddings = tf.constant(np.zeros([slen, sbatch]), dtype)

      # We create src_contexts with even dimensions (0, 2) set to all zero, the
      # rest are set randomly.
      src_contexts = np.random.uniform(size=[slen, sbatch, dims])
      src_contexts[:, :, 0:dims:2] = 0.0
      src_contexts = tf.constant(src_contexts, dtype=dtype)

      inputs = tf.constant(np.random.uniform(size=(tlen, tbatch, dims)), dtype)
      paddings = tf.constant(np.zeros([tlen, tbatch, 1]), dtype)

      # Run after providing separate source context vectors set to the src_encs
      # should provide the same answer.
      atten_ctx, _, _, _ = frnn.FPropDefaultTheta(
          src_encs, src_paddings, inputs, paddings, src_contexts=src_contexts)

      # Initialize all the variables, and then run one step.
      self.evaluate(tf.global_variables_initializer())
      atten_ctx_v = self.evaluate(atten_ctx)

      self.assertEqual(atten_ctx_v.shape, (tlen, tbatch, dims))
      # Verify that the output also has zeros in the locations that the
      # source context has zeros.
      self.assertAllClose(
          np.zeros(shape=(tlen, tbatch, dims // 2)),
          atten_ctx_v[:, :, 0:dims:2])

  def _testFRNNWithAttentionUseZeroAttenState(self, zero_atten_state_fn):
    dtype = tf.float32
    dims = 5
    slen = 4
    tlen = 3
    sbatch = 2
    tbatch = 6

    with self.session(use_gpu=True):
      p = self._CreateFRNNWithAttentionParams(
          dtype=dtype,
          dims=dims,
          slen=slen,
          sbatch=sbatch,
          tlen=tlen,
          tbatch=tbatch)
      p.use_zero_atten_state = True
      p.atten_context_dim = dims
      frnn = p.Instantiate()

      # Override the ZeroAttentionState to have the desired output type
      frnn.atten.ZeroAttentionState = types.MethodType(zero_atten_state_fn,
                                                       frnn.atten)

      src_encs = tf.constant(
          np.random.uniform(size=[slen, sbatch, dims]), dtype)
      src_paddings = tf.constant(np.zeros([slen, sbatch]), dtype)

      inputs = tf.constant(np.random.uniform(size=(tlen, tbatch, dims)), dtype)
      paddings = tf.constant(np.zeros([tlen, tbatch, 1]), dtype)

      atten_ctx, rnn_out, atten_prob, _ = frnn.FPropDefaultTheta(
          src_encs, src_paddings, inputs, paddings)

      self.evaluate(tf.global_variables_initializer())
      atten_ctx, rnn_out, atten_prob = self.evaluate(
          [atten_ctx, rnn_out, atten_prob])

      # Check shapes
      self.assertEqual(atten_ctx.shape, (tlen, tbatch, dims))
      self.assertEqual(rnn_out.shape, (tlen, tbatch, dims))
      self.assertEqual(atten_prob.shape, (tlen, tbatch, slen))

  def testFRNNWithAttentionUseZeroAttenStateTensor(self):

    def _TensorZeroAttenState(self, source_seq_length, decoder_batch_size):
      del source_seq_length
      p = self.params
      zs = tf.zeros([decoder_batch_size, 1], dtype=py_utils.FPropDtype(p))
      return zs

    self._testFRNNWithAttentionUseZeroAttenState(_TensorZeroAttenState)

  def testFRNNWithAttentionUseZeroAttenStateNestedMap(self):

    def _NestedMapZeroAttenState(self, source_seq_length, decoder_batch_size):
      del source_seq_length
      p = self.params
      zs = tf.zeros([decoder_batch_size, 1], dtype=py_utils.FPropDtype(p))
      return py_utils.NestedMap(z=zs)

    self._testFRNNWithAttentionUseZeroAttenState(_NestedMapZeroAttenState)

if __name__ == '__main__':
  test_utils.main()

redpajama_set_name: RedPajamaGithub


As part of the Volkswagen Group, TPS has a national network of Centres. We pride ourselves on creating an environment that allows our people to excel; please take a look at our videos throughout this website to find out more about some of our successful apprentices and our outstanding training programme.
"You'll make friends during your training that you'll have for your career."
You can call us on +44 (0)800 5873051 or email info@tpsapprenticeships.co.uk.


text:
Think tank kan åsyfta
 Think tank – en mer eller mindre lös sammanslutning av skribenter och intellektuella som har som syfte att påverka den politiska opinionen, se Tankesmedja
 Think Tank (musikalbum) – den brittiska rockgruppen Blurs sjunde studioalbum, utgivet 2003
 Think Tank (TV-serie) – en TV-serie på PBS i USA

redpajama_set_name: RedPajamaWikipedia


text:
How Funeral Selfies Made Narcissism a Necessity
ByMichael Thomsen
Michael Thomsen is a writer in New York and author of Levitate the Primate: Handjobs, Internet Dating and Other Issues for Men.
A new Tumblr highlights the growing trend of people taking selfies at funerals. Here's why it's not as bad as you think.
Last week FastCompany writer Jason Feifer made a Tumblr of selfies taken at funerals. The pictures are indistinguishable from any other form of selfie but the special framing suggests they deserve their own sub-category, something requiring new understanding if not outright action.
The funeral selfie is a kind of narcissism that is nauseating and transparent but acceptable so long as it remains a contained indulgence that brings with it a consciousness of potential shaming. And it seems a number of funeral selfie takers are aware of the moral gaze being cast upon them. "I think this makes me a bad person," one woman wrote alongside her photo. "am i going to hell((yes))," another wrote.
If there is good taste to be modeled at a funeral, it's in solemn inaction, sharing memories that flatter the dead and spare the grieving from the strangeness of death, loss, disease, and old age.
The charge of narcissism has become overfamiliar as social media has spread. We are all capable of acts of narcissism, but only other people are true narcissists, so in love with their self-importance that they feel entitled to hijack a family member's funeral. In its original sense, what makes narcissism bad is not self-interest but when it leads to neglect that becomes damaging to those around them. If there is a kind of narcissism involved in selfies, it's not derived from a curiosity about one's own body but in the immutable seriousness we want to invest the media with. Before media became interactive, it was an omnipresent stream of ghost reality that one accessed with tuners and subscriptions.
We attended televisions and radios like seances with a higher and invisible truth, unfolding a narrative reality whose claims required faith in the value of shared experiences defined through singular sources. And in their own way these passive forms of media were more damagingly narcissistic, drawing their audiences into discourse about politics and culture, in which just having an opinion became a moral necessity—Vietnam, Nixon, disco, Reagan, Madonna, Princess Diana's death, Iraq, the Drug War, Viagra, Obama, abortion, The Unibomber, 9/11, Iraq again, Facebook, Selfies. All of these storied spirit beggars needed your judgment to become real, and the energy spent just keeping up was so encompassing maybe you'd walk past the homeless man who needed only money not opinions, and so became invisible.
Funerals are wrought from the same tradition of simply attending other people's truths, and their American permutations come with dozens of deferences to commerce, from the rented church hall and the cost of the catering to the fusty suits and the tastefully expensive flowers almost no one will actually take comfort in. If there is good taste to be modeled at a funeral, it's in solemn inaction, sharing memories that flatter the dead and spare the grieving from the strangeness of death, loss, disease, and old age.
What's threatening about funeral selfies is not their narcissism, but the opportunity to steer our own forms of narcissism with greater precision, weirdness, and irreverence. They aren't exactly noble, but in a way, they are the logical evolution of a media culture.
As one of my grandmother's was dying I remember how strange she became in old age, her body forgetting how to walk, her brain forgetting to keep English and Danish separate, how excited she would become when a pretty bird would fly past the window, how she'd mess herself, and how sleep took over more and more of her day. It was as if her life's accumulation of beliefs and structural convictions were finally releasing her from duty, the dramatic orientation she'd had in the world of opinions were overwritten by the immediacy of new kinds of chronic pain and sudden bursts of sensuality.
Over-identifying with the importance of media requires the dismissal of experiences that can't be taken seriously in it. There is no place for a story about my grandmother having seen a bird on the evening news, not because it's not historic or newsworthy in some small way, but because such smallness and intimacy would destroy our illusions about the media as a supernatural arrangement of global information happenings.
What's threatening about funeral selfies is not their narcissism, but the opportunity to steer our own forms of narcissism with greater precision, weirdness, and irreverence. They aren't exactly noble, but in a way, they are the logical evolution of a media culture that was veering perilously toward self-mockery in the dark days before the Internet—it was a journalist who collated them, after all.
The only way to keep people from disengaging from an increasingly transparent media superstructure is to bestow them the ability to produce a media narrative of their own, falling awkwardly in love with photographic evidence of all the social skin you can shed in a lifetime, everyone else be damned because, in truth, they were already and that had nothing to do with you.
Michael Thomsen is Complex's tech columnist. He has written for Slate, The Atlantic, The New Inquiry, n+1, Billboard, and is author of Levitate the Primate: Handjobs, Internet Dating, and Other Issues for Men. He tweets often at @mike_thomsen.
RELATED: Four Pins - A Guide to Fake Grieving on Social Media
NewsEssaysSelfiesTechTumblr

redpajama_set_name: RedPajamaCommonCrawl


ChurchCentral.com, an online leadership "mega-blog" featuring practical leadership ideas and inspiration, has created a new event called Turnaround 20/20. The conference, which will be held this Oct. 23rd at the LifeWay conference center in Nashville, Tenn., will feature 20 speakers, 20 minutes at a time, giving their best advice on how to help a struggling church.
Turnaround 20/20 speakers include Bob Whitesel, Chuck Lawless, Gary McIntosh, Nelson Searcy, Elmer Towns, George Hunter, Charles Arn, Carl George and John Ewart. Additional speakers to be announced.
The day before the event, the Society for Church Consulting will host advanced training led by Chuck Lawless and John Ewart. Church consultants can take the last two levels required for certification, and additional specialty training will be offered. On Wed., Oct. 24th, the Great Commission Research Network will host its annual conference at the same location, featuring a turnaround roundtable hosted by church growth author Dr. Gary McIntosh.
Pastors.com, a ministry created by Rick Warren, is a promotional partner.
To attend, sponsor or gather more information, visit http://www.turnaround2020.com.
Since 2002, ChurchCentral.com has served church leaders with blogs, articles and videos. Its sister ministry, the Society for Church Consulting, has certified hundreds of consultants since 2002, and offers training and certification to qualified consultants.


text:
Queens Residents Exasperated As Power Outages Linger Days After Tropical Storm Isaias
Filed Under:Flushing, Kevin Rincon, Local TV, New York, power outages, Queens, tropical storm isaias
NEW YORK (CBSNewYork) – Three days after Tropical Storm Isaias, and across the Tri-State Area there are still many homes and businesses who are without power.
In Flushing, Queens, downed power lines and damage from the storm has left many residents without electricity for days.
The destruction is widespread in Flushing, reported CBS2's Kevin Rincon. Trees knocked down power lines, landed on homes and crushed cars.
"That's devastating, across the street there, very, very nasty. Terrible," said Grace Dunn, referring to a downed tree.
For Dunn, the storm was unlike anything she's ever seen.
"We know it was coming. Did we think it was gonna be this bad? No," she said.
Her neighbor watched as the winds whipped through. What was on her mind?
"Holy cow, there was a lot of damage. A lot," the neighbor said.
Photos: Tropical Storm Isaias Leaves Path Of Destruction
That was Tuesday. It took until Friday for the city to send someone to clean up, when they deputized FDNY. Armed with chainsaws and a small tractor, they cleared the road. The crew was one of 25 units set up by the city's Office of Emergency Management. They'll try and clear some 10,000 downed trees reported in Queens alone.
"I just want somebody to come and take the tree," said Flushing resident Teresa Kanaval.
Kanaval had a tree come down on her roof. Her nephew says the timeline for removal is anywhere from six days to a month.
"I spoke to someone at Con Ed who's a supervisor who goes, 'Your house is not a priority. There's nothing in your house that's a priority,'" said Flushing resident Kenneth Rocca.
Click on the links below for information from utilities in your area.
Con Edison
NYSEG
National Grid
PSEG Long Island
Orange & Rockland
PSE&G New Jersey
Jersey Central Power & Light
Connecticut:
United Illuminating
For Rocca, the concern is about more than just his property. His mother Mary Ann, who also lives there, is a diabetic with a lung condition. Her medication is being kept cold in a refrigerator powered by an extension cord plugged into a neighbor's house. They've also relied on dry ice.
Con Ed set up several distribution centers for dry ice, and a number of people on those lines voiced frustration over the ongoing outages. Some believe they have nothing to do with the storm.
For now, Con Ed has been trying to restore power to tens of thousands. The utility says it has more than 1,600 field employees at work. It's also getting the help of more than 800 workers from as far away as Texas, Florida and Wisconsin.


text:
A journey through ShondaLand
Turn of the Screw
Talking comics
By Andy Fry
How to Get Away with Murder star Viola Davis, who spoke on the panel in Cannes
The Cannes Lions International Festival of Creativity is something of a surprise package. While it's aimed primarily at executives in the advertising business, its conference programme attracts some of the biggest names in film, TV and music.
This year, for example, it managed to reel in the likes of Pharrell Williams, David Guetta, Marilyn Manson, Richard Curtis, Kenneth Branagh, Freida Pinto, Norman Reedus and Chiwetel Ejiofor, to name just a few.
From a Writers Room perspective, one of the most interesting conference sessions was hosted by McCann Worldgroup and the Paley Centre for Media. Under the heading 'Is creativity the only way to survive and thrive today?' they brought together three top talents closely associated with ShondaLand, the Shonda Rhimes-led production company that makes hit shows Grey's Anatomy, Scandal, and How to Get Away with Murder (HTGAWM) – all for US network ABC.
It was also behind Private Practice, a Grey's spin-off that ran for six years.
One of the three contributors was producer Betsy Beers, who has been on board the ShondaLand express train since its early days. Beers, who has exec produced all of the above shows, recalled how, back in the middle of the last decade, both she and Rhimes were in the movie business.
"When we met, we had never made TV," she said. "Shonda was a successful movie writer but I sucked at making movies. Everything I did bombed. So when we started talking about making TV, I was really excited. I'd always been a closet TV fan, which was not something you could admit around movie people."
Grey's Anatomy is poised for a 12th season
Finding they shared a similar sensibility, Rhimes and Beers decided to make a show about female war correspondents. "We created a pilot, but were thinking in movie budget terms," said Beers. "So we came up with something that would have had a US$30m budget. Of course, that didn't work out. But we still wanted to work together so we hit on the idea of a medical drama."
'Medical drama' hardly sounds like the most original of premises, but in the hands of creative powerhouse Rhimes it became Grey's Anatomy, one of the most travelled shows in the world.
Beers continued: "Back then, a lot of the female characters on TV didn't look or sound like us. So what we set out to do was make a show about messy, confused, twisty women. In its first year it was the last show to get picked up as a pilot. And then it was the last show to go to series. But the audience liked the show so we survived."
Beers said ABC became increasingly supportive of the show, which has now run for 11 seasons (with a 12th on the way). She admitted there were some people in the business that found it sexually aggressive and, therefore, offensive "but our response was 'this is what the show is, so live with it.'"
ShondaLand's shows all have the ballsy quality displayed by Grey's Anatomy. Pete Nowalk, the creator of HTGAWM, came into the company as a writer on Grey's Anatomy. Also at the Cannes Lions session with Beers, he said: "You can't be generic anymore. There are so many TV shows out there that you have to really raise your game. With my work, I'm always looking at how to put ordinary people in extreme situations."
Grey's Anatomy spin-off Private Practice
While this approach has built a big fanbase, it inevitably exposes ShondaLand shows to the risk of criticism. Nowalk, who has grown up in ShondaLand, working on Grey's, Scandal and HTGAWM, said his way of dealing with this is to write "in a bubble. Constructive criticism makes you better and smarter, I think. But when I'm writing new episodes I keep it small — just me and my computer."
Beers said her ability to cope with adverse criticism is helped by the fact "that I border on being a luddite. But we've also created a safe environment at work, where we can express our fears and passions. I also think you need to really love what you are doing (to deal with criticism). We really love the shows we make."
One of the big successes of HTGAWM (which debuted in 2014 to strong audiences and positive reviews) was the casting of actress Viola Davis in the lead role as Annalise Keating. Nowalk recalled applauding Davis at the first read-through of his script "because Viola really brought my words to life."
Davis, who was the third participant in the Cannes Lions session, talked about the importance of authenticity in the way she handles the part. She said: "The role called for a messy, mysterious, sexy woman. I said 'yes,' then thought to myself, 'I don't look like that.' But I had an 'a-ha' moment. So what if I didn't fit the mould? I just dared to be a woman who fit those adjectives and in doing so was able to release my creativity. Typecasting is kryptonite to actors, so all I want to do is offer my interpretation of a real woman."
Davis's approach has helped make HTGAWM stand out. There was one particularly iconic scene in the fourth episode of season one where it was revealed that her character Annalise was wearing a wig and eyelash extensions.
Scandal is among the shows Pete Nowalk has worked on for ShondaLand
By removing them on screen, Davis made a statement about how women are represented on TV. She said: "I've never seen a woman like me on TV. It's important to bring that up. You have to give yourself permission to have a voice."
Beers said this gradual revelation of the vulnerabilitiesand contradictions of character is one of the beauties of working in serialised TV: "You have time to roll a character out."
To take advantage, however, you need to keep pushing at the uncomfortable boundaries of the character, added Davis. "For actors, TV's trap is that you can create a personality, not a person. The pursuit of likeability can be dangerous because you end up looking for gimmicks. You need to keep characters complex."
Nowalk said this is what he tries to do from a writing perspective, by making sure characters don't always follow the expected path, because it is the contradictions that come closest to truth.
All three panellists were asked what they feared most. Davis admitted to being "terrified not to have the courage to be authentic," while Nowalk said: "I'm always terrified by the next episode – right now, the debut episode of season two."
As for Beers, it was "the fear that I will stop growing, that I will start repeating myself. You have to learn, listen and be curious if you want to keep growing."
Right now, it seems the ShondaLand team is doing a good job of staying curious.
tagged in: Betsy Beers, Cannes Lions International Festival of Creativity, Grey's Anatomy, How To Get Away With Murder, Pete Nowalk, Private Practice, Scandal, Shonda Rhimes, ShondaLand, Viola Davis


The Tripartite structure has stimulated various streams of work over the years and this year's meeting reflected on the mutually significant work that has been undertaken recently, while also reviewing current activities.
At the meeting one of the milestones of the IMO Goal Based Standards (GBS) for bulk carriers and oil tankers, which entered into force for ships contracted for construction from July 2016 was noted. The Tripartite Meetings have helped, over a 5 year period, to oversee inter-industry agreement about the handling of the content of a Ship Construction File (SCF), as required by the GBS. The purpose of SCF is to provide information related to the structural design and construction of a ship to those that need it, to help ensure safe operation throughout the vessel's working life.
Among other Tripartite spin off groups, work is continuing on fuel data collection (as will be required by mandatory IMO and EU CO2 data collection systems), under the leadership of the International Association of Classification Societies (IACS). There will also be further work on cyber risks and security, also being led by IACS but assisted by BIMCO which led development of recent inter-industry guidance on the issue.
During the meeting, ICS raised the importance of collating knowledge on likely ship efficiency improvements from shipbuilders so that better estimates can be made of CO2 reduction performance, and will lead work on this in the coming year.
Following an introduction by Intertanko, all parties agreed that better information flow and data-basing was required on incidents related to ship construction and operation.
Reflecting on requirements directly arising from regulation, the meeting agreed that ICS would initiate the collection of experience with the fitting/retrofitting and operation of ballast water treatment equipment, and that INTERCARGO would lead work to develop appropriate designs for incorporating on-board storage/disposal facilities for HME (Hazardous to the Marine Environment) cargo residues, and HME cargo hold washing water.
Shipbuilder representatives (Committee for Expertise of Shipbuilding Specifics and SEA Europe) initiated discussion on human element issues, and in particular on training requirements arising out of innovative technology.
The Tripartite Meeting will reconvene in China in 2017.


text:
Home South Africa Appalling school dropout stats reveal crisis in Northern Cape municipality
Appalling school dropout stats reveal crisis in Northern Cape municipality
Only half the pupils who start primary school make it to high school in the Northern Cape's Joe Morolong Local Municipality. And of the pupils who make it to high school, just 15% matriculate.
These appalling statistics were revealed in a report released on Thursday by the Zero Dropout Campaign. The findings are based on data gathered by the Campaign, as well as in-depth interviews with school staff, pupils, families of the pupils and community members conducted in 2022.
The researchers, led by Dr Andrew Hartnack, looked at the causes of pupil disengagement and drop out at two Northern Cape no-fee schools in particular, Kgoro Intermediate School and Dithaba High (the names of the schools were changed for the report).
Kgoro Intermediate has eight teachers for its 263 pupils: 107 are in grade 7, 88 in grade 8, and 68 in grade 9. In 2022, at least 30 pupils in the grade 7 class were repeating the year. The school was initially built to accommodate half the number of children they now admit. The classrooms are overcrowded.
The report stated that many pupils at Kgoro have to leave by 2:30pm, often to go do household chores like herding livestock and gathering water.
Meanwhile at Dithaba High, there are 24 teachers for 679 pupils: 341 in grade 10, 178 in grade 11, and 160 in grade 12. The school is also overcrowded; one of the grades has 49 pupils in one class. The recommended class size is 32.
The report says most pupils at Dithaba High have already repeated grades. Despite struggling academically, many pupils are pushed through the intermediary school and often arrive in grade 10 aged 18 or 19 with serious learning gaps. Only half those who start grade 10 at Dithaba typically matriculate, the report adds.
ALSO READ: 'Dropouts are a reflection of bigger societal issues', says expert
Researchers also noted that the Department of Basic Education (DBE)'s "National policy pertaining to the programme and promotion requirements of the National Curriculum Statement" often pressures schools into promoting pupils, even if they are not ready academically.
According to the report, the norm in the community is that people do not feel the need to finish school. "[Children] don't see school as the key to a good future," the report read.
The major contributing factors to the high school dropout rates, according to the the report, are a lack of parental support, especially for children staying alone or with their grandparents as well as teenage pregnancy. Other factors include unsafe journeys to school, pressure to earn an income, and taking risks to get noticed by better-resourced groups (from criminal gangs to sugar daddies).
Speaking at the virtual launch, Coceko Nogoduka, chief director of care and support in schools at the DBE, said that the pandemic had disrupted the already struggling schooling system, losing nearly two years of learning and development.
Nogoduka said there had been a significant rise in pregnancies of children aged between 10 and 14-years-old. Pregnancy was one of the prominent reasons why pupils drop out between grades 10 and 12.
She said the department's policy, launched in 2021, "looks at how we can create a supportive environment for pregnant pupils to ensure that they stay in school for as long as possible. And also that after they deliver, they remain engaged, re-enter school, and are able to continue their education."
Nogoduka said: "Children don't just drop out of the schooling system. It's a process that often starts with disengagement and early warning signs."
She said the department and organisations working to mitigate dropouts, should focus on creating supportive environments within schools which would help to identify issues early.
Nogoduka said the department had developed a "learner support agent" programme to assist struggling pupils. "The vision is that the person will provide first-line kind of social support to the pupil. They will refer them to [for example] social development or NGOs in the area to help them remain in school," she said.
ALSO READ: EFF lauds matric class of 2022's success 'in the face of poor service delivery'
Richard Masemola, principal at Emshukantambo Secondary in Soweto, said at the launch that in most cases teachers were unaware of the pupils' circumstances. "We don't visit homes and we don't know where these children come from. When they don't show up to school, then we start to ask questions. But even then, it's at the tail end."
The Northern Cape education department did not respond to our requests for comment. Spokesperson for the Northern Cape Department of Education Geoffrey van der Merwe promised to respond to GroundUp's questions once he had received approval from the department.
This article first appeared on GroundUp and was republished with permission. Read the original article here.
NOW READ: Hoërskool Jan Viljoen: Government accused of concealing report into alleged school racism
Source: citizen
Previous articleGift of the Givers spends R40m on school upgrades after KZN floods
Next articleUCT: Government ready to step in as strike, investigations rock Africa's top university
Disruptive wind expected in parts of Western Cape on Saturday
Cheick Boucadry Traoré rend hommage à Guillaume Diallo
Par Slim Kooli: Fabulations d'un profane
Prix au marché de Béja [Vidéo]
Kadiolo : la 5ème édition journée culturelle des Samogho célébrée
EN RAISON DU BLOCAGE DE LEURS SUBVENTIONS PAR LE GOUVERNEMENT :...
Morocco Reports 13 New COVID-19 Cases over Past 24 Hours
Atul Gupta loses court bid to get South African passport
Limpopo Health MEC stands by her conduct after yet another viral...


Dynamic Munitions is a proud supporter of law enforcement and defense agencies. We are your resource for an efficient means of supply replenishment. Expect high quality customer service and the flexibility to support the needs of each department individually.
Reliable Law Enforcement Ammunition are always priced affordably at Dynamic Munitions. Order Dynamic Munitions ammunition designed specifically for high performance and backed by our Bullet Back Guarantee.
Please contact info@dynamicmunitions.com for more information on discounted pricing for bulk orders and LEO department purchases.


text:
Home Security News Cybersecurity News Apple now allows U.S. users to download their personal data via its...
Apple now allows U.S. users to download their personal data via its online privacy data portal
Savia Lobo
Yesterday, Apple started allowing U.S. users to download a copy of all their data the company stores as a part of their privacy data portal expansion. The company had announced this feature expansion earlier this year.
Per Bloomberg, prior to making this functionality available to U.S users, Apple rolled out the same functionality in Europe earlier this year as part of the European Union's General Data Protection Regulation (GDPR) rules.
With this effort, U.S. users will be able to download data such as all of their address book contacts, calendar appointments, music streaming preferences and details about past Apple product repairs. Previously, customers could not get their data without contacting Apple directly.
Apple launched its online privacy portal in May during which U.S users were allowed only to correct their data or delete their Apple accounts.
Apple has also added messages across its apps that tell users how their data is being handled. The company is also rolling out an updated privacy page on its website today detailing what data it does and does not store.
Apple says that it does not store much of user's data, which was confirmed by Zack Whittaker, a security editor at TechCrunch, when he asked Apple for his own data and the company turned over only a few megabytes of spreadsheets, including his order and purchase histories, and marketing information.
In his article on ZDNet, Zack says, "The zip file contained mostly Excel spreadsheets, packed with information that Apple stores about me. None of the files contained content information — like text messages and photos — but they do contain metadata, like when and who I messaged or called on FaceTime." He further added, "Any other data that Apple stores is either encrypted — so it can't turn over — or was only held for a short amount of time and was deleted."
About Apple's privacy policy updates, it refreshes its privacy pages once a year, a month after its product launches. It first launched its dedicated privacy pages in 2014. A year later, the company blew up the traditional privacy policy in 2015 by going more full-disclosure.
Zack says that, since then, Apple's pages have expanded and continued to be transparent on how the company encrypts user data on its devices.
To know more about how Apple encrypts user data in detail, visit Zack's post on ZDNet.
Apple bans Facebook's VPN app from the App Store for violating its data collection rules
Apple has introduced Shortcuts for iOS 12 to automate your everyday tasks
Apple buys Shazam, and will soon make the app ad-free
A Data science fanatic. Loves to be updated with the tech happenings around the globe. Loves singing and composing songs. Believes in putting the art in smart.
Must Read in Security
Microsoft adds Telemetry files in a "security-only update" without prior notice...
Savia Lobo - July 12, 2019 - 9:39 am
The recent Windows 7 'security-only' update also includes Telemetry components, which users may be unaware of. It may be used to secretly monitor individual...
Microsoft introduces passwordless feature in its Windows 10 devices, replaces it...
25 million Android devices infected with 'Agent Smith', a new mobile...