text:
# Demystifying Automatic Differentiation, Part 1¶

Recent results in deep and convolutional neural networks with powerful industrial applications have led to widespread adoption of these techniques outside of academia. One key advance has been the introduction of automatic differentiation, which allows developers and scientists to write code using standard numerical routines and functions that can be differentiated and used in training neural networks or other machine learning models. Many developers working with neural networks today, however, do not have a strong understanding of how automatic differentiation functions "under the hood," as well as how it differs from symbolic and numerical differentiation. In this article, using the style of literate programming, I will build some toy domain-specific languages that model how automatic differentiation functions in popular libraries like Tensorflow and PyTorch.

# Background¶

Please note that this is not a primer on automatic differentiation itself or the underlying theory. There are many high-quality sources on these topics, which I will not hope to outdo. The purpose of this article is to show, to those familiar with the concepts of AD but not the implementation, how simply automatic differentiation can be implemented.

For this reason I will present minimal background on AD and the alternatives and invite readers to search out these topics themselves.

There are roughly three ways to perform mathematical differentiation that most people are aware of:

## Symbolic differentiation¶

This is the application of rules on abstract symbols to analytically calculate formulas for the derivative of $f$ at $x$ (where $x$ may be high dimensional).

Symbolic differentiation involves a computer algebra system (CAS). Some people differentiate between symbolic differentiation and "manual" differentiation, where the latter is the same essential procedure as the former, but is performed, generally on paper, by a human.

In practice, this would be something like a CAS looking up a rule in a table to convert an expression $y = x^2$, consisting of exponentiation of a scalar value, to a derivative expression $\frac{dy}{dx} = 2x$, without ever considering the value of x itself. Often, this would occur without even considering the value of 2. Frege, at that point, is rolling in his grave. Alternatively, this would consist of a very bored 16-year-old sitting in math class looking up this rule in a table to generate the second expression.

## Numerical differentiation¶

This is the computation of an approximate derivative of $f$ at $x$ using numerical algorithms.

The most common techniques are forms of finite difference estimation, which consists of perturbing the value of $x$ and analyzing the resultant change in $f(x)$. Again, this can be done by massive linear algebra algorithms, or bored 16-year-olds.

While this technique seems hacky, it's commonly used in engineering and science. It's important to keep in mind that we are working with floating point numbers of finite precision in most cases, and sometimes a "hacky" method can max out our precision. At the same time, numerical differentiation requires significant attention to floating point arithmetic and error explosion.

Here, the derivative is computed from concrete values, and we do not obtain any structured expression.

## Automatic differentiation¶

This is the computation of an exact (to machine arithmetic precision) derivative of $f$ at $x$ by analyzing the computation of $f(x)$. Rather than analyzing symbols that express mathematical operations (as in symbolic differentiation), or perturbing computations that represent mathematical operations, we analyze a program that computes a function $f$. This analysis is far simpler than that of symbolic algebra expressions, and can be formalized into predictable algorithms and procedures.

The concept behind automatic differentiation is simple: every operation that our program is allowed to perform is shadowed by an operation that can compute the derivative of that operation. In practice, we record operations on values, and then use the "shadow" operations to compute derivatives of these operations using the chain rule. You can imagine all of the functions in your programming environment being doubled in this way. With the right abstraction layer, a normally written program can be made differentiable by an interpreter.

We could generalizing formalize this notion using Turing machines and differentiating operations on the tape, but this wouldn't be a very useful kind of differentiation in practice. In most cases, automatic differentiation is defined over a subset of a common programming language or library, or for a domain-specific language. By restricting the domain of AD to operations for which we can define efficient derivative operations, we gain a lot of expressive power and efficiency. An example is autograd, which replaces large portions of numpy and scipy with differentiable operations.

# Lsps¶

To demonstrate the mechanics of AD, we are going to build several languages.

Each of these languages is going to a "lsp" - or not quite a Lisp. Don't be fooled: they are missing some key features that most Lisps have. They aren't even all Turing complete. Lisp syntax is used, because it's easy to parse, and parsing is not the main interest here.

Each lsp is defined by the tuple $l = <\mathcal V, \mathcal O, \mathcal A, \mathcal E>$ which includes:

• A value type $V$, the set of valid computational results in the language.
• An operator type $\mathcal O$, the set of valid transformations $F: \mathcal V \to \mathcal V$
• An atomizer $\mathcal A:\mathcal S \to \mathcal V \cup \mathcal O$, converting atomic tokens from the set of valid tokens $\mathcal S$ (generally a set of strings without parens or whitespace) into value and operator type instances
• An environment $\mathcal E$ that contains predefined items in $\mathcal V \cup \mathcal O$. By default the environment is static, but it could be made dynamic in some lsps (though not all of those defined in this article).

The syntax rules are very similar to standard Lisp:

1. f the atomizer accepts an expression, that expression is valid.

2. If an expression $e$

• consists of a pair of matched parens with at least two subexpressions (a, b ...) separated by whitespace, and
• every subexpression $e$ contains is valid, and
• the first subexpression of $e$, a, evaluates to an operator type for this lsp,

then the whole expression is valid.

All other expressions are invalid.

We can parse a lsp quite easily, by tokenizing the input string and recursively processing parentheses contents. Parsing a tokenized string consists of converting all tokens except parens to the lsp's operator and value types, and then using the paren structure to create a nested list with identical structure.

In [1]:
import operator
from functools import reduce
import re
import math
from pprint import pprint
import numpy as np

In [2]:
class Lsp(object):
open_paren = re.compile("$") close_paren = re.compile("$")
splitter = re.compile("\s+")

def __init__(self, value_type, operator_type, atomizer, environment):
self.value_type = value_type
self.operator_type = operator_type
self.atomizer = atomizer
self.environment = environment

def parse_helper(self, tokens):
token = tokens.pop(0)
if token == "(":
els = []
while tokens[0] != ")":
next_expr, tokens = self.parse_helper(tokens)
els += [next_expr]

return els, tokens[1:]
elif token == ")":
raise ValueError()
else:
return self.atomize(token), tokens

def parse(self, tokens):
parsed, _ = self.parse_helper(tokens)
return parsed

def atomize(self, token):
if token in self.environment.keys():
return self.environment[token]
return self.atomizer(token)

def tokenize(self, code_str):
return list(filter(
lambda x: x is not '',
Lsp.splitter.split(Lsp.close_paren.sub(" ) ", Lsp.open_paren.sub(" ( ", code_str)))
))

We can evaluate an lsp expression by recursively evaluating the subexpressions from the "bottom up".

In [3]:
class EvalLsp(Lsp):
"""A lisp that may be evaluated."""
def eval_stack(self, stack):
if isinstance(stack, self.value_type) or isinstance(stack, self.operator_type):
# atom
return stack

fn = stack[0]

if isinstance(fn, list):
# nested function
return self.eval_stack(self.eval_stack(stack[0]) + stack[1:])

if isinstance(fn, self.operator_type):
# eval this function on the args
return fn(*map(self.eval_stack, stack[1:]))

raise ValueError(f"Unparsable value {fn}")

def eval_str(self, code_str):
raise NotImplemented()

## lsplsp¶

Lsplsp is about the simplest lsp langauge that we can make from this definition that isn't degenerate. It can run a few functions on Python ints.

In [4]:
class SimpleLsp(EvalLsp):
"""A DoubleLsp is an Lsp that returns the same value in the forward and backward results."""
def eval_str(self, code_str):
stack = self.parse(self.tokenize(code_str))
evaled = self.eval_stack(stack)
return (evaled)

In [5]:
LspLspValue = int
class LspLspOperator(object):
"""An lsplsp operator is just a Python function annotated with a symbol."""
def __init__(self, symbol, function):
self.symbol = symbol
self.function = function
def __call__(self, *args):
return self.function(*args)

In [6]:
int_token = re.compile("^\d+$") def lsplsp_atomizer(atom): """Lsplsp attempts to evaluate anything it can't find in it's environment as a Python int.""" if int_token.match(atom): return int(atom) raise ValueError()  In [7]: lsplsp_sum = LspLspOperator('+', lambda *args: sum(args)) lsplsp_diff = LspLspOperator('-', lambda *args: args[0] - sum(args[1:]))  In [8]: lsplsp = SimpleLsp( LspLspValue, LspLspOperator, lsplsp_atomizer, { f.symbol: f for f in [lsplsp_sum, lsplsp_diff] } )  In [9]: lsplsp.eval_str("(+ 5 2 (- 6 3))")  Out[9]: 10 ## Forward-backward lsps¶ The rest of the lsps we're going to take a look at are "forward-backward lsps," or fb-lsps for short. There are two special characteristics of fb-lsps: 1. Evaluation of an fb-lsp expression returns all the intermediate results in a nested structure. 2. Evaluation of an fb-lsp expression results in two results: a 'forward' results and a 'backward' result. Exactly what these two results will be used for is an inscrutable mystery for now... We can recycle lsplsp as fblsplsp, an fb-lsp that simply returns the same value in F and B results. It's cheating a little to call this an fb-lsp, and it doesn't provide us with anything new or useful, but we'll do it anyway. In [10]: class FBCheatLsp(EvalLsp): """An Lsp that that returns the same value in the forward and backward results.""" def eval_str(self, code_str): stack = self.parse(self.tokenize(code_str)) evaled = self.eval_stack(stack) return ( evaled, evaled )  In [11]: FBLspLspValue = int class FBLspLspOperator(object): """An fblsplsp function returns the argument values in a tuple with the result.""" def __init__(self, symbol, function): self.symbol = symbol self.function = function def __call__(self, *args): return (self.function(*(arg if isinstance(arg, FBLspLspValue) else arg[0] for arg in args)), args)  In [12]: fblsplsp_sum = FBLspLspOperator('+', lambda *args: sum(args)) fblsplsp_diff = FBLspLspOperator('-', lambda *args: args[0] - sum(args[1:]))  In [13]: fblsplsp = FBCheatLsp( FBLspLspValue, FBLspLspOperator, lsplsp_atomizer, { f.symbol: f for f in [fblsplsp_sum, fblsplsp_diff] } )  When we evaluate an fblsplsp expression, we get a tuple of the forward and backwards results. Each element of the tuple is a nested tuple, forming a tree of intermediate results. In [14]: pprint(fblsplsp.eval_str("(+ 5 2 (- 6 3 3))"))  ((7, (5, 2, (0, (6, 3, 3)))), (7, (5, 2, (0, (6, 3, 3)))))  Generally, our fb-lsps will compute two different values for the forward and backward result, with some shared computation. We can implement these as follows: In [15]: class FBHookLsp(EvalLsp): """An Lsp that computes the forward and backward results using a hook from stack evaluation.""" def eval_str(self, code_str): stack = self.parse(self.tokenize(code_str)) evaled = self.eval_stack(stack) return ( evaled.forward(), evaled.backward() )  # gradlsp¶ Time to get back on topic: automatic differentiation. gradlsp is an fb-lsp that returns a graph of conventional computation (just like fblsplsp) in the forward result, and returns a graph of gradients taken with respect to the final result in the backward result. We will define gradlsp to work over the domain of real numbers (approximated as Python floating-point values). In [16]: int_token = re.compile("^\d+$")
float_token = re.compile("^\d+(\.\d*)$") #string_token = re.compile("^\"([^\"]*)\"") def gradlsp_atomizer(token): if int_token.match(token): return GradLspValue(float(token)) if float_token.match(token): return GradLspValue(float(token)) raise ValueError(token)  gradlsp implements "reverse-mode" automatic differentiation. Any operation in gradlsp that takes arguments$a$and returns a value$v$records, in$v$, the necessary information to compute the gradients of each of$a$(and$a$'s ancestor arguments) with respect to$v$or a value computed by operations on$v$. The information stored in each value$v$is the complete graph of previous values leading to$v$'s computation, as well as the gradient of the function$f$that produced$v$at the point of evaluation. In theory, gradient computation of all subresults from one of these graphs requires a (computationally expensive) topological sort. However, gradlsp doesn't have any variables. Every operation on a value in gradlsp is pure with respect to the boxed value - it does not modify the original value. Furthermore, all operations are pure with respect to the value's gradient graph. Even the detach operation simply returns a new value with a degenerate graph. This ensures that the gradient graph is a DAG: a cycle in the graph would require a changed (impure) value. Because there are no cycles, we can treat the gradient graph as if it were a tree. This is very inefficient: if the DAG isn't treelike, we are performing wasted computations. In [17]: class GradLspValue(object): def __init__(self, value, gradient = [], children = [], symbol = None, operator_symbol=None): self.gradient = gradient self.children = children self.value = value self.symbol = symbol self.operator_symbol = operator_symbol def __repr__(self): return f"<GLV {self.value}, gradient={self.gradient}, with {len(self.children)} children>" def forward(self): return (self.value, self.operator_symbol or '?') + (( [child.forward() for child in self.children], ) if len(self.children) > 0 else tuple()) def backward(self, adjoint=1): if len(self.children) == 0: return adjoint return (adjoint, self.operator_symbol or '?') + (( [child.backward(adjoint * derivative) for child, derivative in zip(self.children, self.gradient)], ) if len(self.children) > 0 else tuple())  Furthermore, in gradlsp, we require every operator be scalar-valued. For this reason, the compute DAG for any value is by necessity a tree: every operation has only one parent. This decision reduces the power of gradlsp, as we will see soon. In [18]: class GradLspOperator(object): def __init__(self, symbol, function, grad_function): self.function = function self.grad_function = grad_function self.symbol = symbol def __repr__(self): return f"<GLF {self.symbol}>" def __call__(self, *args): result = GradLspValue( self.function(*map(lambda glv: glv.value, args)), self.grad_function(*map(lambda glv: glv.value, args)), args, operator_symbol=self.symbol ) return result  When computing the backward result, we propagate the gradient of the final value$y$backwards. This consists of repeated vector multiplication between the original value and the intermediate gradients. At each intermediate value$v$, we obtain an adjoint value$\frac{dy}{dv}$, the derivative of$v$with respect to the final result$y$. In [19]: plus = GradLspOperator( '+', lambda *args: sum(args), lambda *args: [1,] * len(args) )  In [20]: minus = GradLspOperator( '-', lambda *args: args[0] - sum(args[1:]), lambda *args: [1,] + [-1,] * (len(args) - 1) )  In [21]: prod = GradLspOperator( '*', lambda *args: reduce(operator.mul, args, 1), lambda *args: [reduce(operator.mul, (args[:i] + args[i+1:]), 1) for i in range(len(args))] )  In [22]: quot = GradLspOperator( '/', lambda x, y: x / y, lambda x, y: [1/y, -(x/(math.pow(y, 2)))] )  In [23]: sin = GradLspOperator( 'sin', lambda x: math.sin(x), lambda x: [math.cos(x)] )  In [24]: cos = GradLspOperator( 'cos', lambda x: math.cos(x), lambda x: [-math.sin(x)] )  In [25]: detach = GradLspOperator( '$',
lambda x: x,
[]
)

The detach operation ($foo) detaches the compute graph of foo from the returned result (which has the same value as foo), resulting in a new compute graph consisting of foo only. The (singular) node of the gradient DAG rooted at value, in any case, will be zero. This value, though it may depend on other values in actuality, is "held constant" from the standpoint of the gradient computation. In [26]: pi = GradLspValue(math.pi, symbol='pi')  In [27]: gradlsp = FBHookLsp( GradLspValue, GradLspOperator, gradlsp_atomizer, { f.symbol: f for f in [plus, minus, prod, quot, sin, cos, pi, detach] } )  In [28]: pprint(gradlsp.eval_str("(* (* 3 2) (cos pi))"))  ((-6.0, '*', [(6.0, '*', [(3.0, '?'), (2.0, '?')]), (-1.0, 'cos', [(3.141592653589793, '?')])]), (1, '*', [(-1.0, '*', [-2.0, -3.0]), (6.0, 'cos', [-7.347880794884119e-16])]))  # jacoblsp¶ gradlsp isn't very useful. It works on scalar valued functions only. It's hard to implement more than a best fit line with it. jacoblsp is our next step. Values in jacoblsp are tensors of arbitrary size (though many functions only accept 1D or 2D tensors). An operator in jacoblsp can express a function$\mathbb R^n \to \mathbb R^m$. Correspondingly, instead of the gradient, jacoblsp computes the Jacobian tensor for each operation. Instead of storing the local Jacobian itself in each value, and multiplying by the appropriate adjoint in the backward pass, we store a function that takes the adjoint and returns adjoint matrices for each of the generating function's arguments. This allows us to minimize space use in some operators, and concisely express others. Each value in jacoblsp is backed by an n-dimensional numpy array. An array is expressed as a literal 1,2,3. Trailing commas are allowed, but spaces are not. For instance, (*. 1,2,3 4,5,6) will compute the dot product of$[1,2,3]$with$[4,5,6]$Note that every value in jacoblsp is an array: scalar values are simply arrays of size one. Note that jacoblsp is still very inefficient. We compute every subtree of the compute DAG for every backwards operation. If our DAG isn't treelike, we are performing wasted computations. It's easier to express combinatorically complex programs in jacoblsp than it was in jacoblsp, so this problem is more worrying. However, we still have a treelike DAG, assuming that individual gradient functions do not branch. In [29]: class JacobLspValue(object): def __init__(self, value, adjoint_fns = None, children=[], shape=None, symbol = None, operator_symbol=None): self.adjoint_fns = adjoint_fns if adjoint_fns else [(lambda x: x) for c in children] self.children = children self.value = value self.symbol = symbol self.operator_symbol = operator_symbol self.shape = shape if shape else self.value.shape def __repr__(self): return f"<JLV {self.value}, with {len(self.children)} children>" def forward(self): if not self.children: return (self.value,) return ( self.value, self.operator_symbol or '?', [child.forward() for child in self.children] ) def backward(self, adjoint=None): if adjoint is None: adjoint = self.value if not self.children: return (adjoint,) return ( adjoint, self.operator_symbol or '?', [child.backward(adjoint_fn(adjoint)) for adjoint_fn, child in zip(self.adjoint_fns, self.children) ] )  In [30]: class JacobLspOperator(object): def __init__(self, symbol, function, max_args=None, max_dims=None): self.function = function self.symbol = symbol self.max_args = max_args self.max_dims = max_dims def __repr__(self): return f"<JLF {self.symbol}>" def __call__(self, *args): if self.max_args is not None and len(args) > self.max_args: raise ArgumentException() if self.max_dims and any(np.ndim(arg) > max_dim for arg, max_dim in zip(args, self.max_dims)): raise ArgumentException() arg_values = list(map(lambda jlv: jlv.value, args)) output_value, adjoint_fns = self.function(*arg_values) result = JacobLspValue( output_value, adjoint_fns, args if adjoint_fns else [], operator_symbol=self.symbol ) return result  In [31]: int_arr_token= re.compile("^-?(\d+,)*-?\d+,?$")
float_arr_token= re.compile("^(-?\d+(\.\d*)?,)*-?\d+(\.\d*)?,?$") def jacoblsp_atomizer(token): if int_arr_token.match(token): return JacobLspValue(np.array(list(map(int, filter(bool, token.split(',')))), dtype='int')) if int_arr_token.match(token): return JacobLspValue(np.array(list(map(float, filter(bool, token.split(',')))), dtype='float')) raise ValueError(token)  In [32]: jplus = JacobLspOperator( '+', lambda *args: ( np.stack(args).sum(axis=0), [(lambda g: g) for a in args] ) )  In [33]: jminus = JacobLspOperator( '-', lambda *args: ( args[0] - np.stack(args[1:]).sum(axis=0), [lambda g: g] + [(lambda g: -g) for a in args[1:]] ) )  In [34]: jprod = JacobLspOperator( '*', lambda *args: ( np.stack(args).prod(axis=0), [(lambda g: g.dot(np.stack(args[:i] + args[i+1:]))) for i in range(len(args))] ) )  In [35]: matvecdot = (lambda A, x: [ lambda g: np.tensordot(g, x, 0), lambda g: np.tensordot(g, A, [-1, 0]) ]) vecmatdot = (lambda a, B: [ lambda g: np.tensordot(g, B.T, 1), lambda g: np.tensordot(g, a, 0) ]) vecvecdot = (lambda a, b: [ lambda g: b, lambda g: a, ]) matmatdot = (lambda A, B: [ lambda g: np.tensordot(g, B.T, 1), lambda g: np.tensordot(g, A, [-2, 0]).T ]) selectdot = ({ (1,1): vecvecdot, (2,1): matvecdot, (1,2): vecmatdot, (2,2): matmatdot, })  In [36]: jdot = JacobLspOperator( '*.', lambda A, B: ( np.dot(A, B), selectdot[A.ndim, B.ndim](A,B) ), max_args = 2, max_dims = [2, 2] )  In [37]: jquot = JacobLspOperator( '/', lambda x, y: ( x / y, [lambda g: g/y, lambda g: g.dot(-(x/(math.pow(y, 2))))], ), max_args = 2 )  In [38]: jcos = JacobLspOperator( 'cos', lambda x: ( np.cos(x), [lambda g: g.dot(-np.sin(x))] ), max_args = 1 )  In [39]: jsin = JacobLspOperator( 'sin', lambda x: ( np.sin(x), [lambda g: g.dot(np.cos(x))], ), max_args = 1 )  In [40]: jfill = JacobLspOperator( 'full', lambda fill, *shp: ( np.full(shp[0] if len(shp) == 1 else shp, fill), None, ) )  In [41]: relu = JacobLspOperator( 'relu', lambda x: ( np.where(x < 0, 0, x), [lambda g: (g * np.where(x < 0, 0, 1)),], ), max_args=1 )  In [42]: stack = JacobLspOperator( 'stack', lambda *args: (np.stack(args), [lambda g: g[i] for i in range(len(args))]) )  In [43]: transpose = JacobLspOperator( 'T', lambda arg: (arg.T, [lambda g: g.T]) if len(arg.shape) == 2 else (arg.reshape((1,-1)), [lambda g: g.reshape(1,-1)]), max_args = 1, max_dims = [2] )  In [44]: jacoblsp = FBHookLsp( JacobLspValue, JacobLspOperator, jacoblsp_atomizer, { f.symbol: f for f in [jplus, jminus, jprod, jquot, jsin, jcos, jdot, jfill, relu, stack, transpose] } )  We now have all the building blocks to implement a neural network in jacoblsp, and take the network's gradient. As an example, we can build a classification network, which will compute the binary XOR of two inputs (with an appended bias term). It will consist of two layers with a single nonlinear activation. Using the inputs$(1,1), (0,1), (1,0), (0,0)$, we would expect to obtain the outputs$0, 1, 1, 0\$. This is a toy problem, for which layer weights can be given a priori, with no training necessary. The network is implemented as follows:

In [45]:
pprint(jacoblsp.eval_str('(*. (stack -2,1) (relu (*. (stack 1,1,-1 1,1,0) (T (stack 1,1,1 0,1,1 1,0,1 0,0,1)))))'))

((array([[0, 1, 1, 0]]),
'*.',
[(array([[-2,  1]]), 'stack', [(array([-2,  1]),)]),
(array([[1, 0, 0, 0],
[2, 1, 1, 0]]),
'relu',
[(array([[ 1,  0,  0, -1],
[ 2,  1,  1,  0]]),
'*.',
[(array([[ 1,  1, -1],
[ 1,  1,  0]]),
'stack',
[(array([ 1,  1, -1]),), (array([1, 1, 0]),)]),
(array([[1, 0, 1, 0],
[1, 1, 0, 0],
[1, 1, 1, 1]]),
'T',
[(array([[1, 1, 1],
[0, 1, 1],
[1, 0, 1],
[0, 0, 1]]),
'stack',
[(array([1, 1, 1]),),
(array([0, 1, 1]),),
(array([1, 0, 1]),),
(array([0, 0, 1]),)])])])])]),
(array([[0, 1, 1, 0]]),
'*.',
[(array([[0, 2]]), 'stack', [(array([0, 2]),)]),
(array([[ 0, -2, -2,  0],
[ 0,  1,  1,  0]]),
'relu',
[(array([[ 0, -2, -2,  0],
[ 0,  1,  1,  0]]),
'*.',
[(array([[-2, -2, -4],
[ 1,  1,  2]]),
'stack',
[(array([1, 1, 2]),), (array([1, 1, 2]),)]),
(array([[ 0, -1, -1,  0],
[ 0, -1, -1,  0],
[ 0,  2,  2,  0]]),
'T',
[(array([[ 0,  0,  0],
[-1, -1,  2],
[-1, -1,  2],
[ 0,  0,  0]]),
'stack',
[(array([0, 0, 0]),),
(array([0, 0, 0]),),
(array([0, 0, 0]),),
(array([0, 0, 0]),)])])])])]))

We can see that jacoblsp has computed the correct result and a gradient for it with each of the intermediate functions. In particular, we can see that it has computed the gradients for the layer weights. Appending a loss function:

In [46]:
pprint(jacoblsp.eval_str('(- (T 0,1,1,0) (*. (stack -2,1) (relu (*. (stack 1,1,-1 1,1,0) (T (stack 1,1,1 0,1,1 1,0,1 0,0,1))))))'))

((array([[0, 0, 0, 0]]),
'-',
[(array([[0, 1, 1, 0]]), 'T', [(array([0, 1, 1, 0]),)]),
(array([[0, 1, 1, 0]]),
'*.',
[(array([[-2,  1]]), 'stack', [(array([-2,  1]),)]),
(array([[1, 0, 0, 0],
[2, 1, 1, 0]]),
'relu',
[(array([[ 1,  0,  0, -1],
[ 2,  1,  1,  0]]),
'*.',
[(array([[ 1,  1, -1],
[ 1,  1,  0]]),
'stack',
[(array([ 1,  1, -1]),), (array([1, 1, 0]),)]),
(array([[1, 0, 1, 0],
[1, 1, 0, 0],
[1, 1, 1, 1]]),
'T',
[(array([[1, 1, 1],
[0, 1, 1],
[1, 0, 1],
[0, 0, 1]]),
'stack',
[(array([1, 1, 1]),),
(array([0, 1, 1]),),
(array([1, 0, 1]),),
(array([0, 0, 1]),)])])])])])]),
(array([[0, 0, 0, 0]]),
'-',
[(array([[0, 0, 0, 0]]), 'T', [(array([[0, 0, 0, 0]]),)]),
(array([[0, 0, 0, 0]]),
'*.',
[(array([[0, 0]]), 'stack', [(array([0, 0]),)]),
(array([[0, 0, 0, 0],
[0, 0, 0, 0]]),
'relu',
[(array([[0, 0, 0, 0],
[0, 0, 0, 0]]),
'*.',
[(array([[0, 0, 0],
[0, 0, 0]]),
'stack',
[(array([0, 0, 0]),), (array([0, 0, 0]),)]),
(array([[0, 0, 0, 0],
[0, 0, 0, 0],
[0, 0, 0, 0]]),
'T',
[(array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]),
'stack',
[(array([0, 0, 0]),),
(array([0, 0, 0]),),
(array([0, 0, 0]),),
(array([0, 0, 0]),)])])])])])]))

Here, the gradient is zero, because the loss is zero. Perturbing the layer weights results in nonzero gradients.

We have seen how the operators of a language may be incrementally extended to include gradient operators, and how this results in an automatically differentiable language. In the follow-up to this article, I will show how jacoblsp can be extended to include gradient descent with learnable parameters, and explore how to compute efficient gradients on non-treelike compute graphs.

The sources of this blog are available at Github. If you have any comments, corrections, or questions, please get in touch with me - I'd love to chat.


text:
STACK Documentation

Documentation home | Category index | Site map

An answer test is used to compare two expressions to establish whether they satisfy some mathematical criteria. The prototype test is to establish if they are algebraically equivalent. Answer tests are grouped as follows:

1. Equivalence, e.g. "algebraic equivalence" (many variations).
2. Syntactic form, e.g. "is in factored form".
3. Numerical accuracy, e.g. "is written to 3 decimal places".
4. Scientific, e.g. for dealing with dimensional numerical quantities.
5. Specific subject tests, e.g. sets, logical expressions, calculus (where tests provide feedback automatically in common situations such as a missing constant of integration).

We also provide string match tests. However, the whole point of STACK is that teachers should seek to establish mathematical properties and the string match tests are provided for completeness (and because they are trivial to implement). Experienced question authors almost never use the string match tests. If you find yourself needing to use the string match tests for something mathematical please contact the developers.

Introduction

Informally, the answer tests have the following syntax

[Errors, Result, FeedBack, Note] = AnswerTest(StudentAnswer, TeacherAnswer, [Opt], [Raw])

Where,

Variable Description
Opt If needed, any options which the specific answer test provides. For example, a variable, the accuracy of the numerical comparison, number of significant figures.
Raw If needed, the raw string of the student's input to ensure, e.g. Maxima does not remove trailing zeros when establishing the number of significant figures.

Note that since the tests can provide feedback, tests which appear to be symmetrical, e.g. Algebraic Equivalence, really need to assume which expression belongs to the student and which to the teacher.

Variable Description
Errors Hopefully this will be empty!
Result is either true, false, or fail (which indicates a failure of the test). This determines which branch of the tree is traversed.
FeedBack This is a text string which is displayed to the student. It is CASText which may depend on properties of the student's answer.
Note This is a text string which is used for Reporting. Each answer note is concatenated with the previous notes and any contributions from the branch.

The feedback is only shown to a student if the quiet option is set to 'no'. If feedback is shown, then examples are given in the answer-test test suite.

We expose the exact behaviour of each answer test by giving registered users access to STACK's test suite for STACK Answer tests. This can be found on a live server at .../moodle/question/type/stack/answertests.php. This script compares pairs of expressions and displays the outcomes from each test. This script is mainly used to ensure STACK is working, but it is invaluable for understanding what each test really does. In particular it enables question authors to see examples of which expressions are the same and different together with examples of the automatically generated feedback. We provide a static page giving the outcome of all answer test results.

In general

You can apply functions before applying the tests using the feedback variables. For example, to ignore case sensitivity you can apply the Maxima commands defined by STACK exdowncase(ex) to the arguments, before you apply one of the other answer tests. However, some tests really require the raw student's input. E.g. the numerical decimal place test really requires the name of an input as the SAns field. If you manipulate an input, you may end up dropping trailing zeros and ruining the number of decimal places in the expression. STACK will warn you if you need to use the name of an input.

Equivalence

A crucial component in the assessment process the ability to decide if two expressions are equivalent. It turns out there are many useful senses when trying to assess students' answers.

Let us assume a teacher has asked a student to expand out $$(x+1)^2$$ and the response they have from one student is $$x^2+x+x+1$$. This is "correct" in the sense that it is algebraically equivalent to $$(x+1)^2$$ and is in expanded form (actually two separate mathematical properties) but "incorrect" in the sense that the student has not gathered like terms by performing an addition $$x+x$$. What about a response $$2x+x^2+1$$? This is, arguably, better in the sense that the terms are gathered, but the student here did not order the terms to write their expression in canonical form. Hence, we need quite a number of different answer tests to establish equality in various senses of the word.

This list is in approximate order of the size of the equivalence classes from most to least restrictive.

Test Description (see below for more details)
CasEqual Are the parse trees of the two expressions equal?
EqualComAss Are they equal up to commutativity and associativity of addition and multiplication, together with their inverses minus and division?
EqualComAssRules Are they equal up to commutativity, associativity and with optional rules?
AlgEquivNouns Are they algebraically equivalent, preserving noun forms of operators, e.g. diff?
AlgEquiv Are they algebraically equivalent?
SubstEquiv Can we find a substitution of the variables of $$ex_2$$ into $$ex_1$$ which renders $$ex_1$$ algebraically equivalent to $$ex_2$$?
SameType Are the two expressions of the same types of object? Note that this test works recursively over the entire expression.

AlgEquiv

This is the most commonly used test. The pseudo code

If
simplify(ex1-ex2) = 0
then
true
else
false.

This test will work with a variety of types of object of mathematical objects, including lists, sets, equations, inequalities and matrices.

• This test disregards whether simplification is switched on, it always fully simplifies all its arguments.
• Use AlgEquiv(predicate(ex),true) with predicate functions.

Note: exactly what it does depends on what objects are given to it. In particular the pseudo code above only applies to expressions. We cannot subtract one list or set from another, so we have to use other tests.

For sets, the CAS tries to write the expression in a canonical form. It then compares the string representations these forms to remove duplicate elements and compare sets. This is subtly different from trying to simplify the difference of two expressions to zero. For example, imagine we have $$\{(x-a)^{6000}\}$$ and $$\{(a-x)^{6000}\}$$. One canonical form is to expand out both sides. While this work in principal, in practice this is much too slow for assessment.

Currently we do check multiplicity of roots, so that $$(x-2)^2=0$$ and $$x=2$$ are not considered to be equivalent. Similarly $$a^3b^3=0$$ is not $$a=0 \mbox{ or } b=0$$. This is a long-standing issue and we would need a separate test to ignore multiplicity of roots.

Currently, $$\{-\frac{\sqrt{2}}{\sqrt{3}}\}$$ and $$\{-\frac{2}{\sqrt{6}}\}$$ are considered to be different. If you want these to be considered the same, you need to write them in a canonical form. Instead of passing in just the sets, use the answer test to compare the following.

ev(radcan({-sqrt(2)/sqrt(3)}),simp);

Why doesn't the test automatically apply radcan? If we always did this, then $$\{(x-a)^{6000}\}$$ and $$\{(a-x)^{6000}\}$$ would be expanded out, which would break the system. Since, in a given situation, we know a lot about what a student is likely to answer we can apply an appropriate form. There isn't one rule which will work here, unfortunately.

There are also some cases which Maxima can't establish as being equivalent. For example $\sqrt[3]{\sqrt{108}+10}-\sqrt[3]{\sqrt{108}-10} = 2.$ As Maxima code

(sqrt(108)+10)^(1/3)-(sqrt(108)-10)^(1/3)

This is Cardano's example from Ars Magna, but currently the AlgEquiv test cannot establish these are equivalent. There are some other examples in the test suite which fail for mathematical reasons. In cases like this, where you know you have a number, you may need to supplement the AlgEquiv test with another numerical test.

EqualComAss

A particularly useful test is to establish that two expressions are equal up to commutativity and associativity of addition and multiplication, together with their inverses minus and division. For example, under this test $$x+y = y+x \mbox{ but } x+x \neq 2x$$. Please see the separate documentation.

AlgEquivNouns

Algebraic equivalence evaluates as much as possible, to try to establish equivalence. This means, e.g. that diff(y,x) is always evaluated to $$0$$. If you use AlgEquivNouns then noun forms of operators are not evaluated, so diff(y,x) will be evaluated but 'diff(y,x) and noundiff(y,x) will not.

Note, however that logic nouns such as nounand are still evaluated by this test! Sorry, but logical noun functions are dealt with internally in a very different way than Maxima noun functions such as 'diff(y,x) and the parallel noundiff. Use a different test, such as EqualComAss.

CasEqual

The CAS returns the result of the simple Maxima command

if StudentAnswer=TeacherAnswer then true else false.

There is no explicit simplification here (unlike AlgEquiv). This test always assumes simplification is off, i.e. simp:false, regardless of any question settings. If this is too strict, use ev(ex,simp) in the arguments to simplify them explicitly first. When simplification is off this test effectively tests whether the parse trees are identical.

Please note, the behaviour of this test relies on the internal representation of expressions by Maxima, rather than an explicit mathematical property such as "equivalence". Explicit properties should be tested in preference to using this test!

SubstEquiv

Can we find a substitution of the variables of $$ex_2$$ into $$ex_1$$ which renders $$ex_1$$ algebraically equivalent to $$ex_2$$?

• Because we have to test every possibility, the algorithm is factorial in the number of variables. For this reason, the test only works for 4 or fewer variables.
• This test makes a substitution then uses AlgEquiv.
• If you add an answer test option (not required) in the form of a list of variables, these variables will be "fixed" during the comparison. E.g.
• ATSubstEquiv(x=A+B, x=a+b) will match with [A = a,B = b,x = x].
• ATSubstEquiv(x=A+B, x=a+b, [x]) will match with [A = a,B = b].
• ATSubstEquiv(y=A+B, x=a+b, [x]) will not match since x in the teacher's answer is fixed here.

The optional argument, which must be a list of variables, is useful if you want to establish that a student has used arbitrary constants in $$A\sin(x)+B\cos(x)$$ but make sure $$x$$ really stays as $$x$$.

SysEquiv

The SysEquiv (system equivalence) test takes in two lists of polynomial equations in any number of variables and determines whether the two systems have the same set of solutions. This is done using the theory of Grobner bases to determine whether the ideals generated by the two systems are equal. As the test allows for polynomials in several variables, it can cope with the intersections of the conic sections, as well as a large number of geometrically interesting curves.

• This test does not check if the student actually "fully solved" the equations! E.g. $[x^2=1] \equiv [(x-1)\cdot (x+1)=0]$ under this test.
• This test disregards whether simplification is switched on, it only simplifies its arguments where required. This allows the test to list equations in feedback that the student has erroneously included in their system.
• You can allow the student to include "redundant assignments". For example, if you have [90=v*t,90=(v+5)*(t-1/4)] but the student has [d=90,d=v*t,d=(v+5)*(t-1/4)]) then the systems are not equivalent, becuase the student has an extra variable. Use stack_eval_assignments to eliminate explicit assignments of the form var=num and evaluate the other expression in this context.

Sets

This test deals with equality of sets. The algebraic equivalence functions give very minimal feedback. This test is designed to give much more detailed feedback on what is and is not included in the student's answer. Hence, this essentially tells the student what is missing. This is kind of feedback is tedious to generate without this test.

The test simplifies both sets, and does a comparison based on the simplified versions. The comparison relies on ev(..., simp, nouns) to undertake the simplification. If you need stronger simplification (e.g. trig) then you will need to add this to the arguments of the function first.

Equiv and EquivFirstLast

These answer tests are used with equivalence reasoning. See the separate documentation.

Form

Often, we wish to establish if the student's expression has the correct form. For example, consider the following various written forms of $$x^2-4x+4$$.

$(x-2)(x-2),\quad (x-2)^2,\quad (2-x)^2,\quad 4\left(1-\frac{x}{2}\right)^2.$

Each of these might be considered to be factored. Establishing ex is factored is not the same as comparing it with factor(ex).

Related tests establish that an expression is expanded or in partial fraction form.

Expression Description
LowestTerms This test checks that all numbers written in the first expression are in lowest terms and that the denominator is clear of surds and complex numbers. Notes
* if you want to check whether a rational polynomial is written in lowest terms, this is not the test to use. Instead, apply the predicate functions lowesttermsp to the expression.
* the second argument to this function is ignored, i.e. this test does not confirm algebraic equivalence. You might as well use 0 here.
SingleFrac This test checks that SAns written as a "single fraction". See below.
Expanded Confirms SAns is equal to expand(SAns). Note, the second argument to this answer test is not used (but must be non-empty). Note with this test that an expression such as $$x^2-(a+b)x+ab$$ is not considered to be expanded, and this test will return false.
FacForm This test checks (i) that SAns is algebraically equivalent to TAns , and (ii) that SAns is "factored" over the rational numbers. See below for more details. The answer test expects the option to be the variable, which is needed to generate feedback. If the answer is incorrect, quite detailed feedback is provided.
PartFrac This test checks (i) that SAns is algebraically equivalent to TAns , and (ii) that SAns is in "partial fraction form". The option must be the variable.
CompletedSquare This test checks (i) that SAns is algebraically equivalent to TAns , and (ii) that SAns is in "completed square form". The option must be the variable.

Single fractions

This test checks (i) that SAns is algebraically equivalent to TAns , and (ii) that SAns is written as a single fraction. Notes

• This test works at the top level, making sure the expression as a whole is a single fraction.
• if you also want this expression written in lowest terms, then this is quite a separate test. You need to first confirm you have a single fraction then add a new potential response. One way is to use the ../CAS/Predicate functions lowesttermsp(ex) and compare the result with true with the AlgEquiv test.
• The algebraic equivalence check is for convenience. If you only want to check an expression is a single fraction make $$SAns=TAns$$, i.e. ATSingleFrac(ex,ex) will do.

Factorisation of polynomials

An expression is said to be factored if it is written as a product of powers of distinct irreducible factors. Strictly speaking, in establishing that an expression is in factored form, we might not even care whether the terms in the product are fully simplified, as long as they are irreducible.

Irreducibility on the other hand means we can't find further factors, but here we need some care.

Consider $$x^8+16x^4+48$$.

1. Any non-trivial factorization, e.g. $$(x^4+4)(x^4+12)$$.
2. A factorization into irreducible factors over the integers/rational numbers, i.e. $$(x^2+2x+x)(x^2-2x+2)(x^4+12)$$.
3. A factorization into terms irreducible over the reals, i.e. $$(x^2+2x+x)(x^2-2x+2)(x^2+2\sqrt[4]{3}x+2\sqrt[4]{3})(x^2-2\sqrt[4]{3}x+2\sqrt[4]{3})$$.
4. A factorization into irreducible polynomials over the Gaussian integers, with $$i$$ allowed, i.e. $$(x+1+i)(x+1-i)(x-1+i)(x-1-i)(x^4+12)$$.
5. A factorization over the complex numbers, where the factor $$(x^4+12)$$ would also be split into the four terms $$x\pm\sqrt[4]{3}(1\pm i)$$.

In elementary teaching, meaning 4. is unlikely to occur. Indeed, we might take this example to represent factoring over any extension field of the rational numbers. We normally seek to establish that the factors are irreducible over the integers (which is equivalent to irreducibility over the rational numbers) or the reals. But, unlike a canonical form, we are not particularly interested in the order of the terms in this product, or the order of summands inside these terms.

The FacForm test establishes that the expression is factored over the rational numbers. If the coefficients of the polynomial are all real, at worst you will have quadratic irreducible terms. There are some delicate cases such as: $$(2-x)(3-x)$$ vs $$(x-2)(x-3)$$ and $$(1-x)^2$$ vs $$(x-1)^2$$, which this test will cope with.

It is also possible a student will do something which is just plain odd, e.g. $$x^2-4x+4$$ can be rewritten as $$x(x-4+4/x)$$ which is a "product of powers of distinct irreducible factors" but not acceptable to most teachers. The student's answer must also be a polynomial in the variable (using polynomialp as the test predicate).

Factorisation of integers

If you would like to ask a student to factor a polynomial, then do not use the FacForm answer test. The FacForm answer test is designed to use with polynomials.

Instead, switch off simplification and define

ta:factor(12);

and use EqualComAss as the answer test.

Note however that EqualComAss does not think that 2^2*3 and 2*2*3 are the same!

Numerical Precision

These tests deal with the precision of numbers. See dedicated page on numerical answer tests.

Scientific units

A dedicated answer test for scientific units is described on the units page.

Tests for specific subject areas

PropLogic

An answer test designed to deal with propositional logic. See the separate documentation.

Calculus: Diff

This test is a general differentiation test: it is passed if the arguments are algebraically equivalent, but gives feedback if it looks like the student has integrated instead of differentiated. The first argument is the student's answer. The second argument is the model answer. The answer test option must be the variable with respect to which differentiation is assumed to take place.

There are edge cases, particularly with $$e^x$$ where differentiation is indistinguishable from integration. You may need to use the "quiet" option in these cases.

Calculus: Int

This test is designed for a general indefinite integration question: it is passed if both the arguments are indefinite integrals of the same expression. The first argument is the student's answer. The second argument is the model answer. The answer test option needs to be the variable with respect to which integration is assumed to take place, or a list (see below).

Getting this test to work in a general setting is a very difficult challenge. In particular, the test assumes that the constant of integration is expressed in a form similar to +c, although which variable used is not important.

The issue of $$\int \frac{1}{x} dx = \log(x)+c$$ vs $$\int \frac{1}{x} dx = \log(|x|)+c$$ is a particular challenge. The test is currently defined in such a way that if the teacher uses $$\log(|x|)+c$$ in their answer, then they would expect the student to do so. If they don't use the absolute value function, then they don't expect students to but will accept this in an answer. It is, after all, not "wrong". However, in the case of partial fractions where there are more than one term of the form $$\log(x-a)$$ then we insist the student is at least consistent. If the teacher has any $$\log(|x-a|)$$ then the student must use $$|...|$$ in all of them. If the teacher has no $$\log(|x-a|)$$ (i.e. just things like $$\log(x-a)$$) then the student must have all or none.

The Int test has various additional options.

The question author must supply these options in the form of a list [var, opt1, ...]. The first argument of this list must be the variable with respect to which integration is taking place.

If one of the opt? is exactly the token NOCONST then the test will condone a lack of constant of integration. That is, if a student has missed off a constant of integration, or the answers differ by a numerical constant, then full marks will be awarded. Weird constants (e.g. $$+c^2$$) will still be flagged up.

The answer test architecture only passes in the answer to the test. The question is not available at that point; however, the answer test has to infer exactly which expression, including the algebraic form, the teacher has set in the question. This includes stripping off constants of integration and constants of integration may occur in a number of ways, e.g. in logarithms. In many cases simply differentiating the teacher's answer is fine, in which case the question author need not worry. Where this does not work, the question author will need to supply the expression from the question in the right form as an option to the answer test. This is done simply by adding it to the list of options.

[x, x*exp(5*x+7)]

String match

String This is a string match, ignoring leading and trailing white space which are stripped from all answers, using PHP's trim() function.

StringSloppy This function first converts both inputs to lower case, then removes all white space from the string and finally performs a strict string comparison.

SRegExp

This test uses Maxima's regex_match function.

• Both arguments to the test must be Maxima strings. If you have a general expression, turn it into a string in the feedback variables with Maxima's string function.
• The first argument should be the string, and the second argument should be the pattern to match.
• It yields true if the pattern is matched anywhere within the student answer and false otherwise. Testing for full equality of the answer string can be achieved via regex anchoring by use of ^ or \$.
• Don't forget to escape within the pattern strings as needed. Note that there is a function string_to_regex() that will handle escaping of characters that would otherwise have meaning in the pattern. Also remember that you need to escape the backslashes like normal in Maxima-strings.
• One can read more about the patterns posible from here. Case-insensitivity may be something worth noting there.

STACK also provides a helper function regex_match_exactp(regex, str) to check if the string equals the pattern matched by the regular expression.

Regex           String      Result
(aaa)*(b|d)c    aaaaaabc    true
(aaa)*(b|d)c    dc          true
(aaa)*(b|d)c    aaaaaaabc   false
(aaa)*(b|d)c    cca         false

Currently this is not provided as a separate answer test so you will need to use this predicate in the question variables and check the result against the expected value, or supply the predicate as an argument to an answer test.

(RegExp) NOTE: this test was removed in STACK version 4.3.

format: plain


text:
Posted: Aug 16, 2015 2:06 am
chango369 wrote:Pulsar's Latex Tutorial Part 3 - reboot

Pulsar wrote:

Code: Select all
$\textbf{A }\mathbf{\LaTeX\ }\textbf{Tutorial, part 3}$

Arrows

Code: Select all
\begin{align}&\leftarrow \quad \longleftarrow \quad \Leftarrow \quad \Longleftarrow \quad \rightarrow \quad \longrightarrow \quad \Rightarrow \quad \Longrightarrow \quad\overleftarrow{abc} \quad \overrightarrow{abc} \\&\leftrightarrow \quad \longleftrightarrow \quad \Leftrightarrow \quad \Longleftrightarrow \quad \leftrightarrows \quad \leftleftarrows \quad \rightrightarrows \quad \uparrow \quad \Uparrow \quad \downarrow \quad \Downarrow \quad \updownarrow \quad \Updownarrow\end{align}

If you want an arrow with additional text, use \xleftarrow or \xrightarrow. Be aware of the precise format: text below is placed between square brackets [] and can be omitted, while text above is placed between mandatory curly brackets {}.

Code: Select all
$\xrightarrow{\text{above the arrow}} \qquad \xleftarrow[\text{below}]{}\qquad \xrightarrow[\text{below}]{\text{above the arrow}}$

In combination with the \lim operator, you can write something like this:

Code: Select all
$\lim_{x \rightarrow +\infty} f(x) = 0$

This doesn't look very pretty, but it can be tidied up by enclosing the equation by a \displaystyle{} environment:

Code: Select all
$\displaystyle{\lim_{x \rightarrow +\infty} f(x) = 0}$

Various operators

Again, use \displaystyle{} for nice results:

Code: Select all
$\displaystyle{\sum \quad \sum_{i=1}^{n} \quad \sum_{\substack{i,j \\ i > j}} \quad \prod \quad \prod_{n=0}^{\infty} \quad \coprod\quad \int \quad \int_0^1 \quad \oint \quad \iint \quad \iiint \quad \sqrt{x} \quad \sqrt[3]{26}}$

Note the \substack{} command in the third sum, to stack multiple limits. Fractions and binomials looks as follows, without and with \displaystyle{}:

Code: Select all
$\frac{1}{2} \quad \binom{n}{k}\qquad\displaystyle{\frac{1}{2} \quad \binom{n}{k}}$

Brackets

Code: Select all
$\displaystyle{() \quad [] \quad \{ \} \quad | \quad \langle \rangle \quad \lfloor \rfloor \quad \lceil \rceil \quad \Vert}$

Note the backslash in \{ and \} to produce curly brackets. The size of these delimiters can be increased manually by preceding them with \big, \Big, \bigg or \Bigg. You can also let Latex determine an appropriate size, by using \left and \right. However, every \left delimiter has to be followed by a similar \right delimiter; in case you want only one delimiter, use a dot for the other, i.e. \left. or \right. to generate an empty delimiter. See the examples:

Code: Select all
$\displaystyle{\Bigg( \bigg( \Big( \big( () \big) \Big) \bigg) \Bigg)\qquad \left(\frac{a}{b}\right) \qquad \left[\frac{A}{B} + \left( C + \sqrt{D}\right) \right]\qquad \left\langle\left. \psi_1 \right. \right|}$

The \underbrace{} and \overbrace{} commands produce

Code: Select all
$\underbrace{\underbrace{a + b}_\text{brace 1} +\overbrace{c + d}^\text{brace 2}}_\text{brace 3}= e$


text:
The problem is that you are comparing lists, not Sage objects. achrzesz gives an answer which compares two Sage group elements, so they are equal. The lists you give are simple Python ordered lists, and as ordered things, certainly aren't the same. Why didn't you just compare P[3]*P[6] and P[4]?
sage: P[3] * P[6]


text:
# Bond Basics—Straight-Line Method, Retirement, and Conversion p6. Cozumel Corporation has...

Bond Basics—Straight-Line Method, Retirement, and Conversion

p6. Cozumel Corporation has $10,000,000 of 10.5 percent, 20-year bonds dated June 1, 2014, with interest payment dates of May 31 and November 30. After 10 years, the bonds are callable at 104, and each$1,000 bond is convertible into 25 shares of \$20 par value common stock. The company’s fiscal year ends on December 31. It uses the straight-line method to amortize bond premiums or discounts.

ReQUIReD

1.    Assume the bonds are issued at 103 on June 1, 2014.

a.    How much cash is received?

b.    How much is Bonds Payable?

c.    What is the difference between a and b called, and how much is it?

d.   With regard to the bond interest payment on November 30, 2014:

(1)     How much cash is paid in interest?

(2)     How much is the amortization?

(3)     How much is interest expense?

2.    Assume the bonds are issued at 97 on June 1, 2014.

a.    How much cash is received?

b.    How much is Bonds Payable?

(Continued)

c.    What is the difference between a and b called, and how much is it?

d.   With regard to the bond interest payment on November 30, 2014:

(1)      How much cash is paid in interest?

(2)      How much is the amortization?

(3)      How much is interest expense?

3.    Assume the issue price in requirement 1 and that the bonds are called and retired 10 years later.

a.    How much cash will have to be paid to retire the bonds?

b.    Is there a gain or loss on the retirement, and if so, how much is it?

4.    Assume the issue price in requirement 2 and that the bonds are converted to com- mon stock 10 years later.

a.    Is there a gain or loss on the conversion, and if so, how much is it?

b.    How many shares of common stock are issued in exchange for the bonds?

c.    In dollar amounts, how does this transaction affect the total liabilities and the total stockholders’ equity of the company? In your answer, show the effects on four accounts.

5.    Business appliCation ▶Assume that after 10 years, market interest rates have dropped significantly and that the price on the company’s common stock has risen significantly. Also assume that management wants to improve its credit rating by reducing its debt to equity ratio and that it needs what cash it has for expansion. Which approach would management prefer—the approach and result in require- ment 3 or 4? Explain your answer. What would be a disadvantage of the approach you chose?


text:
# Let f(x)=x^3+x^2+x+1\in Z_2[x]. Write f(x) as a product of irreducible polynomials over Z_2

Let $$\displaystyle{f{{\left({x}\right)}}}={x}^{{3}}+{x}^{{2}}+{x}+{1}\in{Z}_{{2}}{\left[{x}\right]}$$. Write f(x) as a product of irreducible polynomials over $$\displaystyle{Z}_{{2}}$$

• Questions are typically answered in as fast as 30 minutes

### Plainmath recommends

• Get a detailed answer even on the hardest topics.
• Ask an expert for a step-by-step guidance to learn to do it yourself.

Benedict
Given:
$$\displaystyle{f{{\left({x}\right)}}}={x}^{{3}}+{x}^{{2}}+{x}+{1}$$
The given polynomial can be factorized as a product two irreducible polynomials as,
$$\displaystyle{x}^{{3}}+{x}^{{2}}+{x}+{1}={x}^{{2}}{\left({x}+{1}\right)}+{\left({x}+{1}\right)}$$
$$\displaystyle={\left({x}+{1}\right)}{\left({x}^{{2}}+{1}\right)}$$
Here, the polynomials (x+1) is in irreducible form which cannot be factorized further.
$$\displaystyle{\left({x}^{{2}}+{1}\right)}$$ can be factorized over Z as,
$$\displaystyle{x}^{{2}}+{1}={\left({x}+{i}\right)}{\left({x}-{i}\right)}$$
Here, both (x+i) and (x-i) are in irreducible form.
Hence, the given polynomial can be expressed as a product of irreducible polynomial over Z as,
$$\displaystyle{x}^{{3}}+{x}^{{2}}+{x}+{1}={\left({x}+{1}\right)}{\left({x}-{i}\right)}{\left({x}+{i}\right)}$$


text:
+0

0
87
3

Four positive integers p, q, r, s satisfy the following equations:

\begin{align*} pq+2p+q&=348 \\ qr+4q+3r&=373 \\ rs+8r+6s&=544 \end{align*}
What are p,q, r, and s?

Mar 25, 2020

#1
+1968
+1

Hint:

For this question, multiply one equation so that one variable can be subtracted!

For example, multiply pq+2+q by 2 so that it can be equal to the second equation.

Hope this helped!

Mar 25, 2020
#2
0

p=1; q=1;r=1;s=1;p=1;m=0;n=p+q+r+s;if(n==348 and 373 and 544, goto8, goto next);m=(a*b+b*c+c*d);max(m)==p;print max(m);next:a++;if(a<55, goto5, 0);a=1;b++;if(b<55, goto5, 0);a=1;b=1;c++;if(c<55, goto5,0);a=1;b=1;c=1;d++;if(d<55, goto5, discard=0;

OUTPUT: r = 31 and s = 8 and p = 34 and q = 8

Mar 25, 2020
#3
0


text:
An Immediately-invoked Function Expression (IIFE for friends) is a way to execute functions immediately, as soon as they are created.

IIFEs are very useful because they don’t pollute the global object, and they are a simple way to isolate variables declarations.

This is the syntax that defines an IIFE:

(function() {
/* */
})()

IIFEs can be defined with arrow functions as well:

(() => {
/* */
})()

We basically have a function defined inside parentheses, and then we append () to execute that function: (/* function */)().

Those wrapping parentheses are actually what make our function, internally, be considered an expression. Otherwise, the function declaration would be invalid, because we didn’t specify any name:

Function declarations want a name, while function expressions do not require it.

You could also put the invoking parentheses inside the expression parentheses, there is no difference, just a styling preference:

(function() {
/* */
}())

(() => {
/* */
}())

## Alternative syntax using unary operators

There is some weirder syntax that you can use to create an IIFE, but it’s very rarely used in the real world, and it relies on using any unary operator:

-(function() {
/* */
})() +
(function() {
/* */
})()

~(function() {
/* */
})()

!(function() {
/* */
})()

(does not work with arrow functions)

## Named IIFE

An IIFE can also be named regular functions (not arrow functions). This does not change the fact that the function does not “leak” to the global scope, and it cannot be invoked again after its execution:

(function doSomething() {
/* */
})()

## IIFEs starting with a semicolon

You might see this in the wild:

;(function() {
/* */
})()

This prevents issues when blindly concatenating two JavaScript files. Since JavaScript does not require semicolons, you might concatenate with a file with some statements in its last line that causes a syntax error.

This problem is essentially solved with “smart” code bundlers like webpack.

## One more thing! ⚠ ✋

At the end of January I will organize the Web Development Bootcamp.

It's a 10-weeks long cohort online course where I will guide you to becoming a Web Developer.

It's not "just a course". It's a big event I organize once a year.

We'll start from zero, learn the fundamentals of Web Development, HTML CSS, JavaScript, Tailwind, Git, using the command line, VS Code, GitHub, Node.js, we'll then learn React, JSX, how to use PostgreSQL, Astro, Next.js, Prisma, deploying on Netlify/DigitalOcean/Fly/Vercel and much more!

At the end of the first 10 weeks you'll know how to create web sites and web applications and I'll unlock you the 2nd phase of the Bootcamp: you will get access to a large number of projects exclusive to the Bootcamp graduates, so you can follow my instructions to build things like private areas with authentication, clones of popular sites like Twitter YouTube Reddit, create e-commerce sites, and much much more.

Because once you got the fundamentals, you only learn by working on real, exciting projects.

To find out more, visit bootcamp.dev


text:
Displacement vecors

1. Jan 18, 2009

Kitty808

1. The problem statement, all variables and given/known data
A runner jogs around a circular track 160 ft in diameter, what is the magnitude of the displacement vector when the runner has completed 1/4 of a lap? 1/2 a lap? What is the direction of the runner's displacement vector at 1/4 and 1/2 a lap? What is the magnitude of the runners displacement vector when a full lap is completed?

3. The attempt at a solution
Ok, I am pretty much clueless on this, even after spending an hour reading my text book. (I'm -really- bad at physics). I calculated the circumference of the track to be 502 ft which makes 1/4 of the track 125.5 ft. But the online software said that the magnitude of 1/4 of a lap should be 113 ft. How did they get this number. Also the magnitude of a complete lap is obviously 0 ft, but I'm not sure how to go about solving for the directions either. Any help would be appreciated, like I said, I'm pretty clueless XD.

2. Jan 18, 2009

Staff: Mentor

Make yourself a diagram. Draw an arrow from where the runner starts to where he finishes. That's his displacement vector. Use a bit of trig/geometry to find its magnitude.

3. Jan 18, 2009

Kitty808

I tried that and only ended up confusing myself more, I can usually figure out displacement vectors for something thats one direction, then another, but this whole circle thing is throwing me off

4. Jan 18, 2009

Staff: Mentor

Show the diagram that you drew.

Or tell me the coordinates of the runner's starting point and ending point. (I don't know where he starts or whether he runs clockwise or counterclockwise, so I can only guess.)

For example, let's say the radius of the circle was R (with the center at the origin) and that he starts at point (R, 0) and runs 1/4 lap and ends up at point (0, R). What's the distance between those two points?


text:
Calculate strain induced in a rubber(Elastosil 607) against applied force

by phani.kp
Tags: rubber, strain
 P: 3 Hello Team, I have a rubber (harness A of 54 and elongation at break of 100%) on which 30N of force is applied over 45mm2 of ares. I would like to know the strain induced in the rubber against the force. The problem here is i could not find the exact Young's modulus value for this material. But i have tensile strength value, it is 3 N/mm2. Could anybody please let me know the approach to calculate the strain. And also i would like to know why Young's modulus is not an important property for rubbers.
 P: 100 Young's Modulus is a constant that is used with linear infinitesimal elasticity. As you know, metals behave elastically under very small strains (infinitesimal ~ i.e. fraction of a percent strain) and also happen to behave linearly. Thus, Young's Modulus is a constant that is easy to find for metals. Rubbers behave elastically out to very large ("finite") strains. Thus, linear infinitesimal elasticity won't apply, since your strains are likely too large. In addition, the stress vs. strain curve for rubber is nonlinear. For example, in tension, it usually has a "decreasing tangent stiffness" and it compression is will have an "increasing tangent stiffness" for the range of strains that you are probably interested in. You can probably find some info on the "initial shear modulus," but that is a constant that is meant to be used with certain "hyperelastic" stress-strain relationships. Rubbers that are loaded very slowly can be idealized as "hyperelastic," where their stress strain curve is "finite," nonlinear, and the material loads and unloads along the same path (i.e. it conforms to the common definition of elasticity, albeit in a nonlinear fashion). Having said all that, I'm sorry to say that there is no universal method for obtaining the strain that you seek. Whereas aluminum, steel, etc. obey Hooke's Law ($\sigma=E\cdot \epsilon$), silicone, adiprene, etc. have completely different constitutive relationships and there isn't one universal hyperelastic relationship that is analogous to Hooke's Law. On top of that, you have to consider that under realistic rates of loading (even loading rates that may SEEM slow), "viscoelastic" effects will significantly influence your material behavior. Rubber behavior is usually understood to be a combination of hyperelasticity and viscoelasticity. Rubber is complicated! Does that help?
P: 3
Thank you very much for your explanation.
A few properties of this rubber material are as follows.

Hardness Shore A - 54
Tensile strength - 3N/mm^2
Elongation at break - 100%
Volume resistivity - 10^14 Ω cm
(For more details please see the attached data sheet)

In my application this rubber material should undergo 0.5 mm of elastic deformation against a load of 30N over a area of 45mm^2.
Having a look at the above mentioned properties could you please tell me whether this material will undergo 0.5 mm of elastic deformation or not?
Or else Please see if you can tell me the approximate or closer value of young's modulus for my calculations purpose.
Attached Files
 Elastosil_datasheet.pdf (76.1 KB, 3 views)

P: 100

Calculate strain induced in a rubber(Elastosil 607) against applied force

 Quote by phani.kp Thank you very much for your explanation. A few properties of this rubber material are as follows. Hardness Shore A - 54 Tensile strength - 3N/mm^2 Elongation at break - 100% Volume resistivity - 10^14 Ω cm (For more details please see the attached data sheet) In my application this rubber material should undergo 0.5 mm of elastic deformation against a load of 30N over a area of 45mm^2. Having a look at the above mentioned properties could you please tell me whether this material will undergo 0.5 mm of elastic deformation or not? Or else Please see if you can tell me the approximate or closer value of young's modulus for my calculations purpose.
a) You didn't provide a length. To undergo .5mm of deformation will require less force for a longer specimen, compared to a shorter one...
b) You didn't specify if your force is a compression or tension. Recall that rubber behaves very differently in compression vs. tension.
c) You didn't specify the loading rate. Recall that rubber is viscoelastic (its stress vs strain behavior is dependent on strain rate).

Even if you gave me "a," "b," and "c," I wouldn't be able to give you a confident answer because:

"... whereas aluminum, steel, etc. obey Hooke's Law ... silicone, polyurethane, etc. have completely different constitutive relationships and there isn't one universal ... relationship that is analogous to Hooke's Law"

If you still want a simple solution, assuming your ".5mm of elastic deformation" is elongation (not shortening), and you are interested in very slow loading rates only:
1) Draw a straight line from (0,0), to the point ("tensile strength," "elongation at break").
2) Determine the slope
3) There you have it: you may call this the Young's Modulus of Elatosil 607 if you like.

But you didn't need to ask PF to figure that out, right?

note: Under tension, the actual stress-strain curve might lie well above your linear line and have a decreasing slope, for example. What does that curve look like, exactly? I don't know, because:

"[for rubber] there isn't one universal ... relationship that is analogous to Hooke's Law"
 P: 3 Now i understood the reason,Thank you for your explanation.

 Related Discussions Introductory Physics Homework 3 Introductory Physics Homework 5 Introductory Physics Homework 2 Introductory Physics Homework 2


text:
# What could you learn from studying player movement and behaviour on online FPS games?

Just thinking of Counter Strike as a canonical example, if you have a coordinate representation of the map and for every player you have a complete history of everything you might need for every moment of time, say every 10 miliseconds or something and you just recorded where the players went around the map, and what they did. So maybe, for example, at each time t you had v(t) a vector function whose components are (location(t), velocity(t), rotationVector(t), rotationVelocity(t), isFiring(t), ammoCount(t), health(t), isEnemyVisible(t), isTeamMateVisible(t)) or whatever other measurements you think would be useful for the state space.

If you collected a bunch of data like this what sort of hypotheses or models would you consider testing? Would it be possible to construct a probabilistic model to classify histories and possibly predict outcomes of player-enemy encounters? Could you train AI bots on this data?

• The question seems a bit too vague to be answerable. I can think of hundreds of things you might wish to investigate. – edA-qa mort-ora-y Aug 25 '12 at 7:03
• What is your goal? Do you want to write a bot? – Raphael Sep 24 '12 at 5:47
• I asked "what could you learn" with the intent to incite some speculation. If there are hundreds of things to investigate, I'd really like to know of just 2 or 3 of them. I believe the last 3 sentences are perfectly clear well formed questions with enough substance to ruminate on. – taylor Sep 25 '12 at 22:24

File each given vector function with the win/loss of the game and the team each vector was on. Applications of this data: To gauge map balance and provide the least advantage to any one particular team.

Determine how particular elements of the map effect balance.

Make bots more effective by determining common strategies employed against bots, and the manner in which players address situations (going with the counter strike example: We're down players lets group up on the objective).

It would be interesting to see which would provide better balance or more effective bots, conventional testing or this in-depth vector analysis.

Probably if you apply machine learning methods to the basic movements/fire actions of a specific player you can develop an AI bot that plays well against that player.

But adding more high level actions like targets, waypoints, coordinated team actions, terrain features, and so on, you'll probably end up with models that has already been developed for REAL war scenarios :-(

See for example Google results for "combat models soldier behaviour" or "combat XXI simulation model"


text:
# Can LDA assign more than one topic for a word?

I have just started reading about Latent Dirichlet Allocation LDA and want to apply it to my project.

May I know if LDA is able to assign a topic to more than one word?

For example, Article A talks about "river banks" while Article B talks about "The role of banks in finance". Hence, will LDA allow the word "banks" to potentially be assigned to two different topics?

The result of any LDA inference algorithm is $\theta_{d,t}$ and $\phi_{t,w}$, distribution of topics in each document and distribution of terms in each topics. Given these distributions, one can obtain estimate for $p(z|d,w)$, conditional probability of a topic $z$ for word $w$ in document $d$: $$p(z|d,w)=\frac{p(z,d,w)}{\sum\limits_{s=1}^K p(s,d,w)}=\frac{\theta_{d,z} \phi_{z,w}}{\sum\limits_{s=1}^{K} \theta_{d, s} \phi_{s, w}}$$ Further utilizing of this information can be, for example, assigning the single most probable topic for a word. But it's not obligatory.


text:
# Tag Info

## Hot answers tagged open-source

22

OpenMOLCAS MOLCAS is now in it's 5th decade! The first version was written in the 1980s by the group of Bjorn Roos, one of history's giants in quantum chemistry. It can do many of the things most mainstream quantum chemistry packages can do: integrals, Hartree-Fock, DFT, MP2, coupled cluster, geometry optimization, etc., and while it can do "single-...

22

ORCA Official website: https://orcaforum.kofo.mpg.de/app.php/portal What is ORCA? ORCA is a general-purpose quantum chemistry program package that features virtually all modern electronic structure methods (density functional theory, many-body perturbation, and coupled-cluster theories, and multireference and semiempirical methods). It is a flexible, ...

16

Psi4 Psi4 is a fully open-source quantum chemistry package that is gaining a lot of interest in my community (computational) combustion chemistry. Here is the manual for the latest release: it has most, if not all, of the theoretical methods (HF, DFT, CC,...) and also lots of interfaces and integrates into C++ and Python (sadly no Fortran). The bonus on it ...

15

GAMESS For many researchers, GAMESS is the "natural" substitute for GAUSSIAN. Developed and maintained by the group of prof. Gordon (also a banned GAUSSIAN user, see image below). Official website: https://www.msg.chem.iastate.edu/gamess/index.html What is GAMESS: GAMESS is a program for ab initio molecular quantum chemistry. Briefly, GAMESS can ...

10

The PDB file format has a strict specification on the number of columns, etc. (Consider that some proteins have very large sizes, so distances can be many Ångstroms.) Thus, the number of decimal places are generally given by the file format. Some people have asked why Open Babel doesn't give more decimal places, e.g., in the XYZ format, which is, in ...

9

NetworkX This is a widely-used Python library for graph theory stuff. Open source (https://github.com/networkx/networkx) under a 3-clause BSD license. Caveat: since this is a tool that knows nothing about chemistry, the "chemical" nature of the structure might get lost in the projection of the network (e.g., some bond lengths might not look ...

9

Most of the problem here appears to be because constraints in the test system default to app.HBonds, which means all bonds to hydrogen atoms are constrained. Something goes very weird when you force a large, unphysical change on the system, and then try to apply HBond constraints (it looks to me like it isn't even constraining HBonds to the unphysical ...

9

"GAMESS seems to have an old academic user base and a stable code for most traditional calculations" GAMESS has been in development since the 1970s and split into GAMESS(US) and GAMESS(UK) in 1981, whereas MOLCAS has been in development since the 1980s and Psi4 is much newer. However, I would not focus too much on the age of the software. The ...

9

I was able to find a workaround to get open source PyMol installed on Windows. Christoph Gohlke at the University of California has a large library of precompiled Python wheel files of various scientific packages, including PyMol. I was able to find a Windows version for Python 3.8 pymol‑2.5.0a0‑cp38‑cp38‑win_amd64.whl and after downloading this file, I was ...

9

If you're using Linux its better if you install such packages using the apt-get method. apt-get install pymol I've encountered a similar error while installing MatPlotLib through conda. For more info you can follow this link: https://pymolwiki.org/index.php/Linux_Install Hope this helps :)

8

OpenSUSE It is oriented both to desktop and to servers. It has a large collection of software repositories for scientific computing in general (you can have all of GCC 7, 9, 10, 11 at the same time, module system for different versions of MPI libraries, CAD, CFD,...) and even for matter modelling in particular (GROMACS, LAMMPS, Quantum ESPRESSO, etc.). It ...

7

NWChem Official website: https://nwchemgit.github.io/ and https://www.nwchem-sw.org/ It is open source Description: NWChem aims to provide its users with computational chemistry tools that are scalable both in their ability to treat large scientific computational chemistry problems efficiently, and in their use of available parallel computing resources ...

7

Disclaimer: These are just some observations; I don't claim to know the answer to this question. There's a lot of overlap between both camps, but I think people tend to stick mostly with one kind of system, in their research. You may be largely correct - I'd just like to point out that if we are looking at the simulation engines (rather than what people do)...

7

Let me add 2 notes MKL has a secret env-var to force the use of AVX2 extensions etc. For details, see here The magic is setting the environment variable: MKL_DEBUG_CPU_TYPE=5 note however that Intel can at any release remove this flag, so one is not guaranteed this will work in later releases. Another great library is blis it basically works like ...

6

GUI4dft - A SIESTA oriented GUI GUI4dft is a new Graphical User Interface for Density Functional Theory for users of SIESTA. GUI4dft is written in Python language, falls under an MIT license and is a free cross-platform software. GUI4dft allows the user to work with standard SIESTA files and prepare manuscript-quality figures of the atomic structure and ...

5

FCBand FCBand is a Python package for simulating vibronic bands of electronic absorption spectroscopy using the Franck-Condon displaced harmonic oscillator (FC-DHO) models. These models are suitable for simulating the unresolved vibronic band shapes of electronic spectra that involve many electronic transitions and vibrational modes. https://comp.chem.umn....

4

LEVEL For a diatomic molecule, LEVEL can calculate all Franck-Condon factors (FKF): $$|\langle \psi_{v,J} | \psi_{v^\prime,J^\prime}\rangle|^2\tag{1},$$ between rovibrational levels $(v,J)$ of one potential energy curve, and $(v^\prime,J^\prime)$ of another potential energy curve (or the same potential energy curve). To calculate FKF, the variable LXPCT ...

4

AMD's answer to Intel's MKL is called AMD Optimizing CPU Libraries (AOCL), which is coincidentally based on BLIS. In addition, it also includes FLAME (for LAPACK functionality), FFTW, and others.

4

This question has been well-answered. But I would like to add in a few points too. In addition to observing the energy, you can also observe the convergence of outputs which are really sensitive to the K_point grid like the crystal pressure which is given by "P = " above your stress values in your SCF output file. The convergence of such ...

3

You can enter an input file stream in gaussian cubman to add/subtract cube files. Eg. here is an input file named test.txt which has all answers to add two cube files (a.cube and b.cube) and gives an output sum.cube, Add a.cube yes b.cube yes sum.cube yes You can run this simply by entering, cubman < test.txt to get the required output. You can write a ...

2

I tried to answer the question in the github issue github.com/pyxem/diffsims/issues/176 Answer: The reason it doesn't work is that the calibration parameter of the diffraction pattern is not set. A diffraction pattern in diffsims is represented by a list of (x, y) coordinates in projected reciprocal space, and an associated intensities I. Together they ...

Only top voted, non community-wiki answers of a minimum length are eligible


text:
Calculation of Weights in Locally Weighed Regression

In Andrew Ng's lectures he introduces Locally Weighted Linear Regression (page 15) for which he describes the weight as:

\begin{align} w^{(i)} = exp\Big(-\frac{(x^{(i)} - x)^2}{2 \tau^2}\Big) \end{align}

However in the problem assignment for Locally Weighted Logistic Regression, the weight is described as:

\begin{align} w^{(i)} = exp\Big(-\frac{||x^{(i)} - x||^2}{2 \tau^2}\Big) \end{align}

where $x^{(i)}$ had dimensions $1 \times n$ and $x$ has dimensions $m \times n$.

Looking at the solution set, it seems that they first get the difference between $x$ and $x^{(i)}$ (giving an $m \times n$ matrix) followed by element-by-element squaring (using the Matlab/Octave .^2 operator), and then sum along the rows to give a $m \times 1$ vector. In the linear algebra review notes (pages 7-8), multiple types of norms are described, but the sequence of operations performed doesn't seem to be consistent with any of the descriptions of norms. What is going on here?

First off, because I am not sure that this wasn't a part of your question, the norm they are using is defined as $|| \mathbf{x} || = \sqrt{\sum_{i=1}^n x_i^2}$, so if we take the norm of $\mathbf{x} - \mu$ (where $\mu$ is our mean or center) and square it, we get $|| \mathbf{x} - \mu ||^2 = \sum_{i=1}^n (x_i - \mu)^2$. The two expressions are equivalent.
It sounds like they are assigning a different center, $\mu_{i,j}$ for each $x_i$ and each hidden node $h_j$ for $j = 1, 2, \ldots, m$. In that case, for each input value $x_i$ and hidden node $h_j$, you would have:
$$w_{i, j} = \exp \bigg(\frac{(x_i - \mu_{i,j})^2}{2\tau_j^2} \bigg)$$
The input to node $h_j$ is then the sum of these values over all inputs $x_i$ for that node: $\sum_{i = 1}^n w_{i, j}$. I think what they are doing in the code is just the vectorized form of this.


text:
# Stationary policy

In a dynamic program, this is a policy that is independent of time – i.e., $LaTeX: \textstyle x^*(s,t) = T(s)$ (some function of state, but not of time, t).


text:
# Creating Direct Limits and Inverse Limits using Tikz

I am trying to make something that looks like

I am trying to go from here (works):

\begin{tikzcd}
A \arrow{r}\arrow{rd}
&B \arrow{d}\arrow[leftarrow]{r}
&C \arrow{ld}\\
&D
\end{tikzcd}

to here (doesn't work):

\begin{tikzcd}
A \arrow{r}\arrow{rd} \arrow{rdd}
&B \arrow{d}\arrow[leftarrow]{r}
&C \arrow{ld} \arrow{ldd} \\
&D
&E\\
\end{tikzcd}

I try many combinations but don't see how the tikz@x-y-z values work. Any help is appreciated.

Thanks,

Brian

You have to use three columns.

\documentclass{article}
\usepackage{tikz-cd}

\begin{document}

$\begin{tikzcd}[row sep=4em] X_{i} \arrow[rr,"f_{ij}"] \arrow[dr,"\phi_{i}"] \arrow[ddr,swap,end anchor={[xshift=0.2em]north west},"\psi_{i}"] && X_{j} \arrow[dl,swap,"\phi_{j}"] \arrow[ddl,end anchor={[xshift=-0.2em]north east},"\psi_{j}"] \\ & X \arrow[d,"u"] \\ & Y \end{tikzcd}$

\end{document}

The end anchor are just to avoid the arrow ends to be too near to each other.

• For inverse limit, just dualize, of course. ;-) – egreg Jul 25 '14 at 20:49


text:
# Why is it called an 'integral' curve?

The concept of a integral curve is relatively easy to understand as path through a vector field which is tangent to the field at each point.

But why is it called an "integral" curve? It appears to have little to do with integers, or integration.

• @apkg The standard example of a non-integrable equation is this: Can you find smooth surfaces in $\Bbb R^3$ on which we have $dz-x\,dy = 0$ at each point? (Curves are never a problem, at least locally.) Jan 18 at 3:20