text:
# almost convex

A real function, $f\colon\mathbb{R}\to\mathbb{R}$, is said to be almost convex if

 $f(z)\leq\max\{f(x),\,f(y)\},\,\,\,\forall x,y,z\,\,\,x\leq z\leq y$

It is easy to see that a convex function is an almost convex function.

Title almost convex AlmostConvex 2013-03-22 15:20:55 2013-03-22 15:20:55 georgiosl (7242) georgiosl (7242) 5 georgiosl (7242) Definition msc 26A51


text:
# Debugging

30.2 : Finding errors: where, frame, print
30.2.1 : C programs
30.2.2 : Fortran programs
30.3 : Stepping through a program
30.4 : Inspecting values
30.5 : Breakpoints
30.6 : Memory debugging
30.6.1 : Type of memory errors
30.6.1.1 : Invalid pointers
30.6.1.2 : Out-of-bounds errors
30.6.1.3 : Memory leaks
30.6.2 : Memory tools
30.6.2.1 : Valgrind
30.6.2.2 : Electric fence
30.7 : Parallel debugging

# 30 Debugging

\noindent Debugging is like being the detective in a crime movie where you are also the murderer. (Filipe Fortes, 2013)

When a program misbehaves, debugging is the process of finding out why . There are various strategies of finding errors in a program. The crudest one is debugging by print statements. If you have a notion of where in your code the error arises, you can edit your code to insert print statements, recompile, rerun, and see if the output gives you any suggestions. There are several problems with this:

• requiring you to edit, compile, and rerun repeatedly. Furthermore,
• display and inspect effectively, and
• from all processors, making the inspection process very tedious.

For these reasons, the best way to debug is by the use of an interactive \indexterm{debugger}, a program that allows you to monitor and control the behavior of a running program. In this section you will familiarize yourself with gdb lldb, the open source debuggers of the \indexterm{GNU} and \indexterm{clang} projects respectively. Other debuggers are proprietary, and typically come with a compiler suite. Another distinction is that gdb is a commandline debugger; there are graphical debuggers such as \indexterm{ddd} (a~frontend to gdb) or \indexterm{DDT} and TotalView (debuggers for parallel codes). We limit ourselves to gdb, since it incorporates the basic concepts common to all debuggers.

In this tutorial you will debug a number of simple programs with gdb and valgrind. The files can be found in the repository in the directory code/gdb .

TABLE: List of common gdb / lldb commands.

 $\begin{array}{cccccc} \toprule gdb lldb \midrule Starting a debugger run \midrule \ gdb program \ lldb program (gdb) run (lldb) run \midrule Displaying a stack trace \midrule (gdb) where (lldb) thread backtrace \midrule Investigate a specific frame \midrule frame 2 frame select 2 \midrule Run/step \midrule run / step / continue thread continue / step-in/over/out \midrule Set a breakpoint at a line \midrule break foo.c:12 breakpoint set [ -f foo.c ] -l 12 break foo.c:12 if n>0 info breakpoints \midrule Set a breakpoint for exceptions \midrule catch throw break set -E C++ \bottomrule \end{array}$

## 30.1 Invoking the debugger

crumb trail: > debug > Invoking the debugger

There are three ways of using gdb: using it to start a program, attaching it to an already running program, or using it to inspect a core dump. We will only consider the first possibility.

 $\begin{array}{cccccc} \toprule Starting a debugger run \midrule gdb lldb \midrule \ gdb program \ lldb program (gdb) run (lldb) run \bottomrule \end{array}$

Here is an example of how to start gdb with program that has no arguments (Fortran users, use hello.F ): \codelisting{tutorials/gdb/c/hello.c}

%% cc -g -o hello hello.c

# regular invocation:

%% ./hello

hello world

# invocation from gdb:

%% gdb hello

GNU gdb 6.3.50-20050815 # ..... [version info]

(gdb) run

Starting program: /home/eijkhout/tutorials/gdb/hello

Reading symbols for shared libraries +. done

hello world

Program exited normally.

(gdb) quit

%%

Important note: the program was compiled with the \indexterm{debug flag}  -g . This causes the \indexterm{symbol table} (that is, the translation from machine address to program variables) and other debug information to be included in the binary. This will make your binary larger than strictly necessary, but it will also make it slower, for instance because the compiler will not perform certain optimizations\footnote{Compiler optimizations are not supposed to change the semantics of a program, but sometimes do. This can lead to the nightmare scenario where a program crashes or gives incorrect results, but magically works correctly with compiled with debug and run in a debugger.}.

To illustrate the presence of the symbol table do

%% cc -g -o hello hello.c

%% gdb hello

GNU gdb 6.3.50-20050815 # ..... version info

(gdb) list

and compare it with leaving out the -g flag:

%% cc -o hello hello.c

%% gdb hello

GNU gdb 6.3.50-20050815 # ..... version info

(gdb) list

For a program with commandline input we give the arguments to the

run command (Fortran users use say.F ):

\codelisting{tutorials/gdb/c/say.c} \vfill

%% cc -o say -g say.c

%% ./say 2

hello world

hello world

%% gdb say

.... the usual messages ...

(gdb) run 2

Starting program: /home/eijkhout/tutorials/gdb/c/say 2

Reading symbols for shared libraries +. done

hello world

hello world

Program exited normally.

## 30.2 Finding errors: where, frame, print

crumb trail: > debug > Finding errors: where, frame, print

Let us now consider some programs with errors.

### 30.2.1 C programs

crumb trail: > debug > Finding errors: where, frame, print > C programs

The following code has several errors. We will use the debugger to uncover them.

// square.c

int nmax,i;

float *squares,sum;

fscanf(stdin,"%d",nmax);

for (i=1; i<=nmax; i++) {

squares[i] = 1./(i*i); sum += squares[i];

}

printf("Sum: %e\n",sum);

%% cc -g -o square square.c

%% ./square

5000

Segmentation fault

The \indexterm{segmentation fault} (other messages are possible too) indicates that we are accessing memory that we are not allowed to, making the program exit. A debugger will quickly tell us where this happens:

%% gdb square

(gdb) run

50000

0x00007fff824295ca in __svfscanf_l ()

Apparently the error occurred in a function __svfscanf_l , which is not one of ours, but a system function. Using the backtrace

(or  bt , also where or  w ) command we display the \indexterm{call stack}. This usually allows us to find out where the error lies:

 $\begin{array}{cccccc} \toprule Displaying a stack trace \midrule gdb lldb \midrule (gdb) where (lldb) thread backtrace \bottomrule \end{array}$

{\small

(gdb) where

#0  0x00007fff824295ca in __svfscanf_l ()

#1  0x00007fff8244011b in fscanf ()

#2  0x0000000100000e89 in main (argc=1, argv=0x7fff5fbfc7c0) at square.c:7

}

We inspect the actual problem:

 $\begin{array}{cccccc} \toprule Investigate a specific frame \midrule gdb clang frame 2 frame select 2 \bottomrule \end{array}$

We take a close look at line 7, and see that we need to change nmax to  &nmax .

There is still an error in our program: {\small

(gdb) run

50000

0x0000000100000ebe in main (argc=2, argv=0x7fff5fbfc7a8) at square1.c:9

9           squares[i] = 1./(i*i); sum += squares[i];

} We investigate further:

(gdb) print i

$1 = 11237 (gdb) print squares[i] Cannot access memory at address 0x10000f000 (gdb) print squares$2 = (float *) 0x0

and we quickly see that we forgot to allocate squares .

Memory errors can also occur if we have a legitimate array, but we access it outside its bounds. The following program fills an array, forward, and reads it out, backward. However, there is an indexing error in the second loop.

// up.c

int nlocal = 100,i;

double s, *array = (double*) malloc(nlocal*sizeof(double));

for (i=0; i<nlocal; i++) {

double di = (double)i;

array[i] = 1/(di*di);

}

s = 0.;

for (i=nlocal-1; i>=0; i++) {

double di = (double)i;

s += array[i];

}

Program received signal EXC_BAD_ACCESS, Could not access memory.

0x0000000100000f43 in main (argc=1, argv=0x7fff5fbfe2c0) at up.c:15

15          s += array[i];

(gdb) print array

$1 = (double *) 0x100104d00 (gdb) print i$2 = 128608

You see that the index where the debugger finally complains is quite a bit larger than the size of the array.

Exercise Can you think of a reason why indexing out of bounds is not immediately fatal? What would determine where it does become a problem? (Hint: how is computer memory structured?)

In section \ref{sec:valgrind} you will see a tool that spots any out-of-bound indexing.

### 30.2.2 Fortran programs

crumb trail: > debug > Finding errors: where, frame, print > Fortran programs

Compile and run the following program: \codelisting{tutorials/gdb/f/square.F} It should end prematurely with a message such as Illegal instruction'. Running the program in gdb quickly tells you where the problem lies:

(gdb) run

Starting program: tutorials/gdb//fsquare

Reading symbols for shared libraries ++++. done

Illegal instruction/operand.

0x0000000100000da3 in square () at square.F:7

7                sum = sum + squares(i)

We take a close look at the code and see that we did not allocate

squares properly.

## 30.3 Stepping through a program

crumb trail: > debug > Stepping through a program

 $\begin{array}{cccccc} \toprule Stepping through a program \midrule gdb lldb meaning run start a run cont continue from breakpoint next next statement on same level step next statement, this level or next \bottomrule \end{array}$

Often the error in a program is sufficiently obscure that you need to investigate the program run in detail. Compile the following program \codelisting{tutorials/gdb/c/roots.c} and run it:

%% ./roots

sum: nan

Start it in gdb as before:

%% gdb roots

GNU gdb 6.3.50-20050815

Copyright 2004 Free Software Foundation, Inc.

....

but before you run the program, you set a \indexterm{breakpoint} at main . This tells the execution to stop, or break', in the main program.

(gdb) break main

Breakpoint 1 at 0x100000ea6: file root.c, line 14.

Now the program will stop at the first executable statement in main :

(gdb) run

Starting program: tutorials/gdb/c/roots

Reading symbols for shared libraries +. done

Breakpoint 1, main () at roots.c:14

14        float x=0;

Most of the time you will set a breakpoint at a specific line:

 $\begin{array}{cccccc} \toprule Set a breakpoint at a line \midrule gdb lldb break foo.c:12 breakpoint set [ -f foo.c ] -l 12 \bottomrule \end{array}$

If execution is stopped at a breakpoint, you can do various things, such as issuing the step command:

Breakpoint 1, main () at roots.c:14

14        float x=0;

(gdb) step

15        for (i=100; i>-100; i--)

(gdb)

16          x += root(i);

(gdb)

(if you just hit return, the previously issued command is repeated). Do a number of step s in a row by hitting return. What do you notice about the function and the loop?

Switch from doing step to doing next . Now what do you notice about the loop and the function?

Set another breakpoint: break 17 and do cont . What happens?

Rerun the program after you set a breakpoint on the line with the

sqrt call. When the execution stops there do where and

list .

## 30.4 Inspecting values

crumb trail: > debug > Inspecting values

Run the previous program again in gdb: set a breakpoint at the line that does the sqrt call before you actually call run . When the program gets to line 8 you can do print n . Do cont . Where does the program stop?

If you want to repair a variable, you can do set var=value . Change the variable n and confirm that the square root of the new value is computed. Which commands do you do?

## 30.5 Breakpoints

crumb trail: > debug > Breakpoints

If a problem occurs in a loop, it can be tedious keep typing cont

and inspecting the variable with print . Instead you can add a condition to an existing breakpoint. First of all, you can make the breakpoint subject to a condition: with

condition 1 if (n<0)

breakpoint 1 will only obeyed if n<0 is true.

You can also have a breakpoint that is only activated by some condition. The statement

break 8 if (n<0)

means that breakpoint 8 becomes (unconditionally) active after the condition n<0 is encountered.

 $\begin{array}{cccccc} \toprule Set a breakpoint \midrule gdb lldb \midrule break foo.c:12 breakpoint set [ -f foo.c ] -l 12 break foo.c:12 if n gt;0 \bottomrule \end{array}$

Remark You can break on NaN with the following trick:

break foo.c:12 if x!=x

using the fact that NaN is the only number not equal to itself.

Another possibility is to use ignore 1 50 , which will not stop at breakpoint 1 the next 50 times.

Remove the existing breakpoint, redefine it with the condition n<0

and rerun your program. When the program breaks, find for what value of the loop variable it happened. What is the sequence of commands you use?

You can set a breakpoint in various ways:

• info breakpoints .

• number of the breakpoint.
• the breakpoints stay in effect.
• them: save breakpoints <file> . Use source <file> to read them in on the next gdb run.
• as  C++ \index{C++!exception}, you can set a \indexterm{catchpoint}:

 $\begin{array}{cccccc} \toprule Set a breakpoint for exceptions \midrule gdb clang\ catch throw break set -E C++ \bottomrule \end{array}$

Finally, you can execute commands at a breakpoint:

break 45

command

print x

cont

end

This states that at line 45 variable  x is to be printed, and execution should immediately continue.

If you want to run repeated gdb sessions on the same program, you may want to save an reload breakpoints. This can be done with

save-breakpoint filename

source filename

## 30.6 Memory debugging

crumb trail: > debug > Memory debugging

Many problems in programming stem from memory errors. We start with a sort description of the most common types, and then discuss tools that help you detect them.

### 30.6.1 Type of memory errors

crumb trail: > debug > Memory debugging > Type of memory errors

#### 30.6.1.1 Invalid pointers

crumb trail: > debug > Memory debugging > Type of memory errors > Invalid pointers

Dereferencing a pointer that does not point to an allocated object can lead to an error. If your pointer points into valid memory anyway, your computation will continue but with incorrect results.

However, it is more likely that your program will probably exit with a \indexterm{segmentation violation} or a bus error.

#### 30.6.1.2 Out-of-bounds errors

crumb trail: > debug > Memory debugging > Type of memory errors > Out-of-bounds errors

Addressing outside the bounds of an allocated object is less likely to crash your program and more likely to give incorrect results.

Exceeding bounds by a large enough amount will again give a segmentation violation, but going out of bounds by a small amount may read invalid data, or corrupt data of other variables, giving incorrect results that may go undetected for a long time.

#### 30.6.1.3 Memory leaks

crumb trail: > debug > Memory debugging > Type of memory errors > Memory leaks

We speak of a \indexterm{memory leak} if allocated memory becomes unreachable. Example:

if (something) {

double *x = malloc(10*sizeofdouble);

// do something with x

}

After the conditional, the allocated memory is not freed, but the pointer that pointed to has gone away.

This last type especially can be hard to find. Memory leaks will only surface in that your program runs out of memory. That in turn is detectable because your allocation will fail. It is a good idea to always check the return result of your \indextermtt{malloc} or \indextermtt{allocate} statement!

### 30.6.2 Memory tools

crumb trail: > debug > Memory debugging > Memory tools

#### 30.6.2.1 Valgrind

crumb trail: > debug > Memory debugging > Memory tools > Valgrind

(label: sec:valgrind)

Insert the following allocation of squares in your program:

squares = (float *) malloc( nmax*sizeof(float) );

Compile and run your program. The output will likely be correct, although the program is not. Can you see the problem?

To find such subtle memory errors you need a different tool: a memory debugging tool. A popular (because open source) one is valgrind

\codelisting{tutorials/gdb/c/square1.c} Compile this program with cc -o square1 square1.c and run it with

valgrind square1 (you need to type the input value). You will lots of output, starting with: {\small

%% valgrind square1

==53695== Memcheck, a memory error detector

==53695== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.

==53695== Using Valgrind-3.6.1 and LibVEX; rerun with -h for copyright info

==53695== Command: a.out

==53695==

10

==53695== Invalid write of size 4

==53695==    at 0x100000EB0: main (square1.c:10)

==53695==  Address 0x10027e148 is 0 bytes after a block of size 40 alloc'd

==53695==    at 0x1000101EF: malloc (vg_replace_malloc.c:236)

==53695==    by 0x100000E77: main (square1.c:8)

==53695==

==53695== Invalid read of size 4

==53695==    at 0x100000EC1: main (square1.c:11)

==53695==  Address 0x10027e148 is 0 bytes after a block of size 40 alloc'd

==53695==    at 0x1000101EF: malloc (vg_replace_malloc.c:236)

==53695==    by 0x100000E77: main (square1.c:8)

} Valgrind is informative but cryptic, since it works on the bare memory, not on variables. Thus, these error messages take some exegesis. They state that a line 10 writes a 4-byte object immediately after a block of 40 bytes that was allocated. In other words: the code is writing outside the bounds of an allocated array. Do you see what the problem in the code is?

Note that valgrind also reports at the end of the program run how much memory is still in use, meaning not properly free d.

If you fix the array bounds and recompile and rerun the program, valgrind still complains: {\small

==53785== Conditional jump or move depends on uninitialised value(s)

==53785==    at 0x10006FC68: __dtoa (in /usr/lib/libSystem.B.dylib)

==53785==    by 0x10003199F: __vfprintf (in /usr/lib/libSystem.B.dylib)

==53785==    by 0x1000738AA: vfprintf_l (in /usr/lib/libSystem.B.dylib)

==53785==    by 0x1000A1006: printf (in /usr/lib/libSystem.B.dylib)

==53785==    by 0x100000EF3: main (in ./square2)

} Although no line number is given, the mention of printf gives an indication where the problem lies. The reference to an uninitialized value' is again cryptic: the only value being output is sum , and that is not uninitialized: it has been added to several times. Do you see why valgrind calls it uninitialized all the same?

#### 30.6.2.2 Electric fence

crumb trail: > debug > Memory debugging > Memory tools > Electric fence

The \indexterm{electric fence} library is one of a number of tools that supplies a new \indextermtt{malloc} with debugging support. These are linked instead of the malloc of the standard libc.

cc -o program program.c -L/location/of/efence -lefence

Suppose your program has an out-of-bounds error. Running with gdb, this error may only become apparent if the bounds are exceeded by a large amount. On the other hand, if the code is linked with libefence, the debugger will stop at the very first time the bounds are exceeded.

## 30.7 Parallel debugging

crumb trail: > debug > Parallel debugging

FIGURE 30.1: Display of 16 processes in the DDT debugger.

(label: fig:ddt2)

Debugging in parallel is harder than sequentially, because you will run errors that are only due to interaction of processes such as \indexterm{deadlock}; see section \ref{sec:nonblocking}.

As an example, consider this segment of MPI code:

MPI_Init(0,0);

// set comm, ntids, mytid

for (int it=0; ; it++) {

double randomnumber = ntids * ( rand() / (double)RAND_MAX );

printf("[%d] iteration %d, random %e\n",mytid,it,randomnumber);

if (randomnumber>mytid && randomnumber<mytid+1./(ntids+1))

MPI_Finalize();

}

MPI_Finalize();

Each process computes random numbers until a certain condition is satisfied, then exits. However, consider introducing a barrier (or something that acts like it, such as a reduction):

for (int it=0; ; it++) {

double randomnumber = ntids * ( rand() / (double)RAND_MAX );

printf("[%d] iteration %d, random %e\n",mytid,it,randomnumber);

if (randomnumber>mytid && randomnumber<mytid+1./(ntids+1))

MPI_Finalize();

MPI_Barrier(comm);

}

MPI_Finalize();

`

Now the execution will hang, and this is not due to any particular process: each process has a code path from init to finalize that does not develop any memory errors or other runtime errors. However as soon as one process reaches the finalize call in the conditional it will stop, and all other processes will be waiting at the barrier.

Figure \ref{fig:ddt2} shows the main display of the Allinea \indextermdef{DDT} debugger ( http://www.allinea.com/products/ddt ) at the point where this code stops. Above the source panel you see that there are 16 processes, and that the status is given for process 1. In the bottom display you see that out of 16 processes 15 are calling MPI_Barrier on line 19, while one is at line 18. In the right display you see a listing of the local variables: the value specific to process 1. A rudimentary graph displays the values over the processors: the value of ntids is constant, that of mytid is linearly increasing, and it

is constant except for one process.

Exercise Make and run ring_1a . The program does not terminate and does not crash. In the debugger you can interrupt the execution, and see that all processes are executing a receive statement. This is probably a case of deadlock. Diagnose and fix the error.

Exercise The author of ring_1c was very confused about how MPI works. Run the program. While it terminates without a problem, the output is wrong. Set a breakpoint at the send and receive statements to figure out what is happening.

format: plain


text:
We're sorry Aspose doesn't work properply without JavaScript enabled.

# Table Rendering problem

Hi,
We have been seeing a consistent issue in regards how tables are being rendered in Aspose Word (PDF) with ODT documents. In fact it seems to be an issue relating to how styles are managed differently between the two in general. Potentially Character vs Paragraph styles.

I have attached a test document and PDF’s rendered by Openoffice and Aspose. You will see the Aspose one formats over a page and this seems to relate to how paragraph spacing or table spacing is done.

We suspect Aspose is ignoring the character style/formatting and falling back to the styesheet or some other format and therefore adding pararaph spacing when it renders the PDF.

I am wondering if you could shed some light on this for us as it seems to be one the major pitfalls we are hitting when migrating our customers between odt pdf generation in Openoffice vs Aspose.
Aspose Table Formatting test.zip (134.1 KB)

Kind Regards
Tony

@tdekeizer
We have opened the following new ticket(s) in our internal issue tracking system and will deliver their fixes according to the terms mentioned in Free Support Policies.

Issue ID(s): WORDSNET-24872

You can obtain Paid Support services if you need support on a priority basis, along with the direct access to our Paid Support management team.

You should note that Aspose.Words mimics MS Word upon importing ODT documents. If render your document to PDF using MS Word it also moves part of the table into the next page:
out.pdf (68.2 KB)
ms.pdf (133.6 KB)

Investigation shows that this is caused by different handling of paragraph line spacing. If reset it to singe, the table is rendered closer to Open Office or Libre Office:

Document doc = new Document("C:\\Temp\\in.odt");

Iterable<Paragraph> paragraphs = doc.getChildNodes(NodeType.PARAGRAPH, true);
for (Paragraph p : paragraphs)
{
if (p.getParagraphFormat().getLineSpacingRule() == LineSpacingRule.MULTIPLE)
{
p.getParagraphFormat().setLineSpacing(12);
}
}

doc.save("C:\\Temp\\out_preprocessed.pdf");

out_preprocessed.pdf (67.6 KB)

Thanks again Alexey,

I am not sure this will work for us as in some cases users have templates that require non single line spacing and this would result in the format being incorrect as well ?

I actually expect that the styles difference between MS Word and ODT (Openoffice) is more pervasive then just line spacing as we are seeing multiple issues relating to how formatting/styles are applied.

Is it possible to provide some more detail of the differences you mention between Word and Openoffice so we can advise customers how to modify their documenst to work as expected and previously ?

Cheers
Tony

@tdekeizer You are right, MS Word and OpenOffice behavior difference is not limited to line spacing. But I am afraid these differences are not documented, so there are no general recommendations how to modify ODT documents in Open Office to make them look as they look in Open Office after rendering the document in MS Word or Aspose.Words. I can only suggest you to open the problematic document in MS Word and see the difference in the document appearance, since Aspose.Words in most cases mimics MS Word behavior when process non-native MS Word formats.

Hi Alexey,
I must admit that is feels wrong that Word ignores the direct formatting set on the text and always uses the style format ? Here is the style on the text:

Here is the default style

The issue with this document is the difference in the below Paragraph spacing (which again makes the document expand to more space then in Open Office), but it can relate to any paragraph formating.

Word supports direct formatting and style formatting. I found this article about the rules that apply to applying formatting in regards paragraphs.
https://shaunakelly.com/word/styles/stylesoverridedirectformatting.html
In the test the direct formatting applies to over half (the whole) paragraph so why is the style overriding the direct formatting ?
Also the above applies to applying the formatting not to a document that already has that formatting.
Hopefully you can clarify this for me as I am confused and still think something is not quite right

Cheers
Tony

@tdekeizer The problem occurs because Open Document and MS Word document formats are quite different. For example let’s consider a paragraph from the original ODT document:

<text:p text:style-name="P35">
<text:span text:style-name="Default_20_Paragraph_20_Font">
<text:span text:style-name="T15">Day:</text:span>
</text:span>
</text:p>

Paragraph has P35 style applied:

<style:style style:name="P35" style:family="paragraph" style:parent-style-name="Normal">
<style:paragraph-properties fo:line-height="150%" fo:text-align="start" style:justify-single-word="false" fo:orphans="0" fo:widows="0" style:text-autospace="none" style:punctuation-wrap="simple"/>
</style:style>

As you can see fo:line-height="150%", line height specified in percepts is converted to LineSpacingRule.Multiple. 150% means line spacing 1.5 lines, what we see in MS Word UI:

Open Office as well as Libre Office also shows this in the UI:

But in Open Office this value is taken in account only when there are more than one line in the paragraph:

While MS Word interprets it differently:

Thanks Alexey for the detailed explanation. very much apprecaited and makes sense.

So the

<text:span text:style-name="Default_20_Paragraph_20_Font">

, which I assume is the direct style on the actual text, is being ignored by MS in this case ?

Can you think of any programatic way to make it behave as per Open Office ?

Cheers
Tony

@tdekeizer Default_20_Paragraph_20_Font definition is in style.xml and look like this:

<style:style style:name="Default_20_Paragraph_20_Font" style:display-name="Default Paragraph Font" style:family="text"/>

So it does not redefine fo:line-height="150%" value specified in P35 style. Also, Default_20_Paragraph_20_Font is applied to text, and line-height (line spacing) is a paragraph’s option, not the text option.
The text formatting is specified using T15 style:

<style:style style:name="T15" style:family="text">
<style:text-properties style:use-window-font-color="true" fo:font-size="9pt" fo:font-weight="bold" style:letter-kerning="true" style:font-name-asian="SimSun" style:font-size-asian="9pt" style:language-asian="zh" style:country-asian="CN" style:font-weight-asian="bold" style:font-name-complex="Arial" style:font-size-complex="9pt" style:language-complex="hi" style:country-complex="IN" style:font-weight-complex="bold"/>
</style:style>

Unfortunately, the only way to workaround the problem is to reset LineSpacing to single, as I suggested earlier.


text:
# Jigsaw Puzzle Orbitals

### Preliminaries

Questaal’s basis functions consist of augmented waves : smooth envelope functions with holes punched out around atoms the smooth envelope replaced (“augmented”) by numerical solutions of the Schrodinger equation. These numerical solutions are very accurate, so basis incompleteness mostly originates from the envelope functions. This page is concerned with a new construction for them, the Jigsaw Puzzle Orbitals.

Similar to Questaal’s traditional basis set, Jigsaw Puzzle orbitals are constructed out of smooth Hankel functions and generalised Gaussian orbitals. The reader is advised to read at least the introductory part of this documentation describing smooth Hankel functions.

### Special properties of Jigsaw Puzzle Orbitals

Jigsaw Puzzle Orbitals (JPOs) have a number of highly desirable properties of a basis set.

• They are short ranged and atom centred, with pure $Y_{lm}$ character on the augmentation boundary where they are centred. Thus, they serve as good projectors for special subspaces, e.g. correlated atoms where the correlation is strong in a particular l channel such as transition metals and f shell metals.
• They are smooth everywhere. This greatly facilitates their practical implementation.
• They have an exponentially decaying asymptotic form far from a nucleus, making them short range.
• They are tailored to the potential. Inside or near an augmentation site, the Schrödinger equation is carried by almost entirely by a single function. Thus they form a nearly optimum basis set to solve the Schrödinger equation over a given energy window.

In the figure on the left, the JPO envelope functions are shown for s and p orbitals in a 1D model with two atom centers.

The solid parts depict the interstitial region, where the envelope functions carry the wave function. Dashed lines depict augmented regions where the envelope is substituted by partial waves — numerical solutions of the Schrödinger equation. It is nevertheless very useful that the envelope functions are smooth everywhere, since sharply peaked envelope functions require many plane waves to represent the smooth charge density $n_0$

At points where the envelope functions and augmented functions join, the function value is unity on the head site ($V_{1p}$ or $V_{1s}$ on the left, and $V_{2p}$ or $V_{2s}$ on the right) and zero on the other site. (By unity we refer to the radial part of the partial wave; the full wave must be multiplied by $Y_{lm}$, which for p orbitals is $\pm 1$ depending on whether the point is right or left of centre.) Moreover, the kinetic energy is tailored to be continuous at the head site and vanish at the other site.

These two facts taken in combination are very important. Consider the Schrödinger equation near an augmentation point. Inside the augmentation region, the partial wave is constructed numerically, and is very accurate. It is not quite exact: the partial wave is linearized, and the potential which constructs the partial wave is taken to be the spherical average of the actual potential. But it is well established that errors are small: the LAPW method, for example, considered to be the gold standard basis set for accuracy, makes the same approximation. Since the kinetic energy is continuous across the boundary, the basis function equally well describes the Schrödinger on the other side of the boundary, at least very near the boundary.

This alone is not sufficient to make the basis set accurate. Tails in some channel from heads centred elsewhere will contribute to the eigenfunction. They can “contaminate” the accurate solution of the head. However, consider the form of the Schrödinger equation:

By construction both value and kinetic energy of all basis functions $V_{Rl}$ vanish except for the single partial wave that forms the head. Thus any linear combination of them will yield a nearly exact solution of the Schrödinger equation locally.

In the 1D model described above, a the JPO basis was applied to double-exponential potential well shown in black in the Figure on the right. The kinetic energy of a traditional smooth Hankel (green) shows a discontinuity at the augmentation boundary; with JPO’s the discontinuity disappears (red). The JPO kinetic energy is everywhere very close to the exact solution (the exact potential and kinetic energies lie on top of each other).

In many respects, JPOs are nearly ideal basis functions: they are close to being as compact as possible for solving the Schrödinger equation in a given energy window. JPO’s have two important drawbacks, however:

• They are complex objects, complicating their augmentation and assembling of matrix elements. We can however make use analytic properties of JPO envelope functions to greatly ameliorate the increase in computational cost in making matrix elements. The extra cost is not important provided it does not dominate the total cost.
• There is no analytic form for products of two of them, as there are for plane waves and Gaussian orbitals. However the same technology we have built for Questaal’s traditional orbitals based on smooth Hankel functions. can be applied here.

### Construction of Jigsaw Puzzle Orbitals*

JPOs are tangentially related to Andersen’s tight-binding transformation and NMTO method.2 This transformation makes conventional LMTO’s short ranged (see Fig. 5 in the Questaal methods paper, Ref. 3). It makes both conventional LMTO’s and smooth Hankel functions short range in the same way, but Questaal’s functions are more accurate, shorter ranged, and suited for modern full-potential methods. NMTOs do establish proof-of-principle: they are very accurate over the relevant energy window (roughly $E_{F}\pm 1\,$Ry).

To date we have succeeded in rotating Questaal’s standard basis functions (convolutions of Hankel and Gaussian functions) into a screened form; see Section 3.12 in the Questaal methods paper, Ref. 3) This basis set spans the same hilbert space as Questaal’s traditional basis — good, but not optimal.

The figure on the left shows screened $d$ envelope functions, $xy$, $yz$, $3z^2{-}1$, $xz$, and $x^2{-}y^2$, for a zincblende lattice. They resemble a traditional Slater-Koster form (having pure $lm$ character), making easy identification with an atomic orbital.

The figure on the right shows the bandgap in NiO calculated in the Quasiparticle Self-Consistent GW (QSGW) approximation, comparing Questaal’s traditional basis and a screened form. The bandgap is plotted as a function of the cutoff $\mathbf{|R'-R|}$ in the QSGW self-energy $\Sigma^{0}_{\mathbf{RR}'}$ for traditional envelope functions (blue) and screened basis set (orange). The first point contains onsite terms only, the second adds first neighbours etc. The conventional basis shows erratic behaviour until at least the fourth neighbours are present. JPO’s converge quickly, because their range is smaller than the range of the physical $\Sigma^{0}({\mathbf{r,r}'})$, so what the evolution of the gap is a reflection of the nonlocality in the $\Sigma^{0}({\mathbf{r,r}'})$.
It is seen to be relatively short-ranged, but not confined to a single site which theories such as LDA+U and LDA+DMFT assume.

The final step, making the kinetic energy everywhere continuous, slightly perturbs the shape and should significantly improve its accuracy.

### References and Other Resources

1. Many mathematical properties of smoothed Hankel functions and the $H_{kL}$ family are described in this paper: E. Bott, M. Methfessel, W. Krabs, and P. C. Schmid, Nonsingular Hankel functions as a new basis for electronic structure calculations, J. Math. Phys. 39, 3393 (1998)

2. O. K. Andersen et al., “Electronic Structure and Physical Properties of Solids: The Uses of the LMTO Method,” in Lecture notes in physics, Vol. 535 (Springer-Verlag, Berlin, 2000) H. Dreysse, ed.

3. Dimitar Pashov, Swagata Acharya, Walter R. L. Lambrecht, Jerome Jackson, Kirill D. Belashchenko, Athanasios Chantis, Francois Jamet, Mark van Schilfgaarde, Questaal: a package of electronic structure methods based on the linear muffin-tin orbital technique, Comp. Phys. Comm. 249, 107065 (2020).

This remark must be qualified. A basis set that is nearly perfect in describing a local potential may still be incomplete in important ways. On particularly important example arises in the context of many-body theory. A cusp appears in the two-particle wave function $\psi(\mathbf{r}_1,\mathbf{r}_2)$ as $\mathbf{r}_1\rightarrow\mathbf{r}_2$, because of the singularity in the coulomb potential $|\mathbf{r}_1-\mathbf{r}_2|^{-1}$. Smooth functions have difficulty approximating this cusp. Treatment of forces and phonons are another context: as the nucleus moves, the basis must shift with it because the partial waves are tailored only to solve the Schrodinger equation when the nuclear potential is not displaced. These are sometimes called “Pulay” corrections.

The QSGW bandgap is slightly larger than 5 eV, as shown in the original paper describing Quasiparticle Self-Consistent GW (Phys. Rev. Lett. 93}, 126406 (2004)), larger than the experimental gap of about 4.3 eV. Including ladder diagrams in W reduces this gap close to the experiment, as will be shown in a paper by Brian Cunningham and co-workers.


text:
1. ## linear equations in 2 variables....help please

Translate into a pair of linear equations in two variables.
solve the equations using elimination or substitution

Mrs. Boyd has a desk full of quarters and nickels. if she has a total of 25 coins with a total face value of 2.85 , how many are nickels?

I have not been able to figure out how to do this problem and have tried.

2. Originally Posted by southrngrl1966
Translate into a pair of linear equations in two variables.
solve the equations using elimination or substitution

Mrs. Boyd has a desk full of quarters and nickels. if she has a total of 25 coins with a total face value of 2.85 , how many are nickels?

I have not been able to figure out how to do this problem and have tried.
1. Let x denote the number of quarters and y the number nickels.

2. Then you know:

$\left|\begin{array}{rcl}x+y&=&25 \\ 0.25 \cdot x + 0.05 \cdot y&=& 2.85\end{array}\right.$

3. Solve for x and y.

3. Let the number quarters be represented by q, and nickels by n.

The total number of coins is q + n, which is equal to 25.

So, you have

q + n = 25

Now, the value of a nickel is 5 and that of a quarter is 25 (I'm not sure for the value of a nickel, correct me if I'm wrong, it's not my currency).

0.25q gives the value of the total quarters, and 0.05n gives the value of the total nickels.

Their value add up to 2.85.

So,

0.25q + 0.05n = 2.85

You now have:

q + n = 25
0.25q + 0.05n = 2.85

Solve them simultaneously

Post what you get!

EDIT: Oops, didn't know you posted before me earboth

4. Thank you for the help. Now I will be ableto finish this. I had the first part but was not sure if it was right.

THanks again. have a great week


text:
Well, my point wasn't that we have a functor, but rather that was this is a more category theoretic way to express what the category \$$\mathbb{ME}\$$ is.

Then, it makes it straight forward to see that we have the functor that you were claiming.

Having said that, \$$\mathbb{ME}\$$ doesn't quite feel like a natural category to me, unless you can give me a place where it crops up naturally.

What feels more natural is the following which I'll call \$$\mathbb{ECM}\$$ (rather than \$$\mathbb{ME}2\$$).

- An object is a pair \$$(\mathcal{V}, \mathcal{X})\$$ where \$$\mathcal{V}\$$ is a monoid and \$$\mathcal{X}\$$ is an enriched category over \$$\mathcal{V}\$$.

- A morphism \$$(\mathcal{V}, \mathcal{X}) \to (\mathcal{U}, \mathcal{Y})\$$ consists of a pair \$$\left(\hat\phi\colon \mathcal{V}\to \mathcal{U}, \,\,\phi \colon \text{Ob}(\mathcal{X})\to \text{Ob}(\mathcal{Y})\right)\$$ where \$$\hat\phi\$$ is a monoid map and we have \$$\hat\phi(\mathcal{X}(a,b)) = \mathcal{Y}(\phi(a),\phi(b))\$$.

Then you still get the functor \$$\mathcal{G}\colon \textbf{Grp}\to \mathbb{ECM}\$$, but I think that I can now construct two natural left inverses.

My point here is that \$$\mathbb{ECM}\$$ feels more like an enriched category theory construction, and easily generalises to enriching over arbitary monoidal categories. However, that's not to say you won't have a naturally occurring mathematical situation in which you want to deal with \$$\mathbb{ME}\$$.


text:
# Action invariance under galilean boost

1. Oct 12, 2012

### Petraa

Hello,

I've been spending a lot of time trying to solve this problem but I can't figure out a good solution.

I have to show that the action of a non-relativistic particle ( Schrodinger density Lagrangian ) is invariant under Galilean boost with the form

ψ(x0,x)→ψ'(x0,x)=eimvx-(im/2)x0v2ψ(x0,x-vx0)

x0= t

I've tried to find the transformed Lagrangian by replacing the wave functions and the derivatives but I'm not sure I did it correctly because I get monstrous expressions

I'm using this density Lagrangian L= ihψ*$\partial$0ψ+h2/2m($\partial$iψ*)($\partial$iψ)

If someone can give me a good tip I'll appreciate it
thank you!

2. Oct 12, 2012

### Bill_K

Ok, hints only, because this is an assignment.

Important fact: Lagrangians are not unique. You can freely add a total divergence to a Lagrangian without changing the equations of motion. And in this example the Lagrangian will not be invariant under the transformation, you will need to add a total divergence.

Let φ(x,t) be the phase factor in front you've added. You'll get three types of terms:

φ*φ x (derivatives on ψ) + ψ*ψ x (derivatives on φ) + (derivatives on both φ and ψ)

The first two types are easily handled. For terms of the third type you'll need to split off a total divergence, e.g. by throwing derivatives of ψ over onto derivatives of φ.

3. Oct 12, 2012

### Petraa

I've tried some, the "best" total divergence I've found is ∂i(ψ*φ*∂iψφ)

Fits all the terms on the equation but ( obviously ) φ*∂0φ and also leaves me a term ∂iψ∂iψ* that doesn't appear on the L'

I'm close to the solution ?


text:
# Limiting behavior of the expected number of Hamiltonian cycles in the random graph $G(n, p)$.

So we have that the expected number of Hamiltonian cycles in the random graph $$G(n,p)$$ is given by:

$$\mu_{n}(p)=\frac{1}{2}(n-1)!p^{n}$$ for $$n \geq 3$$.

We now want to find lim$$_{n\to \infty}\mu_{n}(\frac{c}{n})$$, where $$c$$ is a constant.

Ok, we know that $$0 \leq p \leq 1$$ $$\implies$$ $$0 \leq \frac{c}{n} \leq 1 \implies c \geq 0$$ and $$n \geq c$$. I'm not sure how one should think about the last constraint, i.e. that $$n \geq c$$. So when we think about evaluating lim$$_{n\to \infty}\mu_{n}(\frac{c}{n})$$ do we just fix some $$c \geq 0$$? But then, for instance if take $$c = 111$$ and let $$n$$ go to infinity, for $$n<111$$ we're considering events with probability $$>1$$ which doesn't make sense. Or does it? Or should we restrict $$c$$, s.t. $$0\leq c \leq 3$$, so the constraints above are always satisfied?

How would we then go about finding lim$$_{n\to \infty}\mu_{n}(\frac{c}{n})$$?

• Since we are only considering the limit when n goes to infinity we can pick any c and assume that n is bigger than it. – ericf Mar 19 at 5:50

We can use Stirling's approximation, in the form $$n! \sim \sqrt{2 \pi n} \left(\frac ne\right)^n.$$ Here, we have $$\mu_n(\tfrac cn) = \frac{n! \cdot (\frac cn)^n}{2n} \sim \sqrt{\frac{\pi}{2n}} \left(\frac ne\right)^n \left(\frac cn\right)^n = \sqrt{\frac{\pi}{2n}} \left(\frac ce\right)^n.$$ Therefore,

• for $$c < e$$, $$\mu_n(\frac cn) \to 0$$ exponentially quickly.
• for $$c = e$$, $$\mu_n(\frac cn) \sim \sqrt{\frac{\pi}{2n}}$$, so $$\mu_n(\frac cn) \to 0$$ again, but at a reduced rate.
• for $$c > e$$, $$\mu_n(\frac cn) \to \infty$$ exponentially quickly.

In a finer parametrization $$p(n) = \frac{en + \frac12\ln n + f(n)}{n^2},$$ the same approximation shows that $$\mu_n(p(n)) \sim \sqrt{\frac\pi 2} e^{f(n)}$$ and the behavior of $$f(n)$$ determines the behavior of $$\mu_n(p(n))$$.

• That is nice, thank you! – amator2357 Mar 20 at 9:20


text:
## Chemistry: The Molecular Science (5th Edition)

Since the $m_{l}$ quantum number of the electrons is different, they are in different atomic orbitals. Both electrons have n= 3 and l=1, so they are in 3p orbital.


text:
# What now?¶

Beware of Methodologies. They are a great way to bring everyone up to a dismal, but passable, level of performance, but at the same time, they are aggravating to more talented people who chafe at the restrictions that are placed on them. –Joel Spolsky

If you stayed with it to here, well done – you’re going to Top Gun.

You should have enough skills to make a considerable dent in a problem collection such as Advent of Code, Project Euler, Perl Weekly Challenge or Project Rosalind. What you need now is practice.

A good place to start is the Dyalog Problem Solving Competition, an annual event that’s been running for a while. All the old problems are available and they are typically constructed to flatter APL’s capabilities.

Advent of Code is great – a nice ramp-up of difficulty and a Grand Tour of Computer Science 101. It’s my benchmark – I find that starting on Day01 knowing near nothing, by the time I get to Day25 I am reasonably proficient in a new language. So I thought we’d solve a couple of Advent of Code problems here. Obviously, if you don’t want to spoil it for yourself, come back here once you’ve had a go yourself.

⎕IO ← 0
⎕PP ← 34
]box on
]rows on
assert←{⍺←'assertion failure' ⋄ 0∊⍵:⍺ ⎕signal 8 ⋄ shy←0}

Was ON
Was OFF

## 20/3 Toboggan Trajectory¶

This one is day 3 from 2020. Note that my data will be different from yours - a feature of Advent of Code.

We have a 2D array of dots and hashes. The pattern is repeated indefinitely to the right. The example given is:

⎕←data ← 11 11⍴'..##.......#...#...#...#....#..#...#.#...#.#.#...##..#...#.##......#.#.#....#.#........##.##...#...#...##....#.#..#...#.#'

..##....... #...#...#.. .#....#..#. ..#.#...#.# .#...##..#. ..#.##..... .#.#.#....# .#........# #.##...#... #...##....# .#..#...#.#

which is then assumed to repeat to the right indefinitely, the first few like so:

data,⍣3⊢data

..##.........##.........##.........##....... #...#...#..#...#...#..#...#...#..#...#...#.. .#....#..#..#....#..#..#....#..#..#....#..#. ..#.#...#.#..#.#...#.#..#.#...#.#..#.#...#.# .#...##..#..#...##..#..#...##..#..#...##..#. ..#.##.......#.##.......#.##.......#.##..... .#.#.#....#.#.#.#....#.#.#.#....#.#.#.#....# .#........#.#........#.#........#.#........# #.##...#...#.##...#...#.##...#...#.##...#... #...##....##...##....##...##....##...##....# .#..#...#.#.#..#...#.#.#..#...#.#.#..#...#.#

The task is to calculate how many # we’d encounter if, starting from the top-left corner, we follow a path which for each iteration moves three steps to the right, and one step down. In this test case, we’d encounter 7, here indicated by quads, ⎕:

11 44⍴'0.##.........##.........##.........##.......#..0#...#..#...#...#..#...#...#..#...#...#...#....⎕..#..#....#..#..#....#..#..#....#..#...#.#...#0#..#.#...#.#..#.#...#.#..#.#...#.#.#...##..#..⎕...##..#..#...##..#..#...##..#...#.##.......#.⎕#.......#.##.......#.##......#.#.#....#.#.#.#.0..#.#.#.#....#.#.#.#....#.#........#.#........⎕.#........#.#........##.##...#...#.##...#...#.⎕#...#...#.##...#...#...##....##...##....##...#⎕....##...##....#.#..#...#.#.#..#...#.#.#..#...⎕.#.#..#...#.#'

0.##.........##.........##.........##....... #..0#...#..#...#...#..#...#...#..#...#...#.. .#....⎕..#..#....#..#..#....#..#..#....#..#. ..#.#...#0#..#.#...#.#..#.#...#.#..#.#...#.# .#...##..#..⎕...##..#..#...##..#..#...##..#. ..#.##.......#.⎕#.......#.##.......#.##..... .#.#.#....#.#.#.#.0..#.#.#.#....#.#.#.#....# .#........#.#........⎕.#........#.#........# #.##...#...#.##...#...#.⎕#...#...#.##...#... #...##....##...##....##...#⎕....##...##....# .#..#...#.#.#..#...#.#.#..#...⎕.#.#..#...#.#

We can generate the coordinates for the path directly. The y-coordinate just increases by 1. The x-coordinate increases by 3 until we “fall off” the end of the pattern, after which it resets back to zero. Let’s look at this specific case.

If we ignore the wrap-around in x, we can get our coordinates as (assuming the pattern is shape 11 11):

↓⍉↑1 3×⊂⍳11 ⍝ Remix to create y x pairs

┌───┬───┬───┬───┬────┬────┬────┬────┬────┬────┬─────┐ │0 0│1 3│2 6│3 9│4 12│5 15│6 18│7 21│8 24│9 27│10 30│ └───┴───┴───┴───┴────┴────┴────┴────┴────┴────┴─────┘

The wrap-around is just a mod by the shape of the pattern, in our case 11 11:

↓⍉↑11 11|1 3×⊂⍳11

┌───┬───┬───┬───┬───┬───┬───┬────┬───┬───┬────┐ │0 0│1 3│2 6│3 9│4 1│5 4│6 7│7 10│8 2│9 5│10 8│ └───┴───┴───┴───┴───┴───┴───┴────┴───┴───┴────┘

So we pick out the chars along this vector, look for # and sum:

+⌿'#'=data[↓⍉↑11 11|1 3×⊂⍳11]

7

So that works on the test data at least; good. Let’s make that generic in the pattern shape and the path offset:

data {+⌿'#'=⍺[↓⍉↑(⍴⍺)|⍵×⊂⍳⌈(≢⍺)÷⊃⍵]} 1 3

7

At this point we’re ready to try it on the real pattern.

data ← ↑⊃⎕NGET'../../AoCDyalog/data/2020/day03.txt'1

The real pattern is considerably larger, but as we’ve made our solution independent of pattern shape and path offset, this should not present a problem:

⍴data

323 31
⎕ ← part1 ← data {+⌿'#'=⍺[↓⍉↑(⍴⍺)|⍵×⊂⍳⌈(≢⍺)÷⊃⍵]} 1 3
assert 203=part1

203

That was the correct answer for part 1 (for me – you will get a different number).

For part 2, we’re given five slopes, and we need to multiply all results together. Not much left to do:

⎕ ← part2 ← ×⌿data∘{+⌿'#'=⍺[↓⍉↑(⍴⍺)|⍵×⊂⍳⌈(≢⍺)÷⊃⍵]}¨(1 1)(1 3)(1 5)(1 7)(2 1)
assert 3316272960=part2

3316272960

So that was kind of APL home-court advantage - a nice, array-oriented solution.

## 16/6 Signals and Noise¶

This one is day 6 from 2016.

Given a bunch of random-looking strings of letters of equal lengths, we’re to find the most commonly occurring letter in each position. We’re given the following test set:

data ← 'eedadn' 'drvtee' 'eandsr' 'raavrd' 'atevrs' 'tsrnev' 'sdttsa' 'rasrtv' 'nssdts' 'ntnada' 'svetve' 'tesnvt' 'vntsnd' 'vrdear' 'dvrsen' 'enarar'
↑data

eedadn drvtee eandsr raavrd atevrs tsrnev sdttsa rasrtv nssdts ntnada svetve tesnvt vntsnd vrdear dvrsen enarar

Picking out the most commonly occurring letter in each column spells easter. “Most commonly occurring” should have us reach for Key, ⌸. We remix the input data so that we get a vector of the columns, which we then Key:

{⍺,≢⍵}⌸¨↓⍉↑data

┌───┬───┬───┬───┬───┬───┐ │e 3│e 2│d 2│a 2│d 2│n 2│ │d 2│r 2│v 1│t 3│e 3│e 2│ │r 2│a 3│n 2│d 2│s 2│r 3│ │a 1│t 2│a 2│v 2│r 2│d 2│ │t 2│s 2│e 2│n 2│t 2│s 2│ │s 2│d 1│r 2│r 2│v 2│v 2│ │n 2│v 2│t 2│s 2│n 1│a 2│ │v 2│n 2│s 3│e 1│a 2│t 1│ └───┴───┴───┴───┴───┴───┘

Now we sort each array based on the last column, and select the element at 0 0 in each, which should be the most frequent letter:

∊{0 0⌷⍵[⍒⍵[;1];]}¨{⍺,≢⍵}⌸¨↓⍉↑data

easter

Looks promising. Let’s try that on the real data:

data ← ⊃⎕NGET'../../AoCDyalog/data/2016/06.txt'1

⎕ ← part1 ← ∊{0 0⌷⍵[⍒⍵[;1];]}¨{⍺,≢⍵}⌸¨↓⍉↑data
assert 'qrqlznrl'≡part1

qrqlznrl

Part 2 - the same, but now pick the least frequent. All we need to do is to swap grade down for grade up:

⎕ ← part2 ← ∊{0 0⌷⍵[⍋⍵[;1];]}¨{⍺,≢⍵}⌸¨↓⍉↑data
assert 'kgzdfaon'≡part2

kgzdfaon

If we wanted to be clever, we could make that an operator:

_day6 ← {∊⍺⍺{0 0⌷⍵[⍺⍺⍵[;1];]}¨{⍺,≢⍵}⌸¨↓⍉↑⍵}

⍒_day6 data
⍋_day6 data

qrqlznrl
kgzdfaon

## 18/10 The Stars Align¶

Back in 2018, we were treated to this gem on day 10. We have a bunch of “stars” each having a position and a velocity. At some moment in time, these stars will align, forming a message.

We can reasonably guess that the message forms when the bounding box of all the stars is at its smallest. If that isn’t the case, then this becomes much harder, so let’s try this hypothesis first.

Step one is to pick out four numbers, including negatives, from each line of the input data:

data ← ↑'-?\d+'⎕S{⍎⍵.Match}¨⊃⎕NGET'../../AoCDyalog/data/2018/10.txt'1

The first two columns are the position, and the last two columns the velocity.

pos ← ,/data[;0 1]
vel ← ,/data[;2 3]

We want to minimise the bounding box. We can use the sum of the width and height of the bounding box as a single number that we can compare. Here’s a function to compute that:

bbx ← {+/|-⌿2 2⍴(⌊⌿,⌈⌿)↑⍵} ⍝ Width + height of bounding box

So we want to repeatedly apply a function (the adding of the velocity to the position) to the output of itself until we reach a defined stopping condition – when the width plus the height of the bounding box no longer decreases. This sounds like a job for the power operator:

msg ← vel-⍨{vel+⍵}⍣{⍺>⍥bbx⍵}⊢pos ⍝ Add velocity until bounding box no longer decreases.

That’s actually it – but in order to actually see the message we need to prune it down to the bounding box, and for extra flair, use the * character, as shown in the question.

(w h) ← |-⌿2 2⍴(⌊⌿,⌈⌿)↑msg           ⍝ Width and height of bounding box
(-h+1)(-w+1)↑'*'@(⊖¨msg)⊢143 203⍴' ' ⍝ Prune our display, and make stars

****** ***** ***** ***** ***** ***** ****** ** * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ***** ***** ***** ***** ***** ***** * * * * * * * * * * * * * * ****** * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * ***** * * * * ****** * *

## 17/25 The Halting Problem¶

The Christmas Day problem from 2017. We’re asked to solve the Halting Problem for a Turing machine. We’re given a description of the state transitions – the program, basically. The description for state A is:

In state A:
If the current value is 0:
- Write the value 1.
- Move one slot to the right.
- Continue with state B.
If the current value is 1:
- Write the value 0.
- Move one slot to the left.
- Continue with state C.

Compiling the state descriptions into a table, we get to something along the lines of

0

1

A

1 R B

0 L C

B

1 L A

1 R D

C

1 R A

0 L E

D

1 R A

0 R B

E

1 L F

1 L C

F

1 R D

1 R A

which we can translate to a neat APL 6×6 matrix:

⎕ ← STATE ← 6 6⍴1 1 1 0 ¯1 2 1 ¯1 0 1 1 3 1 1 0 0 ¯1 4 1 1 0 0 1 1 1 ¯1 5 1 ¯1 2 1 1 3 1 1 0

1 1 1 0 ¯1 2 1 ¯1 0 1 1 3 1 1 0 0 ¯1 4 1 1 0 0 1 1 1 ¯1 5 1 ¯1 2 1 1 3 1 1 0

We’ll need a honking loooong magnetic tape, initially set to all zeros. We don’t know the exact length, but we know we are running 12919244 iterations, so let’s make the tape that size to start with.

TAPE ← 12919244⍴0

We also need a state transition function, applying the rules from the STATE table, reading from, and writing to TAPE:

]dinput
stf ← {
(pos state) ← ⍵
args ← (0 3[pos⊃TAPE])+⍳3
(nv m ns) ← STATE[state;args]
TAPE[pos] ← nv
(pos+m) ns
}

We need to run 12919244 iterations. We’ll start in the middle of the tape, and hope that it is long enough:

_ ← stf⍣12919244⊢6459622 0  ⍝ Takes ~20s

assert 4287=⎕←+/TAPE

4287

## Statsify All The Words!¶

Here’s a task that I was set many, many years ago for a job interview. The original task sheet is included below, suitably redacted, although I’m sure they have moved on from this by now. They were somewhat arrogant to say “use any programming language” – I only wish I’d known APL by then…

So the task is deliberately open-ended and not fully specified; part of the expectation was to show ability to work with ambiguous requirements yadda yadda. So we’ll make a note of the assumptions we make. First we need some test data in a file; let’s make a few paragraphs of Lorem Ipsum:

data←⊃⎕NGET'/Users/stefan/work/dyalog/learnapl/loremipsum.txt'1

1↑data

┌───────────────────────────────────────────────────────────────────────────────────────────────────────┐ │Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nullam non finibus felis, id imperdiet purus. │ └───────────────────────────────────────────────────────────────────────────────────────────────────────┘

This file contains Latin words, mixed case, punctuation, whitespace including empty lines. Our first assumption is that we will do all calculations case-insensitively. So we’ll pick out the words, and turn them to upper-case. There are a number of ways we could do this, but let’s use a regex solution that’s UTF-8-safe – we do want to be able to deal with not just English, right?

words←⎕←('\W' '.'⎕S 3⍠'UCP'1⊆⊢)1⎕C⊃,/data

Is that sufficient? Well, it depends if we expect the file to contain numbers, too:

('\W' '.'⎕S 3⍠'UCP'1⊆⊢) 'These are words 55, 56 and 57 in total. Number1nmiddle'

┌─────┬───┬─────┬──┬──┬───┬──┬──┬─────┬──────────────┐ │These│are│words│55│56│and│57│in│total│Number1nmiddle│ └─────┴───┴─────┴──┴──┴───┴──┴──┴─────┴──────────────┘

The problem statement says nothing about numbers. We could remove those first, like so:

('\W' '.'⎕S 3⍠'UCP'1⊆⊢) '\d'⎕R''⊢1⎕C'These are words 55, 56 and 57 in total. Number1nmiddle'

┌─────┬───┬─────┬───┬──┬─────┬─────────────┐ │THESE│ARE│WORDS│AND│IN│TOTAL│NUMBERNMIDDLE│ └─────┴───┴─────┴───┴──┴─────┴─────────────┘
words←('\W' '.'⎕S 3⍠'UCP'1⊆⊢) '\d'⎕R''⊢1⎕C⊃,/data

So we wanted to know the number of words and lines in the text. That’s just

(≢words)(≢data)

555 42

Next we want to find the most frequent letter. This is a fairly standard Key pattern, and here we also assume that if there are several most frequent letters, we should return all of them.

letters⌷⍨⊂0~⍨⊢/0,⊢⌸letters←∊words

E

Finally, we need the average word length, to one decimal place. Let’s build a vector of word lengths and average with a neat little train:

(+/÷≢)≢¨words

5.531531531531532

Oh, and to one decimal place:

⍎1⍕(+/÷≢)≢¨words

5.5

I think we have all the pieces. Let’s golf that down to confuse the interviewers properly:

{(≢w)(≢⍵)(⍎1⍕(+/÷≢)≢¨w)(l⌷⍨⊂0~⍨⊢/0,⊢⌸l←∊w←('\W' '.'⎕S 3⍠'UCP'1⊆⊢)'\d'⎕R''⊢1⎕C⊃,/⍵)}data

┌───┬──┬───┬─┐ │555│42│5.5│E│ └───┴──┴───┴─┘

If we didn’t care about numbers, and we were dealing with only the letters A-Z, we could have written the more compact

(≢w)(≢data)(⍎1⍕(+/÷≢)≢¨w)(l⌷⍨⊂0~⍨⊢/0,⊢⌸l←∊w←(∊∘⎕A⊆⊢)1⎕C⊃,/data)

┌───┬──┬───┬─┐ │555│42│5.5│E│ └───┴──┴───┴─┘

Now we just have to wait for them to “assess the program for style and readability; and also look at how easy your program is to extend”. Once we’re done being smug, we could of course have structured that a bit friendlier for people new to APL (what Aaron Hsu calls “the didactic style”):

]dinput
stats ← {
words   ← ('\W' '.'⎕S 3⍠'UCP'1⊆⊢) '\d'⎕R''⊢1⎕C⊃,/⍵ ⍝ Flatten, upcase, remove numbers, split on "non-word"
letters ← ∊words                                   ⍝ Join
common  ← letters⌷⍨⊂0~⍨⊢/0,⊢⌸letters               ⍝ Frequency table, picking the most frequent
lengths ← ≢¨words                                  ⍝ Vector of length of each word
ave     ← ⍎1⍕(+/÷≢)lengths                         ⍝ Average, and round to 1 decimal place
(≢words) (≢⍵) ave common
}

stats data

┌───┬──┬───┬─┐ │555│42│5.5│E│ └───┴──┴───┴─┘

Extensibility is an amorphous topic, and typically something lauded by Object Orientationists as something that has intrinsic value. Is the above function extensible? Sure! I can add another line to count (say) the number of paragraphs and tag that on the end of the returned vector. However, in an OO language, perhaps you’d created a class that could be extended through inheritance. As APLers, we’d say that this obscures the intent, hiding the functionality away in perhaps other files, instead of having all available within a line or two.

So the question you’d have to ask yourself – would the above effort have gotten you the job?

I think we know the answer to that :) They expected five pages of Java… Anyway, if you aspire to dazzle interviewers, take a leaf out of Aphyr’s book.


text:
# Positive definite matrix product

I want to prove that the matrix product $A^{-1}B$ is positive definite where A is a symmetric positive definite matrix and B is a symmetric matrix. I have tried to use the following theorem but I did not come up with a good proof. Any good reference it would be appreciated also.

Theorem1:

If A is positive definite, then A is invertible and $A^{-1}$ is positive definite.

Theorem2:

Let C be positive definite and D symmetric of the same order. Then there exist a non-singular matrix P and a diagonal matrix Λ such that $$C = PP^{T} \, and \,D = PΛP^{T}$$

Theorem2 is extracted from the book "Matrix Differential Calculus with Applications in Statistics and Econometrics" and the proof is highly interesting. Also the first equality that refers to the positive definite matrix is also called the Cholesky decomposition.

• Your hypotheses are probably incomplete. If $B=0$ (which is symmetric), then $A^{-1}B=0$ is not positive definite. – minar Aug 23 '13 at 12:29
• If we suppose that B can not be zero can we prove what am I asking? – darkmoor Aug 23 '13 at 12:39
• You got your answer. IMO, you should open a new question now, instead of just editing the old one. Also, I don't understand what your current question is, since you say that the above theorem has a "highly interesting proof" (so, you know it). – Vedran Šego Aug 24 '13 at 15:38

Write $A=R^2$ where R is the "classic" square root of $A$ and is symmetric positive definite.
Then $$AB=R(RBR)R^{-1}$$ $RBR$ is symmetric, positive.
If you want to deal with $A^{-1}$ instead of $A$ do the same.
• Read more carefully please : _ Diagonalize A and take the square root of the diagonal to get R symmetric (e.g en.wikipedia.org/wiki/… ) _ R is then symmetric positive definite. _ Then I never said that AB was symmetric positive, but I said that is was similar to a symmetric positive matrix (which is $RBR$ in this case). – Bertrand R Aug 23 '13 at 14:00


text:
Forum moved here!

# Home / <Ctrl> + Left Mouse select area (can then use with copy, print or +/- zoom)

matrix

I’ve recently discovered that you can zoom to a highlighted area with the use of the + key as per topic description. Great stuff!

I was wondering whether there could be a single keystroke to zoom immediately to the selected area rather than multiple (+) keystrokes based on the zoom percentage scale?

I.e. select a particular area of PDF via the ctrl + left mouse, then a single keystroke to zoom to this exact location? (fit selection)

Is this possible?

kjk

I’ve opened https://github.com/sumatrapdfreader/sumatrapdf/issues/1699 to track this request.

Russ

You can also place the mouse in the area that you want to zoom into, hold the right mouse button down and use the scroll wheel to zoom in and out - no keystrokes required at all.

GitHubRulesOK

Thanks for the reminder, that the zoom works where the focus is, however for touchscreen there is not a single focus and the zoom for those users works around top left origin, not the mid point between digits. that is covered by issue

There is as per above point of focus a semi woraround since if you can double tap a selectable word or right click to select all then the pinch to zoom tends to work about that word or the whole page as pinch points !


text:
A list of interesting R/Stats quickies to keep the mind distracted:

• A long draft Advanced Data Analysis from an Elementary Point of View by Cosma Shalizi, in which he uses R to drive home the message. Not your average elementary point of view.
• Good notes by Frank Davenport on starting using R with data from a Geographic Information System (GIS). Read this so you get a general idea of how things fit together.
• If you are in to maps, Omnia sunt Communia! provides many good tips on producing them using R.
• Mark James Adams reminded us that Prediction ≠ Understanding, probably inspired by Dan Gianola‘s course on Whole Genome Prediction. He is a monster of Bayesian applications to genetic evaluation.
• If you are in to data/learning visualization you have to watch Bret Victor’s presentation on Media for thinking the unthinkable. He is so far ahead what we normally do that it is embarrassing.
• I follow mathematician Atabey Kaygun in twitter and since yesterday I’ve been avidly reading his coverage of the protests in Turkey. Surely there are more important things going on in the world than the latest R gossip.

I’m marking too many assignments right now to have enough time to write something more substantial. I can see the light at the end of the tunnel though.

Winter flying between Blenheim and Christchurch (Photo: Luis, click to enlarge).

I was working with a small experiment which includes families from two Eucalyptus species and thought it would be nice to code a first analysis using alternative approaches. The experiment is a randomized complete block design, with species as fixed effect and family and block as a random effects, while the response variable is growth strain (in $$\mu \epsilon$$).

When looking at the trees one can see that the residual variances will be very different. In addition, the trees were growing in plastic bags laid out in rows (the blocks) and columns. Given that trees were growing in bags siting on flat terrain, most likely the row effects are zero.

Below is the code for a first go in R (using both MCMCglmm and ASReml-R) and SAS. I had stopped using SAS for several years, mostly because I was running a mac for which there is no version. However, a few weeks ago I started accessing it via their OnDemand for Academics program via a web browser.

The R code using REML looks like:

# Options options(stringsAsFactors = FALSE) setwd('~/Dropbox/research/2013/growth-stress')   # Packages require(ggplot2) require(asreml) require(MCMCglmm)     # Reading data, renaming, factors, etc gs = read.csv('eucalyptus-growth-stress.csv') summary(gs)   # Both argophloia and bosistoana gsboth = subset(gs, !is.na(strain)) gsboth = within(gsboth, { species = factor(species) row = factor(row) column = factor(column) fam = factor(fam) })       ma = asreml(strain ~ species, random = ~ fam + row, rcov = ~ at(species):units, data = gsboth)   summary(ma)$varcomp # gamma component std.error z.ratio constraint #fam!fam.var 27809.414 27809.414 10502.036 2.6480022 Positive #row!row.var 2337.164 2337.164 3116.357 0.7499666 Positive #species_E.argopholia!variance 111940.458 111940.458 26609.673 4.2067580 Positive #species_E.bosistoana!variance 63035.256 63035.256 7226.768 8.7224681 Positive While using MCMC we get estimates in the ballpark by using: # Priors bothpr = list(R = list(V = diag(c(50000, 50000)), nu = 3), G = list(G1 = list(V = 20000, nu = 0.002), G2 = list(V = 20000, nu = 0.002), G3 = list(V = 20000, nu = 0.002))) # MCMC m2 = MCMCglmm(strain ~ species, random = ~ fam + row + column, rcov = ~ idh(species):units, data = gsboth, prior = bothpr, pr = TRUE, family = 'gaussian', burnin = 10000, nitt = 40000, thin = 20, saveX = TRUE, saveZ = TRUE, verbose = FALSE) summary(m2) # Iterations = 10001:39981 # Thinning interval = 20 # Sample size = 1500 # # DIC: 3332.578 # # G-structure: ~fam # # post.mean l-95% CI u-95% CI eff.samp # fam 30315 12211 55136 1500 # # ~row # # post.mean l-95% CI u-95% CI eff.samp # row 1449 5.928 6274 589.5 # # R-structure: ~idh(species):units # # post.mean l-95% CI u-95% CI eff.samp # E.argopholia.units 112017 71152 168080 1500 # E.bosistoana.units 65006 52676 80049 1500 # # Location effects: strain ~ species # # post.mean l-95% CI u-95% CI eff.samp pMCMC # (Intercept) 502.21 319.45 690.68 1500 <7e-04 *** # speciesE.bosistoana -235.95 -449.07 -37.19 1361 0.036 * The SAS code is not that disimilar, except for the clear demarcation between data processing (data step, for reading files, data transformations, etc) and specific procs (procedures), in this case to summarize data, produce a boxplot and fit a mixed model. * termstr=CRLF accounts for the windows-like line endings of the data set; data gs; infile "/home/luis/Projects/euc-strain/growthstresses.csv" dsd termstr=CRLF firstobs=2; input row column species$ family $strain; if strain ^= .; run; proc summary data = gs print; class species; var strain; run; proc boxplot data = gs; plot strain*species; run; proc mixed data = gs; class row species family; model strain = species; random row family; repeated species / group=species; run; /* Covariance Parameter Estimates Cov Parm Group Estimate row 2336.80 family 27808 species species E.argoph 111844 species species E.bosist 63036 */ SAS boxplot for the data set. I like working with multiple languages and I realized that, in fact, I missed SAS a bit. It was like meeting an old friend; at the beginning felt strange but we were quickly chatting away after a few minutes. Snowless December in Uppsala (Photo: Luis, click to enlarge). At School we use R across many courses, because students are supposed to use statistics under a variety of contexts. Imagine their disappointment when they pass stats and discovered that R and statistics haven’t gone away! When students start working with real data sets one of their first stumbling blocks is subsetting data. We have data sets and either they are required to deal with different subsets or there is data cleaning to do. For some reason, many students struggle with what should be a simple task. If one thinks of data as as a matrix/2-dimensional array, subsetting boils down to extracting the needed rows (cases) and columns (variables). In the R world one can do this in a variety of ways, ranging from the cryptic to the explicit and clear. For example, let’s assume that we have a dataset called alltrials with heights and stem diameters for a number of trees in different trials (We may have a bunch of additional covariates and experimental design features that we’ll ignore for the moment). How do we extract all trees located in Christchurch? Two common approaches are: mytrial = alltrials[alltrials$location == 'Christchurch', ]   mytrial = subset(alltrials, location == 'Christchurch')

While both expressions are equivalent, the former reads like Klingon to students, while the latter makes explicit that we are obtaining a subset of the original data set. This can easily be expanded to more complex conditions; for example to include all trees from Christchurch that are taller than 10 m:

mytrial = alltrials[alltrials$location == 'Christchurch' & alltrials$height > 10, ]   mytrial = subset(alltrials, location == 'Christchurch' & height > 10)

I think the complication with the Klingonian notation comes mostly from two sources:

• Variable names for subsetting the data set are not directly accessible, so we have to prefix them with the NameOfDataset$, making the code more difficult to read, particularly if we join several conditions with & and |. • Hanging commas: if we are only working with rows or columns we have to acknowledge it by suffixing or prefixing with a comma, which are often forgotten. Both points result on frustrating error messages like • Error in [.data.frame(alltrials, location == 'Christchurch', ) : object 'location' not found for the first point or • Error in [.data.frame(alltrials, alltrials$location == 'Christchurch') undefined columns selected for the second point.

The generic forms of these two notations are:

dataset[what to do with rows, what to do with columns]   subset(dataset, what to do with rows, what to do with columns)

We often want to keep a subset of the observed cases and keep (or drop) specific variables. For example, we want to keep trees in 'Christchurch' and we want to ignore diameter, because the assessor was 'high' that day:

# With this notation things get a bit trickier # The easiest way is to provide the number of the variable # Here diameter is the 5th variable in the dataset mytrial = alltrials[all.trials$location == 'Christchurch' & all.trials$height > 10, -5]   # This notation is still straightforward mytrial = subset(alltrials, location == 'Christchurch' & height > 10, select = -diameter)

There are, however, situations where Klingon is easier or more efficient. For example, to take a random sample of 100 trees from the full dataset:

  mytrial = alltrials[sample(1:nrow(alltrials), 100, replace = FALSE),]

If you are interested in this issue Quick-R has a good description of subsetting. I'm sure this basic topic must has been covered many times, although I doubt anyone used Klingon in the process.

Gratuitous picture: building blocks for R (Photo: Luis).

Suburbia bus stop, Christchurch (Photo: Luis, click to enlarge).

This morning Radio New Zealand covered a story (audio) where Tomatoes New Zealand (TNZ, the growers association) was asking the Government to introduce compulsory labeling for irradiated products (namely imported Australian tomatoes), stating that consumers deserve an informed choice (TNZ Press Release). Two points that I think merit attention:

• Food irradiation is perfectly safe: it does not make food radioactive, it does not alter the nutritional value of food and reduces the presence (or completely eliminates) the presence of microorganisms that cause disease or pests (the latter being the reason for irradiation in this case).
• The second point is that the call for labeling does not come from consumers (or an organization representing them) but from producers that face competition.

This situation reminded me of Milton Friedman talking about professional licenses The justification offered is always the same: to protect the consumer. However, the reason is demonstrated by observing who lobbies. Who is doing the lobbying is very telling in this case, particularly because there is no real reason to induce fear on the consumer, except that irradiation sounds too close to radioactive, and therefore TNZ is hoping to steer consumers away from imported tomatoes. Given that TNZ is for informing the consumer they could label tomatoes with ‘many of the characteristics in this tomato are the product of mutations‘. Harmless but scary.

P.S. The media is now picking up the story. Shameful manipulation.
P.S.2 Give that TNZ is in favor of the right to know I want a list of all chemicals used in the production of New Zealand tomatoes, how good is their water management and the employment practices of tomato growers.

I was fascinated by the shape of the power cords (Photo: Luis, click to enlarge).

It used to be that the one of the first decisions to make when learning to program was between compiled (e.g. C or FORTRAN) and interpreted (e.g. Python) languages. In my opinion these days one would have to be a masochist to learn with a compiled language: the extra compilation time and obscure errors are a killer when learning.

Today the decision would be between using a generic interpreted language (e.g. Python) and an interpreted domain specific language (DSL) like R, MATLAB, etc. While some people prefer generic languages, I’d argue that immediate feedback and easy accomplishment of useful tasks are a great thing when one is learning something for the first time.

As an example, a while ago my son asked me what I was doing in the computer and I told him that I was programming some analyses in R. I showed that the program was spitting back some numbers and plots, a fact that he found totally unremarkable and uninteresting. I searched in internet and I found Scratch, a visual programming language, that let’s the user moves blocks representing code around and build interactive games: now my son was sold. Together we are learning about loops, control structures and variables, drawing characters, etc. We are programming because the problems are i- much more interesting for him and ii- achievable in a short time frame.

An example scratch script.

Learning to program for statistics, or other scientific domains for that matter, is not that different from being a kid and learning programming. Having to do too much to get even a mildly interesting result is frustrating and discouraging; it is not that the learner is dumb, but that he has to build too many functions to get a meager reward. This is why I’d say that you should use whatever language already has a large amount of functionality (‘batteries included’ in Python parlance) for your discipline. Choose rightly and you are half-way there.

‘But’ someone will say, R is not a real language. Sorry, but it is a real language (Turing complete and the whole shebang) with oddities, as any other language, granted. As with human languages, the more you study the easier it gets to learn a new language. In fact, the syntax for many basic constructs in R is highly similar to alternatives:

# This is R code # Loop for(i in 1:10){ print(i) } #[1] 1 #[1] 2 #[1] 3 #[1] 4 #[1] 5 #[1] 6 #[1] 7 #[1] 8 #[1] 9 #[1] 10   # Control if(i == 10) { print('It is ten!') }   #[1] "It is ten!"
# This is Python code # Loop for i in range(1,11): # Python starts indexing from zero print(i)   #1 #2 #3 #4 #5 #6 #7 #8 #9 #10   # Control if i == 10: print("It is ten!")   #It is ten!

By the way, I formatted the code to highlight similarities. Of course there are plenty of differences between the languages, but many of the building blocks (the core ideas if you will) are shared. You learn one and the next one gets easier.

How does one start learning to program? If I look back at 1985 (yes, last millennium) I was struggling to learn how to program in BASIC when, suddenly, I had an epiphany. The sky opened and I heard a choir singing György Ligeti’s Atmosphères (from 2001: a Space Odyssey, you know) and then I realized: we are only breaking problems in little pieces and dealing with them one at the time. I’ve been breaking problems into little pieces since then. What else did you expect? Anyhow, if you feel that you want to learn how to code in R, or whatever language is the best option for your problems, start small. Just a few lines will do. Read other people’s code but, again, only small pieces that are supposed to do something small. At this stage is easy to get discouraged because everything is new and takes a lot of time. Don’t worry: everyone has to go through this process.

Many people struggle vectorizing the code so it runs faster. Again, don’t worry at the beginning if your code is slow. Keep on writing and learning. Read Norman Noam Ross’s FasteR! HigheR! StongeR! — A Guide to Speeding Up R Code for Busy People. The guide is very neat and useful, although you don’t need super fast code, not yet at least. You need working, readable and reusable code. You may not even need code: actually, try to understand the problem, the statistical theory, before you get coding like crazy. Programming will help you understand your problem a lot better, but you need a helpful starting point.

Don’t get distracted with the politics of research and repeatability and trendy things like git (noticed that I didn’t even link to them?). You’ll learn them in time, once you got a clue about how to program.

P.S. The code used in the examples could be shortened and sped up dramatically (e.g. 1:10 or range(1, 11)) but it is not the point of this post.
P.S.2. A while ago I wrote R is a language, which could be useful connected to this post.

It was a rainy day in Christchurch. Looking forward to receive the new windscreen for the recorder.