text:
# Need some help on a proof!

1. Dec 12, 2004

### MathematicalMatt

Howdy, I just stumbled on this forum and was hoping someone could help with this proof:

If a and b are elements of the Rational Number set, then a*b=0, if and only if a=0 or b=0

With that in mind, I need to prove that:

• Prove that if a=0 or b=0, then a*b=0
• Prove that if a*b=0, then a=0 or b=0

Any help is appreciated, cheers.

2. Dec 12, 2004

### Muzza

a) One can assume, without loss of generality, that a = 0. Then for any b, we have that ab = 0b = (0 + 0)b = 0b + 0b. Subtract 0b from both sides and you'll find that 0b - 0b = 0b, or equivalently 0b = 0.

b) Can we assume that every non-zero rational has an inverse? If both a and b are zero, then we are done. Suppose a is non-zero. Then b = 0*a^-1 = 0. The same argument can be repeated if b is non-zero.

Last edited: Dec 12, 2004
3. Dec 12, 2004

### HallsofIvy

Staff Emeritus
One of the "axioms" or defining properties of the rational numbers is that every rational number, except 0, has a multiplicative inverse.

Given that ab= 0, either
1) a= 0 in which case we are done, or

2) a is not 0, in which case a-1(ab)= a-10 or b= 0.

4. Dec 12, 2004

### MathematicalMatt

Thanks for the help!

format: plain


text:
# Homework Help: Even parity => symmetric space wave function?

1. Mar 5, 2007

### mrandersdk

If I have af wavefunction that is a product of many particle wavefunctions

$$\Psi = \psi_1(r_1)\psi_2(r_2) ... \psi_n(r_n)$$

If I then know that the parity of $$\Psi$$ is even. Can I then show that the wavefunction i symmetric under switching any two particles with each other. That is

$$\psi_1(r_1)\psi_2(r_2) ...\psi_i(r_i) ... \psi_j(r_j) ... \psi_n(r_n) = =\psi_1(r_1)\psi_2(r_2) ...\psi_j(r_j) ... \psi_i(r_i) ... \psi_n(r_n)$$

for any i and j between 1 and n?

It may be used that the parity operator commutes with the hamilton of the system if nessesary, and that the interaction between the particles only depends on the distance between any two particles.

It is clear that if the system only consist of two particles, and we use one of the particles as the 0-point of our coordinatesystem, the parity operator does the same as changing the particles, and then even parity means even space function, but when n is greater than 2, I can't see it.

Hope someone understand what i'm asking, because the result is used frequently in my course of particle physics.

2. Mar 5, 2007

### Meir Achuz

For two particles, parity changes r_12 into -r_12, which has the same effect as interchanging r_1 and r_2. For your many body wave function this isn't true, so parity is unrelated to particle interchange. Parity changes all r_i to -r_i. You have to look at the detailed wave function to see what this does. It depends on how your WF relates all the coordinates.

3. Mar 5, 2007

### mrandersdk

ok that was my thought too. Let me give an example how it is used, because I would like to know how th author solves the excercise. It's from nuclear and particle physics by B.R. Martin.

Consider a scenario where overall hadronic wavefunction $$\Psi$$ consist of just spin and space part, i.e. $$\Psi = \psi_{space} \psi_{spin}$$. What would be the resulting multplet structure of the lowest-lying baryon states composed of u,d and s quarks?

The autors own solution:
'Low lying' implies that the internal orbital momentum between the quarks is zero. Hence the parity is P = +1 and space is symmetric. Since the Pauli principle requires the overall wavefunction to be antisymmetric under the interchange of any pair of like quarks, it follows that $$\psi_{spin}$$ is antisymmetric...

The rest of the excercise makes sense, but how does he conclude that the space part i symmetric, to me it seems like he use parity symmetric and symmetric under switching of particles.

4. Mar 5, 2007

### Meir Achuz

That is a three body case. If the three body WF is written using Jacobi coordinates, the state with Daliz angular momenta L and l both equal zero is spatially symmetric. This is most easily seen for a harmonic oscillator wave function. Martin has made a common (historic) mistake of assuming more about parity than is needed. There are positive parity three body states that are not spatially symmetric.

5. Mar 5, 2007

### mrandersdk

ok i never heard of Jacobi coordinates or Daliz angular momenta. I'm not sure what you are saying, is it correct of him to say that positive parity gives a symmetric WF under switching any pair of quarks ?

6. Mar 5, 2007

### Meir Achuz

No. There are many positive parity quark wave functions that are not spatially symmetric. The Jacobi coordinates are r=r1-r2, and rho=r3-(r1-r2)/2, but normalized with some square roots to make them more symmetric.
For three bodies there are only two independent angular momenta in the cm system. Dalitz introduced them to analyze three body systems. One is related to r1-r2, and the other to the rho coordinate.

7. Mar 5, 2007

### mrandersdk

thanks for your help. There are many faults in this book so sometimes it's hard to figure out what to believe, so you got to be critical.

I was wandering if the parity of a system is independent of what coordinatsystem you use? Or do you got to have a coordinatsystem before you find the parity?

8. Mar 5, 2007

### Meir Achuz

The parity is independent of the coord system, but it is easier to see in some systems. It is easiest if all independent angular momenta are known.


text:
×

# Inequality that is Symmetrical but not Symmetric

1) Show that for any 3 real numbers $$a, b, c$$, we have

$\sqrt{ a^2 - ab + b^2 } + \sqrt{ b^2 - bc + c^2 } \geq \sqrt{ a^2 + ac + c^2 }.$

When does equality occur?

2) If $$a, b$$ and $$c$$ are real numbers such that $$a \geq c, b \geq c, c > 0$$, show that

$\sqrt{c(a-c) } + \sqrt{ c (b-c) } \leq \sqrt{ab} .$

When does equality occur?

3) Let $$a, b$$ and $$c$$ be real numbers with absolute value less than or equal to 1. Show that

$ab+bc+ca + 1 \geq 0.$

When does equality occur?

Hint: I used a tag of #Geometric Interpretation.

Note by Calvin Lin
3 years ago

## Comments

Sort by:

Top Newest

Diagram for (2):

http://i.imgur.com/FXhnSs5.png

· 3 years ago

Log in to reply

1) Angle between a and b is 60 deg, angle between b and c is 60 deg and it is a case of triangular inequality. · 3 years ago

Log in to reply

×

Problem Loading...

Note Loading...

Set Loading...


text:
# [NTG-context] Strange behavior with the command \color[]{}

Thu Nov 6 20:45:36 CET 2014

On Thu, 6 Nov 2014, Fabrice wrote:

> Hi Hans,
> It's true, I'm a little confused. In fact, the problem is the \
> starttikzpicture \ endtikzpicture.

As Hans said, you need to create a smaller example, and explain what you
mean by "the problem".

A blind guess is that \starttikzpicture ... \stoptikzpicture creates a
\vbox rather than an \hbox. You can try:

\dontleavehode\starttikzpicture ... \stoptikzpicture

or

\hbox{\starttikzpicture ... \stoptikzpicture}

and see if you get the desired output.


text:
## Abstract

This document describes the best practices for building and deploying large-scale recommender systems using NVIDIA GPUs. These practices are the culmination of years of research and development in GPU-accelerated tools for recommender systems, as well as building recommender systems for our in-house products and top-performing solutions for international recommendation systems competitions.

## 1. Introduction

Recommender systems are the economic engine of the Internet. These applications have a direct impact on our daily digital lives in which they filter products and services among an overwhelming number of options, easing the paradox of choice that most users face.

The primary goal of this document is to provide best practices for building and deploying large-scale recommender systems using NVIDIA® GPUs. These best practices are the culmination of years of research and development in GPU-accelerated tools for recommender systems, as well as building recommender systems for our in-house products and top-performing solutions for international recommendation systems competitions.

This document doesn’t only provide best practices for building and deploying large-scale recommender systems, but it also covers all facets of recommender systems with a focus on GPU acceleration and optimization.

## 2. Stages and Components

At a high level, the goal of a recommender system is to recommend relevant items to users from a large catalog that may contain up to millions or even billions of items. The system narrows them down to a consumable subset of items that are most likely to be of the user’s interest.

### 2.1. Recommendation Stages

A complete recommendation process generally consists of four stages as depicted in Figure 1.
Figure 1. Four stages of recommender systems

Retrieval
Candidate items are generated from the full item catalog, which may be very large. A popular approach to retrieval involves building an embedding model. This is a dense vector space that represents users and items. Approximate Nearest Neighbor (ANN) search is then employed to efficiently retrieve the most suitable candidates.
Filtering
Invalid candidates are filtered out. For example, movies that are not suitable for user demographic, such as region and age, availability, or movies that users already watched.
Scoring
Valid candidates are scored using a richer set of user, item, and session features and a more expressive model such as a deep neural network.
Ordering
A set of top-K candidates are identified. These top-K candidates are reordered according to other business logics and priorities. For example, if there was a marketing campaign for a certain product category or manufacturer, this would be taken into account.

### 2.2. Components of a Recommender System

Recommender systems contain several key components. As depicted in Figure 2, these components are categorized into four functional areas:
Figure 2. Components of a recommender system

### 2.2.1. Data Pipeline

The data pipeline contains components for:
• Data Acquisition, Validation, and Data Store: User, product, and interaction data, such as views and purchases, are continuously collected and put into a data store like a database or data lake. This data can reach terabytes (TBs) or petabytes (PBs) in scale.
• Data Extraction, Preprocessing, and Feature Engineering: The raw data sources for training a recommender model need to be aggregated, extracted, and cleaned to create features that are relevant to the problem at hand and help increase the accuracy of the system. This processing can be specific to the model that needs to be trained. For example, a decision tree model and a DLRM model may require different preprocessing. This is often an iterative and time consuming process. However, there are tools, such as NVIDIA NVTabular and RAPIDS, available to accelerate this on GPUs for development and production.

Data preprocessing and feature engineering occurs at the beginning in bulk during the building phase of the system, and then they occur continuously during the production phase where models get updated using the new data.

• Feature Store: Engineered features are often stored in a feature store that can be used across both training and production inference for consistency.

### 2.2.2. Training and Continuous Retraining

Initial training of models takes place so that candidates can be generated and scored. These model candidates are trained on a large amount of available data and deployed. Continuous incremental retraining ensures that the model stays up to date and captures the latest trends and user preferences. A model validation module is used to ensure that the model meets a specified quality threshold.

### 2.2.3. Deployment and Serving

Newly qualified models are automatically redeployed into production in a seamless manner after initial deployment. This should ideally support canary rollout and rollback. The number of inference servers should also automatically scale up and down as needed.

### 2.2.4. Logging and Monitoring

Modules are continuously monitored so that the quality of the recommendation can be measured in real time through a range of KPIs, such as hit rate and conversion rate. The modules trigger full retraining should model drift occur, such as when certain KPIs fall below known established baselines.

## 3.1. Multi-Modal Feature Extraction Techniques

Aside from tabular data, which is the most popular data source for building recommendation systems, multi-modal data can also be used where possible. Examples include user and product images, free text descriptions, and user text profiles. Features from these modalities can be extracted using domain specific neural networks, such as ResNet for images and BERT for text. These features are then combined with other numerical and categorical features. During the SIGIR eCommerce challenge, we discovered that it was better to apply L2-normalization instead of using the original vectors for the features that are based on text and image. Applying layer normalization individually to each multimodal feature before concatenation with other features improves performance.

## 3.2. Data Augmentation Techniques

Domain specific data augmentation techniques can be used to enrich the training data. Examples of this include reversing sequences and adding auxiliary data to "non-products".

The goal of the WSDM WebTour Challenge 2021 by Booking.com was to predict the last city destination for a traveler’s trip based on their previous booking history throughout the course of that trip. To augment the data, the trip sequence was reversed, which effectively resulted in twice the amount of training data. Using this approach, NVIDIA data scientists found that the prediction accuracy was improved.

In the SIGIR eCommerce Workshop Challenge 2021 by Coveo, leveraging users’ interactions with "non-product" pages by treating them as a virtual product largely improved our recommendation accuracy. This technique is worth trying for other recommendation problems where page views not associated with items are available.

## 3.3. Creating and Extracting More Features

Machine learning practitioners have adopted various feature engineering techniques. These feature engineering techniques include the following:
• Target Encoding: Replaces a categorical value with the mean of the target variable.
• Count Encoding: Replaces categories with their counts computed on the train set.
• Lagged Features: Generated from the difference between two points in time.
• Temporal Cyclic Features: Extracts features that exhibit cyclical patterns such as the hour of day and day of the week.
• Feature Crossing: Two or more features are combined to create a new feature.

In addition to these feature engineering techniques, domain knowledge should always be leveraged to create features. Examples of this include the extraction of mentions and hashtags in Twitter tweets as they represent entities of special interest.

## 4. Model Training

For the purposes of simplicity, we’re categorizing recommendation models as either traditional machine learning models or deep learning models.

According to the NVIDIA KGMON team, it’s always important to start really simple by using a few features, simple model types, and architectures to develop a baseline, and then add more complex model types and architectures together with additional features.

### 4.1. Traditional Machine Learning Models

Traditional machine learning models include models such as matrix factorization and gradient boosted models, which are well-proven models that have been deployed widely in production. Tree-based gradient boosted models are high performing models that are often featured in machine learning and recommendation system competitions.

In the "Why Are Deep Learning Models Not Consistently Winning Recommender Systems Competitions Yet?" research paper, NVIDIA researchers gave an explanation as to why newer deep learning based recommendation models are not consistently winning over traditional machine learning models in public competitions. We can substantiate this claim with our recent RecSys 2020, Recsys 2021, and the Booking.com challenge wins, as well as our second place solution in the SIGIR eCommerce Challenge (Task 2), in which Gradient Boosting Machines (GBM) were featured as an important ingredient. Although deep learning models were also used in these competitions, they don’t completely dominate traditional machine learning models.

### 4.1.1. Training and Deployment of Tree-Based GBM on GPUs

Popular GBM libraries such as XGboost and LightGBM are all accelerated on the GPU. NVIDIA Triton Inference Server FIL Backend allows forest models that are trained by several popular machine learning frameworks, including XGBoost, LightGBM, Scikit-Learn, and cuML, to be deployed in a Triton Inference Server using the RAPIDS Forest Inference Library for fast GPU-based inference. Using this backend, forest models can be deployed seamlessly alongside deep learning models for fast, unified inference pipelines.

### 4.2. Deep Learning Recommender Models

The increasing availability of large multi-modal datasets in the RecSys domain along with advancements of deep learning within NLP, computer vision, and speech processing has motivated the exploration of deep learning models for building recommender systems in recent years.

Figure 3 depicts the architecture of DLRM (Deep Learning Recommender Model). The inputs to a DLRM can be numerical features or categorical features. Numerical features, such as the number of movies watched, can be directly inputted into MLP layers. However, categorical features, such as country, language, movie title, and genre need to have a numerical representation that can be processed.

Figure 3. Architecture diagram of DLRM

Unlike image or audio data, which are typically numerical and continuous values, the categorical inputs in cases of recommenders are non-numerical and thus require an intermediate processing step to convert them into numerical vectors so that they can be processed. There are two primary ways to do this:
• Encoding: One hot or multi-hot encoding is used to turn categorical features into sparse vectors of 0s and 1s. These sparse numerical vectors can then be directly inputted to a machine learning model, such as a tree-based gradient boosted model or a neural network like the wide part of the Wide & Deep Learning model (WDL).
• Embedding: Categorical features are mapped to integer IDs for indexing, and these IDs are embedded into a multi-dimensional vector space. This multi-dimensional representation/mapping is learned during training. As compared to the sparse vectors generated by encoding, the embedding dimensionality is much smaller in comparison.

### 4.2.1.1. Batch Sizes and GPU Memory Bandwidth

Since the size of embedding tables depends primarily on the number of distinct categories (vocabulary size) in the feature, the embedding vector size (dimensionality of the embedding vector), and the precision, the embedding table size is calculated as follows:

An embedding table lookup for a one-hot feature operation roughly accesses bytes in memory with embeddings in single-precision.

To generalize it for multi-hot features, and also accounting for any memory used due to index lookup, the memory accessed would be calculated as follows where $\mathrm{NNZ}$ represents the number of non-zeros or number of entries in the feature field with indices in INT64:

Optimizing GPU memory traffic requires having enough accesses (read/write) in flight at any given time. Based on observation, for smaller batch sizes, higher embedding widths use the GPU memory bandwidth more effectively. For larger batch sizes, the memory traffic saturates as even the narrower width creates enough workload to better utilize it.

Figure 4. GPU memory bandwidth (GB/s) usage for different batch sizes

GPU memory bandwidth (GB/s) usage for different batch sizes shows the GPU memory bandwidth (GB/s) usage for different batch sizes using the following experimental settings:
• A100 40 GB GPU with 1.55 TB/s memory bandwidt
• Single embedding table that contains 10M categories tested with three different widths (32, 64, and 128 FP32 elements)
• A single 64-hot, which requires reading 64 vectors, and summing them so that they can be combined
• Embedding table row accesses follow the uniform distribution

Based on GPU memory bandwidth (GB/s) usage for different batch sizes, a batch size of 4K-16K is preferred for one 64-hot table.If N such tables need to be looked up concurrently, the same total memory traffic can be achieved with (batch size / N).This is because the total memory accessed in a batch remains the same. As a best practice, use the largest batch size that allows you to achieve a good enough accuracy to remain efficient.

### 4.2.1.2. Embedding Width

Embedding width refers to the size (in bytes) of each vector in an embedding table. For example, if each vector in a table has an embedding vector size of 16 and is represented in FP32 precision, the embedding width will be 64 bytes.

It is recommended to pad embedding widths to a multiple of 16 bytes. Typically, multiples of 64 or 128 bytes lead to even better performance.

Based on Figure 5, we see that choosing or padding embedding widths to a multiple of 16 bytes leads to an average performance speedup of 1.05, a maximum performance speedup of 1.26, and a slowdown of 0.97. The larger speedup is seen closer to the edges before the multiple occurs. An example of this would be choosing a width of 32 bytes in place of 31 bytes. The additional capacity may even benefit the model with a better representation.

Figure 5. Benefits of padding embedding widths to a multiple of 16 bytes

Figure 5 demonstrates the speed of the embedding access operation at different embedding widths using the following experimental settings:
• A100 40 GB GPU with 1.55 TB/s memory bandwidth
• Single embedding table with 10M categories tested for different embedding widths
• A 64-hot, singular feature with a batch size of 8K
• Embedding table row accesses follow the uniform distribution

Using similar experimental settings that were used in Figure 5, if row accesses are distributed according to a power-law distribution, which essentially means that a few rows are accessed very often and many rows are accessed infrequently, an average speedup of 1.03 is observed.

### 4.2.1.4. Fusing Embedding Tables

In recommender models, categorical features need to be embedded to be fed into the MLP layers.

If several embedding tables have the same width (embedding vector size), it can be beneficial to fuse them to create one larger table. This is because frameworks, such as PyTorch and TensorFlow, can have additional overhead for calling embedding lookup individually. As a result, combining tables when possible, unless there is a good reason not to, will give better efficiency in terms of GPU execution and scheduling.

Take a look at the following pseudocode where two tables, embedding1 and embedding2, with the same width are declared and accessed separately.
# Unfused implementation pseudocode
# declaration
embedding1 = embeddingBag(cardinality_emb_1, vector_dim=128)
embedding2 = embeddingBag(cardinality_emb_2, vector_dim=128)

# Lookup
batch_emb1 = embedding1([list of indices feature 1])
batch_emb2 = embedding2([list of indices feature 2])

As a best practice, fusing the embedding tables to create one larger table leads to better performance.
# Fused implementation pseudocode
# declaration
embedding12 = embeddingBag(cardinality_emb_1+cardinality_emb_2, vector_dim=128)

# Lookup
new_indices_feature2 = [list of indices feature 2] + cardinality_emb_1
lookup_indices = concat([list of indices feature 1] , new_indices_feature2 )
batch_emb = embedding12([lookup_indices])

### 4.2.2. MLP Layers

Fully-connected layers typically follow embedding layers in deep learning recommender models and feature crossing layers in some models. The inputs for MLPs include the feature embeddings and numerical features. The output depends on the type of model being trained. In ranking models, the MLPs output a scalar probability, which indicates relevance of the item to the user. Whereas in embedding models, the output may be a vector, which represents the embedding.

Typically, MLPs in recommender ranking models are quite shallow (between 3-5 layers) and often take the shape of a tower as layers get narrower towards the output. For example, N-2048-1024-512-256-1 where N equals all input features combined. However, it is not uncommon to see other topologies like equal width MLPs in which each layer has the same number of nodes, and research indicates that equal width MLPs perform slightly better.

Larger batch sizes help saturate the available compute (FLOPs). The following batch sizes are recommended for optimum training:
• 8K-16K for large MLPs such as N-2048-1024-512-256-1
• 32K-64K for smaller MLPs
Figure 6. TFlops and Elapsed Time (ms) compared against batch size for four-layer MLPs with different layer sizes

The results depicted in Figure 6 are from a forward pass and MLP layers only, which were tested on an NVIDIA A100 GPU (theoretical peak of 312 dense FP16 TFlops).

In Figure 6, we observe that the elapsed time is roughly flat for the three MLP configurations until the batch size reaches between 2K and 4K. Ranking inference models could work with larger batch sizes without much loss in latency performance in the MLP portion. Ranking models have a higher fidelity than candidate generation models, so keeping everything the same will make it possible to increase the number of candidates generated for a minimal additional cost.

In general, considering both embeddings and MLPs, it can be beneficial to keep the batch size to be 8K or above for optimum training.

### 4.2.4. Sequential and Session-Based Recommendation Models

Traditional recommendation algorithms, such as collaborative filtering, usually ignore the temporal dynamics and the sequence of interactions when trying to model user behavior. However, users' preferences do change over time. Sequential recommendation algorithms are able to capture sequential patterns in the users' browsing that might help to predict the users’ interests for better recommendation. For example, users who are starting a new hobby like cooking or cycling might explore products for beginners, and may move to more advanced products over time. They may also completely move on to another hobby of interest. Therefore, recommending items related to their past preferences would become irrelevant.

A special case of sequential-recommendation is the session-based recommendation task where you only have access to the short sequence of interactions within the current session. This is very common for online services like e-commerce, news, and media portals where the user might be brand new or prefers to browse anonymously, and no cookies are collected as a result of the GDPR compliance. This task is also relevant for scenarios where the user’s interests change a lot over time depending on the user’s context or intent, so leveraging the current session interactions is more promising than old interactions to provide relevant recommendations.

To deal with sequential and session-based recommendation, many sequence learning algorithms previously applied in machine learning and NLP research have been explored for RecSys, such as k-Nearest Neighbors, Frequent Pattern Mining, Hidden Markov Models, Recurrent Neural Networks, and more recently neural architectures using the Self-Attention Mechanism and the Transformer architectures.

In our own research at NVIDIA, we found out that Transformer architectures are able to provide more accurate recommendation for sequential and session-based recommendation. We have developed the Transformers4Rec Library, which works as a bridge between Transformer architectures. For more information, refer to the Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation paper and Transformers4Rec: A Flexible Library for Sequential and Session-based Recommendation article.

Figure 7. Sequential and session-based recommendations with Transformers4Rec

The lessons learned from our extensive empirical analysis that involved benchmarking different algorithms and our recent wins during the following challenges have been applied to the development of the Transformers4Rec Library: Recsys Challenge 2021, WSDM WebTour Challenge 2021 by Booking.com, and SIGIR eCommerce Workshop Challenge 2021 by Coveo. For example, we learned that the XLNet transformer architecture trained with a masked language modeling approach, autoencoding as proposed for BERT, was a top performer across competition and research datasets and would be a good first choice for sequential recommendation problems.

## 5. Frameworks

The best practices referenced here will help you build deep learning recommenders using frameworks such as NVIDIA NVTabular, NVIDIA HugeCTR, PyTorch, and TensorFlow.

### 5.1. NVIDIA NVTabular

NVIDIA NVTabular is a feature engineering and preprocessing library for tabular data. It was designed for quickly and easily manipulating terabyte scale datasets so that deep learning based recommender systems can be trained.

NVIDIA NVTabular provides highly optimized data loaders for training recommender systems in TensorFlow and PyTorch, which reads data directly into the GPU.

The NVIDIA NVTabular data loader is designed to feed tabular data efficiently to deep learning model training using either PyTorch or TensorFlow. In our experiments, we were able to speed up existing TensorFlow pipelines by nine times and existing PyTorch pipelines by five times with our highly-optimized dataloaders. For more information, refer to Accelerated Training with TensorFlow and Accelerated Training with PyTorch.

### 5.1.3. Preferred Storage Format

Parquet is the preferred storage format when using NVIDIA NVTabular. It is a compressed tabular data format that is widely used and can be read by libraries like Pandas and CuDF to quickly search, filter, and manipulate data using high level abstractions. NVIDIA NVTabular can read parquet files very quickly.

### 5.2. NVIDIA HugeCTR

NVIDIA HugeCTR is a high efficiency GPU framework designed for Click-Through-Rate (CTR) estimating training. It provides distributed training with model-parallel embedding tables, embedding cache, and data-parallel neural networks across multiple GPUs and nodes for maximum performance.

### 5.2.2. Embedding Vector Size

The embedding vector size is related to the size of the Cooperative Thread Array (CTA) for the NVIDIA HugeCTR kernel launch. It should not exceed the maximum number of threads per block (1024 for NVIDIA A100 GPU). Configuring the embedding vector size to a multiple of the warp size (32 threads in a warp for NVIDIA A100 GPU) is recommended to improve occupancy.

### 5.2.3. Embedding Performance

NVIDIA HugeCTR embedding supports model parallelism where a gigantic embedding table spans multiple GPUs. It implies that inter-GPU communication exists between embedding layers and dense layers with MLPs. To achieve the best performance, we recommend that a NVSwitch equipped machine, such as DGX A100, be used although NVIDIA HugeCTR also works on machines that use NVIDIA Collective Communications Library (NCCL) without NVSwitch.

### 5.3. PyTorch

PyTorch includes several features that can help accelerate training of deep learning based recommender models on NVIDIA GPUs, such as Automatic Mixed Precision and CUDA Graphs.

### 5.3.1. Using CUDA Graphs

To reduce overhead due to multiple kernel launches, CUDA graphs can be used to cluster a series of CUDA kernels together in a single unit. CUDA graphs were introduced in CUDA 10, and were made available natively in PyTorch v1.10 as a set of beta APIs. For more information about CUDA graphs, refer to the Accelerating PyTorch with CUDA Graphs article, which presents API examples and benchmarks for various models.

The benefits of using CUDA graphs are illustrated in Figure 8 where a CUDA graph is launched in place of a sequence of short kernel launches by the CPU. Initially, a bit more time is used to create and launch the whole graph at once, but overall time is saved due to the minimal gap between kernels.

Figure 8. Time saved by reducing kernel launch overheads with CUDA graphs

In PyTorch, one of the most performant methods to scale out GPU training involves torch.nn.parallel.DistributedDataParallel coupled with the NCCL backend. It applies to both single-node and multi-node distributed training. The CUDA graph benefits mentioned previously also extend to distributed multi-GPU workloads with NCCL kernel launches. Additionally, CUDA graphs establish performance consistency in cases where the kernel launch timings are unpredictable due to various CPU load and operating system factors.

A testament to its performance benefits can be seen in NVIDIA’s MLPerf v1.0 training results where CUDA graphs were instrumental in scaling to over 4000 GPUs, setting new records across the board. For more information, refer to the Global Computer Makers Deliver Breakthrough MLPerf Results with NVIDIA AI blog post.

Benchmark results (refer to Figure 9) for NVIDIA’s implementation of the Deep Learning Recommendation Model (DLRM) model show significant speedups for both training and inference. The effects are more visible with small batch sizes where CPU overheads are more pronounced.

Figure 9. Training and inference performance benefits by using CUDA graphs

If you are programming with CUDA or curious about the underlying principles, the CUDA Graphs section of the programming guide and Getting started with CUDA Graphs article are great resources.

### 5.4. TensorFlow

TensorFlow provides several resources that you can leverage to optimize performance on GPUs. This section contains additional techniques for maximizing deep learning recommender performance on NVIDIA GPUs. For more information about how to profile and improve performance on GPUs, refer to TensorFlow's guide for analyzing and optimizing GPU performance.

### 5.4.1. I/O Pipeline

TFRecord is a format for storing a sequence of binary records that is preferred by TensorFlow and can be read fast with TensorFlow's native data reader.

Deep learning recommender training is frequently limited by the speed of the data loading as compared to the MLP GEMM compute, which tends to be less time consuming in contrast.

TFRecords store field names along with their values for every example in the data. For tabular data where small float or int features have a smaller memory footprint than string field names, the memory footprint of such a representation can get very big really fast.

### 5.4.2. Short Field Names

Considering the memory footprint of string field names in TFRecords, a minor optimization could be to keep the field names really short, keeping to the minimum character limit for unique feature names to help improve optimization. This is situational. However, if the data pipeline is limited by disk read speed and the dataset has many small features, this alone can potentially improve the speedup.

### 5.4.3. Prebatching TFRecords

One way to make data loading faster is by "pre-batching" your data. This means that in each entry (key, value pair) we store a serialized list of values instead to reduce the overhead due to redundant string field names. Therefore, each entry is essentially creating a batch at data preparation time, and we call this "prebatching". The number of values stored in one entry is the "prebatch size".
Figure 10. TFRecord prebatching

As depicted in Figure 10, let’s use a TFRecord with N entries as an example. Instead of storing each value individually, we serialize M such values for each feature and store the list for each entry. By doing this, we store N/M entries and reduce the overhead due to string field names.

We observe significant speedup at data loading time and overall, which is often the bottleneck in training recommender models on GPUs.

This isn’t without limitations though, and may lead to a loss of some flexibility. You may have to add a component in your data engineering pipeline to create this prebatch, as well as your tf.data pipeline to parse it. Additionally, doing so can limit the ability to shuffle the data, and poses a restriction that the prebatch size is the minimum batch size one can train on. These limitations may or may not be significant depending on the use case.

### 5.4.4. Grouping Similar Data Types

Along the same lines as prebatching, another option is to group key-value pairs together for the data type.

For example,suppose your data has three categorical features. Instead of having {cat1: [[0]], cat2: [[5]], cat3: [[4]]}, we can store them as {cat: [[0,5,4]]}. The same applies to numerical values. By doing this, we reduce the overhead to store the keys to two strings (one for numeric and one for categorical). However, this may add overhead to keep track of which category is on which index, and would make it more difficult to address multi-hots.

For more information about building performant data input pipelines for training on GPUs, refer to the Tensorflow Performance Guide for tf.data API.

### 5.4.5. Using XLA Compiler

XLA or Accelerated Linear Algebra is an optimizing compiler for TensorFlow, which supports optimizing code for GPUs. TensorFlow provides documentation to get started with XLA.

NVIDIA’s implementation of the DLRM model also supports XLA JIT compilation. It delivers a steady 10% to 30% performance boost depending on the hardware platform, precision, and the number of GPUs.

### 5.4.6. Optimized embedding layers

TensorFlow native embedding layers, such as tf.nn.embedding_lookup and tf.nn.embedding_lookup_sparse, do not support model parallelism out of the box, such as distributed embeddings over multiple GPUs or nodes. This limits the size of the embedding tables, imposing a restriction that tables must fit within the memory for one GPU.

To solve this problem, the NVIDIA HugeCTR SOK offers an optimized embedding layer implementation for both TensorFlow 2.x and TensorFlow 1.15. The SOK TensorFlow embedding plugin is designed to work conveniently and seamlessly with TensorFlow as a drop-in replacement for the TensorFlow-native embedding layers. It also offers advanced out of the box features, such as model parallelism, that distributes the embedding tables over multiple GPUs. For more information, refer to our blog and SOK documentation.


text:
# Achievements & Publications

### The Formation of Population III Binaries

#### Saigo, Kazuya,   Matsumoto, Tomoaki,   Umemura, Masayuki

We explore the possibility for the formation of Population III binaries. The collapse of a rotating cylinder is simulated with a three- dimensional, high-resolution nested grid, assuming the thermal history of primordial gas. The simulations are done with dimensionless units, and the results are applicable to low-mass as well as...

### Gas stripping by radiation drag from an interstellar cloud

#### Sato, Jun'ichi,   Umemura, Masayuki,   Sawada, Keisuke,   Matsuyama, Shingo

We perform two-dimensional hydrodynamic simulation of gas stripping by radiation drag from interstellar clouds moving in uniform radiation fields. To include relativistic radiation drag properly, the radiation hydrodynamic equation is solved taking into account the dilution of radiation fields by the optical depth of the cloud. As a result, it...

### A Strong Ly\ensuremathα Emitter at z = 6.33 in the Subaru Deep Field Selected as an i'-Dropout

#### Nagao, T.,   Taniguchi, Y.,   Kashikawa, N.,   Kodaira, K.,   Kaifu, N.,   Ando, H.

We report on the discovery of a star-forming galaxy at z=6.33 in the Subaru Deep Field. This object is selected as a candidate of an i$^′$-dropout high-redshift galaxy around z=6 because of its red i$^′$-z$^′$ color in our deep optical imaging survey in the Subaru Deep Field. Our follow-up optical...

### The Effects of Early Cosmic Reionization on the Substructure Problem in Galactic Halos

#### Susa, Hajime,   Umemura, Masayuki

Recent observations of the cosmic microwave background by the Wilkinson Microwave Anisotropy Probe strongly suggest that the reionization of the universe took place quite early (zåisebox-0.5ex~17). On the other hand, it has been pointed out that the cold dark matter cosmology suffers from a substructure problem in which more subgalactic...

### The Formation of Population III Binaries

#### Saigo, Kazuya,   Matsumoto, Tomoaki,   Umemura, Masayuki

We explore the possibility for the formation of Population III binaries. The collapse of a rotating cylinder is simulated with a three- dimensional, high-resolution nested grid, assuming the thermal history of primordial gas. The simulations are done with dimensionless units, and the results are applicable to low-mass as well as...

November 2004   The Astrophysical Journal Letters

### Gas stripping by radiation drag from an interstellar cloud

#### Sato, Jun'ichi,   Umemura, Masayuki,   Sawada, Keisuke,   Matsuyama, Shingo

We perform two-dimensional hydrodynamic simulation of gas stripping by radiation drag from interstellar clouds moving in uniform radiation fields. To include relativistic radiation drag properly, the radiation hydrodynamic equation is solved taking into account the dilution of radiation fields by the optical depth of the cloud. As a result, it...

October 2004   Monthly Notices of the Royal Astronomical Society

### A Strong Ly\ensuremathα Emitter at z = 6.33 in the Subaru Deep Field Selected as an i'-Dropout

#### Nagao, T.,   Taniguchi, Y.,   Kashikawa, N.,   Kodaira, K.,   Kaifu, N.,   Ando, H.

We report on the discovery of a star-forming galaxy at z=6.33 in the Subaru Deep Field. This object is selected as a candidate of an i$^′$-dropout high-redshift galaxy around z=6 because of its red i$^′$-z$^′$ color in our deep optical imaging survey in the Subaru Deep Field. Our follow-up optical...

September 2004   The Astrophysical Journal Letters

### The Effects of Early Cosmic Reionization on the Substructure Problem in Galactic Halos

#### Susa, Hajime,   Umemura, Masayuki

Recent observations of the cosmic microwave background by the Wilkinson Microwave Anisotropy Probe strongly suggest that the reionization of the universe took place quite early (zåisebox-0.5ex~17). On the other hand, it has been pointed out that the cold dark matter cosmology suffers from a substructure problem in which more subgalactic...

July 2004   The Astrophysical Journal Letters


text:
### Symmetry, Integrability and Geometry: Methods and Applications (SIGMA)

SIGMA 8 (2012), 039, 17 pages      arXiv:1204.0254      https://doi.org/10.3842/SIGMA.2012.039

### Some Remarks on Very-Well-Poised ${}_8\phi_7$ Series

Jasper V. Stokman
Korteweg-de Vries Institute for Mathematics, University of Amsterdam, Science Park 904, 1098 XH Amsterdam, The Netherlands

Received April 05, 2012, in final form June 18, 2012; Published online June 27, 2012

Abstract
Nonpolynomial basic hypergeometric eigenfunctions of the Askey-Wilson second order difference operator are known to be expressible as very-well-poised ${}_8\phi_7$ series. In this paper we use this fact to derive various basic hypergeometric and theta function identities. We relate most of them to identities from the existing literature on basic hypergeometric series. This leads for example to a new derivation of a known quadratic transformation formula for very-well-poised ${}_8\phi_7$ series. We also provide a link to Chalykh's theory on (rank one, BC type) Baker-Akhiezer functions.

Key words: very-well-poised basic hypergeometric series; Askey-Wilson functions; quadratic transformation formulas; theta functions.

pdf (450 kb)   tex (23 kb)

References

1. Askey R., Wilson J., Some basic hypergeometric orthogonal polynomials that generalize Jacobi polynomials, Mem. Amer. Math. Soc. 54 (1985), no. 319.
2. van de Bult F.J., Ruijsenaars' hypergeometric function and the modular double of ${\mathcal U}_q(\mathfrak{sl}_2({\mathbb C}))$, Adv. Math. 204 (2006), 539-571, math.QA/0501405.
3. van de Bult F.J., Rains E.M., Stokman J.V., Properties of generalized univariate hypergeometric functions, Comm. Math. Phys. 275 (2007), 37-95, math.CA/0607250.
4. Chalykh O.A., Macdonald polynomials and algebraic integrability, Adv. Math. 166 (2002), 193-259, math.QA/0212313.
5. Chalykh O.A., Etingof P., Orthogonality relations and Cherednik identities for multivariable Baker-Akhiezer functions, arXiv:1111.0515.
6. Gasper G., Rahman M., Basic hypergeometric series, Encyclopedia of Mathematics and its Applications, Vol. 96, 2nd ed., Cambridge University Press, Cambridge, 2004.
7. Gupta D.P., Masson D.R., Contiguous relations, continued fractions and orthogonality, Trans. Amer. Math. Soc. 350 (1998), 769-808, math.CA/9511218.
8. Haine L., Iliev P., Askey-Wilson type functions with bound states, Ramanujan J. 11 (2006), 285-329, math.QA/0203136.
9. Ismail M.E.H., Rahman M., The associated Askey-Wilson polynomials, Trans. Amer. Math. Soc. 328 (1991), 201-237.
10. Koelink E., Stokman J.V., The Askey-Wilson function transform, Int. Math. Res. Not. (2001), no. 22, 1203-1227, math.CA/0004053.
11. Koelink E., Stokman J.V., Fourier transforms on the quantum ${\rm SU}(1,1)$ group, Publ. Res. Inst. Math. Sci. 37 (2001), 621-715, math.QA/9911163.
12. Koornwinder T., Comment on the paper "Macdonald polynomials and algebraic integrability" by O.A. Chalykh, available at http://staff.science.uva.nl/~thk/art/comment/ChalykhComment.pdf.
13. Letzter G., Stokman J.V., Macdonald difference operators and Harish-Chandra series, Proc. Lond. Math. Soc. (3) 97 (2008), 60-96, math.QA/0701218.
14. van Meer M., Bispectral quantum Knizhnik-Zamolodchikov equations for arbitrary root systems, Selecta Math. (N.S.) 17 (2011), 183-221, arXiv:0912.3784.
15. van Meer M., Stokman J., Double affine Hecke algebras and bispectral quantum Knizhnik-Zamolodchikov equations, Int. Math. Res. Not. (2010), no. 6, 969-1040, arXiv:0812.1005.
16. Noumi M., Stokman J.V., Askey-Wilson polynomials: an affine Hecke algebra approach, in Laredo Lectures on Orthogonal Polynomials and Special Functions, Adv. Theory Spec. Funct. Orthogonal Polynomials, Nova Sci. Publ., Hauppauge, NY, 2004, 111-144, math.QA/0001033.
17. Rahman M., The linearization of the product of continuous $q$-Jacobi polynomials, Canad. J. Math. 33 (1981), 961-987.
18. Rahman M., Verma A., Quadratic transformation formulas for basic hypergeometric series, Trans. Amer. Math. Soc. 335 (1993), 277-302.
19. Ruijsenaars S.N.M., A generalized hypergeometric function satisfying four analytic difference equations of Askey-Wilson type, Comm. Math. Phys. 206 (1999), 639-690.
20. Ruijsenaars S.N.M., Quadratic transformations for a function that generalizes ${}_2F_1$ and the Askey-Wilson polynomials, Ramanujan J. 13 (2007), 339-364.
21. Sauloy J., Systèmes aux $q$-différences singuliers réguliers: classification, matrice de connexion et monodromie, Ann. Inst. Fourier (Grenoble) 50 (2000), 1021-1071.
22. Singh V.N., The basic analogues of identities of the Cayley-Orr type, J. London Math. Soc. 34 (1959), 15-22.
23. Slater L.J., A note on equivalent product theorems, Math. Gaz. 38 (1954), 127-128.
24. Stokman J.V., An expansion formula for the Askey-Wilson function, J. Approx. Theory 114 (2002), 308-342, math.CA/0105093.
25. Stokman J.V., The $c$-function expansion of a basic hypergeometric function associated to root systems, arXiv:1109.0613.
26. Suslov S.K., Some orthogonal very-well-poised ${}_8\phi_7$-functions that generalize Askey-Wilson polynomials, Ramanujan J. 5 (2001), 183-218, math.CA/9707213.


text:
# How to check whether the production function below is neoclassical or not

Say I'm given the production function of the form

\begin{equation*} Y= F(K,L,A) = (\alpha K^{\epsilon} + (1 - \alpha)Y^{\epsilon})^{\frac{1}{\epsilon}} \end{equation*}

Where $\epsilon, \alpha \in (0,1)$.

Now I need to check three things:

1. That marginal products are positive and they are decreasing
3. Essentiality
4. Linear homogeneity

The conditions 2 and 4 imply the condition 3 so no need to check for that. When taking derivatives with respect to $K$ I see that

\begin{equation*} F_K = \alpha K^{\epsilon -1}(\alpha K^{\epsilon} + (1 - \alpha)Y^{\epsilon})^{\frac{1 - \epsilon}{\epsilon}} >0 \end{equation*}

Which is positive. Now I need to show that $F_{KK}< 0$. See:

\begin{equation*} F_{KK} = \alpha (\epsilon -1) K^{\epsilon -2}(\alpha K^{\epsilon} + (1 - \alpha)Y^{\epsilon})^{\frac{1 - \epsilon}{\epsilon}} + \alpha^2 K^{2\epsilon -2} \frac{1- \epsilon}{\epsilon} (\alpha K^{\epsilon} + (1- \alpha) L^{\epsilon})^{\frac{1}{\epsilon}-2} \end{equation*}

However I couldn't find a way to decide on the sign of this expression. Any help would be appreciated. Thanks.

• Have you tried simplifying it? There are a lot of common factors in this sum, all of which have positive signs. E.g.: $\alpha$ appears in both. Apr 2, 2018 at 16:02
• @denesp I couldn't simplify it that's why it is on here Apr 2, 2018 at 16:16
• It appears there is a typo, you write $Y$ in the right-hand-side instead of $L$ ? (on three different occasions) Apr 2, 2018 at 19:04

C.E.S production functions are indeed tiresome to differentiate, and it is very easy to make a mistake. But hey, this (algebraic manipulation) is part of the job, and one needs to practice moving around symbols , creating common factors, etc.

But there is another way to show that $F_{KK} <0$ in totally abstract terms.

First, prove 4. (linear homogeneity), it is easy.

Second, exploit Euler's Homogeneous Function Theorem, and decompose $Y$ using the first derivatives and the variables (this is a very well known result).

Third, re-arrange the previous to obtain an alternative abstract expression of the first derivative of say, capital.

Fourth, differentiate what you got in step Three, to obtain an expression for $F_{KK}$.

All the above are done using abstract notation $F_K, F_{KK}$,etc and nowhere the actual C.E.S functional form. After step Four, you will need to calculate the cross-partial, which is easy, even for the C.E.S. function.


text:
# Autonomous system of 2 non-linear differential equations

Hello I am trying to solve this system: $$\alpha '(t)= -\frac{e^{-3 \alpha (t)} \sin (2 \mu p_\alpha(t))}{\mu} \tag{1}\label{1}$$ $${ p_\alpha}'(t)= 3 \rho _0^2 (w-1) e^{-3 w \alpha (t)} \tag{2}\label{2}$$ Where $\alpha(t)$ and $p_\alpha(t)$ are the two functions of time that I am trying to calculate. Actually I am interested only in $\alpha(t)$. $\rho_0,w,\mu$ are constant parameters where $\rho_0>0$, $-1<w<1$ and $\mu>0$ is a small perturbation.

As you might have already guessed these are nothing else than the Hamilton's equations of a dynamical system endowed with an Hamiltonian. The Hamiltonian of the system, modulo a constant, is:

$$\mathcal{H}= e^{-3\alpha(t)}\left(p_{-}^2+p_{+}^2-\frac{\sin(\mu p_\alpha(t))^2}{\mu^2}+\rho_0^2 e^{-3(w-1)\alpha(t)}\right)=0$$ Since we are dealing with general relativity the Hamiltonian is null (frozen time formalism).

Anyway I am able to solve the system in the limit $\mu\to0$ and the result is correct (I know for sure).

This is the procedure I follow: I divide \eqref{2} over \eqref{1} to find $\frac{d p_\alpha}{d\alpha}$. I integrate it to find $p_\alpha(\alpha)$ that I then substitute in \eqref{1}. Now I have only an ODE in one variable that I can integrate to find $\alpha(t)$.

As I said in the unperturbed case ($\mu\to0$) this procedure gives me the correct result but for $\mu\neq0$ the very same procedure gives me nonsense.

I am using Wolfram Mathematica to do the calculations, but I suspect the problem is not the code that I tried to manipulate in every conceivable (to me at least) way.

Every kind of advice is welcomed. I don't expect you to work out the solution for me. I only would like to know how would you tackle the problem. Thank you

• To me your approach doesn't have a flaw. What kind of non-sense this procedure gives to you? Can you describe it more explicitly? – Evgeny Oct 10 '17 at 22:15
• First of all it is physically nonsense. The system I am studying is a semiclassical cosmological model. Based on calculations made be a colleague of mine who I trust, I expect a certain behavior in function of $w$. To be more precise for $-1<w\leq1/3$ I expect a singularity and for $1/3<w<1$ I expect a bounce. Moreover it is mathematically "fishy", since if I let Mathematica solve the system it gives me a solution. If I solve the system with the aforesaid procedure I obtain another solution (both of them not expressible in closed form). – LastStarDust Oct 11 '17 at 7:02
• With "my" solution I have always a singularity and with Mathematica solution I always have a bounce (actually I have a Universe whose volume behaves like a sinus that is strange indeed). – LastStarDust Oct 11 '17 at 7:09
• Well, I have plenty of questions, some of them a bit stupid and paranoic, but I still have to ask them. 1) I struggle to see that this system is Hamiltonian. Is it written in non-canonical variables? All usual necessary conditions for being Hamiltonian failed here (equality of mixed derivatives, zero divergence test) 2) Does your hand-written and Wolfram solutions are really solutions when you plug them in? 3) Can we reformulate "bounce" and "singularity" in terms of phase portrait? In that case we can do bifurcation analysis — your system looks pretty tame, it's analytic, a lot of good... – Evgeny Oct 11 '17 at 7:42
• ... theory can be applied. I've seen few papers on cosmology where people did this kind of analysis and it might work in your case too. – Evgeny Oct 11 '17 at 7:43

This is a part of the chat discussion that is too long to be put in chat. And chat doesn't render math.

In theory I might be doing mistake in my calculations, but if I hadn't then the qualitative conclusion about this system is the same as yours about cyclic change of Universe's volume. Actually, all trajectories of this system are periodic. I would be very-very glad if someone could re-check my calculations!

In principle this system could be solved, but I've decided to study its Poincaré map. It actually requires less calculations and these calculations are very simple. Because the system is periodic in $p_{\alpha}$ the phase space is a cylinder actually. This is due to the fact that the system posesses a discrete translation symmetry: if $(\xi(t), \chi(t))$ is a solution then $(\xi(t), \chi(t) + 2\pi k)$ is also a solution for any $k \in \mathbb{Z}$. This system has no equilibria so its trajectories are likely to wind up around the cylinder. Note that if you take a line $p_{\alpha} = \frac{\pi k}{\mu}$ the vector field is strictly orthogonal to it — it's one of the nullclines and a good candidate for a Poincaré section. Thus we can represent cylinder as a strip $- \frac{\pi}{\mu} \leqslant p_{\alpha} \leqslant 0$ with glued lines $p_{\alpha} = 0$ and $p_{\alpha} = -\frac{\pi}{\mu}$.

The reason why I've chosen exactly this strip is the following. Note that $p'_{\alpha} < 0$ everywhere because of constraints on values of parameters. If we start at $p_{\alpha} = 0$ we go below until we hit $p_{\alpha} = -\frac{\pi}{\mu}$. Basically this would be our Poincaré map $\mathcal{F}$: start from $(\alpha, 0)$, go with the flow until you hit $p_{\alpha} = -\frac{\pi}{\mu}$ at point $(\mathcal{F}(\alpha), -\frac{\pi}{\mu})$. The good thing is that in your case the Poincaré map can be explicitly computed.

How will we do it? We won't write the solution of the system, solve for arrival time and plug it into the solution. Instead we will use equation

$$\frac{d \alpha}{d p_{\alpha}}= -\frac{e^{-3 \alpha} \sin (2 \mu p_\alpha)}{3\mu \rho^2_0 (w-1) e^{-3w\alpha}}.$$

What is it good for? Why we can use it? Well, $t$ changes strictly monotonously, $p_{\alpha}$ changes in the same fashion (and derivative is never zero). By some sort of inverse theorem we can now consider $p_{\alpha}$ as our new independent variable. This is very convenient: the equation is separable and Poincaré map can be written simply as a solution of some Cauchy problem. Let's do it: if trajectory starts at $(u, 0)$ and goes to $(\mathcal{F}(u), -\frac{\pi}{\mu})$, then the solution is

$$-3(w-1) e^{-3\alpha(w-1)} \,d\alpha = \frac{\sin{(2\mu p_{\alpha})}}{\mu\rho^2_0}\, d p_{\alpha}$$

$$\int\limits_{u}^{\mathcal{F}(u)}-3(w-1) e^{-3\alpha(w-1)} \,d\alpha = \int\limits_{0}^{-\frac{\pi}{\mu}}\frac{\sin{(2\mu p_{\alpha})}}{\mu\rho^2_0}\, d p_{\alpha}$$

$$\left . \left (e^{-3\alpha (w-1)} \right ) \right \vert^{\mathcal{F}(u)}_{u} = \left . \left ( - \frac{\cos{2 \mu p_{\alpha}}}{2\rho^2_0 \mu^2} \right ) \right \vert^{-\frac{\pi}{\mu}}_{0}$$

The function in the RHS is $\frac{\pi}{\mu}$-periodic and we take the integral over the period so it's zero (the same conclusion follows from direct substitution) and we arrive at equation

$$e^{-3\mathcal{F}(u) (w-1)} - e^{-3{u} (w-1)} = 0$$

From this we get that $\mathcal{F}(u) = u$ which means that every solution is periodic. It doesn't necessarily mean that solution $\alpha(p_{\alpha})$ is constant — no, we can check this directly and it's not so. All solutions are just some closed curves that wrap around the cylinder. They don't necessarily have the same period w.r.t. to time $t$ (although they have the same period w.r.t. $p_{\alpha}$ when it's considered as independent variable). But alas, this map is an identity and doesn't have a good non-trivial dynamics which could be in accordance with your colleague's observations.

• Ok I studied your answer. It seems all legit. I am really convinced that this is indeed the behavior of the system under study. So I am faced with two options: 1) there is some basic difference between my assumptions and the assumptions of my colleaque. 2) I have made some mistakes in writing the Hamiltonian. But since I have the expected behavior in the classical case I really doubt that 2) is the case. I will discuss with my mentor this issue (I am meeting him just now). Thank you very much for all your help. I am really moved. I will keep you updated. – LastStarDust Oct 13 '17 at 8:53
• Let's see what your advisor would say about our findings :) I'm pretty interested in coping with such models. Always wondered about peculiar applications of this kind of knowledge :) – Evgeny Oct 13 '17 at 10:03
• Hello Eugevny! I have written to you in the chat. If you haven't already, please take a look! Giorgio – LastStarDust Nov 22 '17 at 12:44


text:
# Exploring IBM HR data using Python

## Intro

This tutorial series explores the IBM HR data set. This data is typically used to demonstrate the ability of various machine learning algorithms applied to HR data.

In this series, I'll use it to demonstrate the awesome power Python can bring to HR data

Sections

• Statistics
• Matplotlib
• Pandas
• Seaborn
• Plotly
• Findings
In [1]:
__author__ = "adam"
__version__ = "1.0.0"

In [2]:
# imports
import numpy as np
import pandas as pd

import matplotlib.pyplot as plt

%matplotlib inline

In [3]:
# if continuing on from the previous section, read the data from saved file

In [4]:
# read the data directly from IBM Watson Analytics
# using pandas read excel file into dataframe

# save data for later
# empl_data.to_excel("WA_Fn-UseC_-HR-Employee-Attrition.xlsx")

### pandas

Pandas? We've already looked at pandas. Yes, but we've not explored the plotting capabilities of pandas.

This section, let's explore Gender.

In [5]:
empl_data['Gender'].value_counts(normalize=True).plot(kind='bar', title='Employee Gender')

Out[5]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a422d30>

~60% of the employees are Male, ~40% Female.

#### Salary</mh4>

In [6]:
empl_data.pivot_table(values='HourlyRate', columns='Gender', aggfunc='mean')\
.plot(kind='bar', title='Average Hourly Salary by Gender and Job Level')

Out[6]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a593c88>

Positively, there does not appear to be any variance in the overall average pay between gender.

Let's explore further

In [7]:
# by job level
empl_data.pivot_table(values='HourlyRate',index='JobLevel', columns='Gender', aggfunc='mean')\
.plot(kind='bar', title='Average Hourly Salary by Gender and Job Level')

Out[7]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a858668>

Again, no significant findings. Let's look at one more...

In [8]:
# by department
empl_data.pivot_table(values='HourlyRate',index='Department', columns='Gender', aggfunc='mean')\
.plot(kind='bar', title='Average Hourly Salary by Gender and Department')

Out[8]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a8bb470>

Here, it would initially appear that within Sales there is a a significant pay gap for Female employees.

#### Performance

In [9]:
empl_data.pivot_table(values='PerformanceRating', columns='Gender', aggfunc='mean')\
.plot(kind='bar', title='Average Performance Rating by Gender and Job Level')

Out[9]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9b90fe48>

Performance also appears equal between Genders. What about by Job Level?

In [10]:
empl_data.pivot_table(values='PerformanceRating',index='JobLevel', columns='Gender', aggfunc='mean')\
.plot(kind='bar', title='Average Performance Rating by Gender and Job Level')

Out[10]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a3db978>

And now, what about by Department? Will we see the same as with Average Salary?

In [11]:
empl_data.pivot_table(values='PerformanceRating',index='Department', columns='Gender', aggfunc='mean')\
.plot(kind='bar', title='Average Performance Rating by Gender and Department')

Out[11]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a3daf98>

Average Performance appears equal - even in Sales; this does not explain the salary difference between genders.

In [12]:
empl_data.columns

Out[12]:
Index(['Age', 'Attrition', 'BusinessTravel', 'DailyRate', 'Department',
'DistanceFromHome', 'Education', 'EducationField', 'EmployeeCount',
'EmployeeNumber', 'EnvironmentSatisfaction', 'Gender', 'HourlyRate',
'JobInvolvement', 'JobLevel', 'JobRole', 'JobSatisfaction',
'MaritalStatus', 'MonthlyIncome', 'MonthlyRate', 'NumCompaniesWorked',
'Over18', 'OverTime', 'PercentSalaryHike', 'PerformanceRating',
'RelationshipSatisfaction', 'StandardHours', 'StockOptionLevel',
'TotalWorkingYears', 'TrainingTimesLastYear', 'WorkLifeBalance',
'YearsWithCurrManager'],
dtype='object')
In [13]:
empl_data.pivot_table(values='HourlyRate',index='PerformanceRating', columns='Gender', aggfunc='mean')\
.plot.barh(title="Average Performance Rating by Gender")

Out[13]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9a321550>

With pandas, we can even strip away some of the transformations and call methods directly on the DataFrame.

In [14]:
empl_data.boxplot(column='HourlyRate', by='Gender', grid=False)

Out[14]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9b95ad30>
In [16]:
# suppress warning message from matplotlib
import warnings; warnings.simplefilter('ignore')

In [17]:
empl_data[empl_data['Gender'] == 'Male']['HourlyRate'].plot.hist(by='Gender', alpha=0.5, normed=True)
empl_data[empl_data['Gender'] == 'Female']['HourlyRate'].plot.hist(by='Gender', alpha=0.5, normed=True)

Out[17]:
<matplotlib.axes._subplots.AxesSubplot at 0x26e9b99fd68>

### Section Findings

Having explored Education, we learned:

• Compensation does not vary between genders significantly.
• Compensation variance is not noted across Job Level nor Department.
• Performance Ratings generally appear similar across gender.

### Pandas Recap

Pandas plotting is powerful; you have the ability to plot directly from your DataFrames.

Here's the secret - pandas doesn't do any plotting. As was mentioned in the previous section, matplotlib, other packages build on top of matplotlib. pandas is no exception. In fact, the .plot method is just a wrapper around matplotlib calls.

Still, this can be more effective that calling matplotlib directly. When working with DataFrames, it's easy to transform the data and pass to .plot matplotlib via the wrapper methods.


text:
# Complex Integration - Poles on the Imaginary axis

1. Apr 27, 2012

### knowlewj01

1. The problem statement, all variables and given/known data

evaluate the integral:
$I_1 =\int_0^\infty \frac{dx}{x^2 + 1}$

by integrating around a semicircle in the upper half of the complex plane.

2. Relevant equations

3. The attempt at a solution

first i exchange the real vaiable x with a complex variable z & factorize the denominator. Also, the contour of integration is a semicircle with radius= infinity

$I_2 = \int_{-\infty}^{\infty}\frac{dz}{(z+i)(z-i)}$

the contour contains only the pole in the upper half, so from residue theorem we know:
$I_2 = 2\pi i R(i)$

where R(i) is the residue at the point z=i

R(i) = 1/(i+i) = 1/2i

Hence $I_2 = \pi$

Now, i know the answer to the original integral is supposed to be pi/2.
Can i say that: because the original limits range from 0 to infinity, and i have integrated twice this amount, my answer should be divided by 2? or is this reasoning flawed?

2. Apr 27, 2012

### Whovian

I THINK you do divide by two, as noted, but how I'd approach this would be by using trig substitution, substituting x = tan(θ). :P But yep, I think your reasoning's right, or at least close!

3. Apr 27, 2012

### jackmell

4. Apr 28, 2012

### knowlewj01

Thanks for the replies. Is this because the function is even in the upper half of the complex plane?

I thought of doing this by integrating a contour in only the positive quadrent, ie:
(0,0) to (R,0)
(R,0) to (0,iR) along contour ω [a radial path of radius R from the real axis to the imaginary axis in the positive quadrent]
(0,iR) to (0,i+δ) [where δ is a small value which we will take to 0 around the pole]
(0,i+δ) to (0,i-δ) along a contour λ [radial path of radius δ in the clockwise direction such that the pole is not included in the enclosed path]
(0,i-δ) to (0,0)

we have:

$F(z) = \frac{1}{Z^2+1}=\frac{1}{(z+i)(z-i)}$

$\oint_C F(z)dz =\int_0^R F(z)dz + \int_\omega F(z)dz + \left[\int_{iR}^{i+\delta}F(z)dz+\int_{i-\delta}^0 F(z)dz\right] + \int_\lambda F(z)dz =0$

as F(z) is analytic everywhere inside the contour we demand that the integral = 0 by Green's theorem.

we know from residue theorem that integration on the contour λ will give a contribution of -πiR(i) as it is counterclockwise (& where R(i) is the residue at i, which is 1/2i)

now let R→∞ and δ→0:

First: consider the integral along the contour ω, if we have $z=Re^{i\theta}$

$\int_\omega F(z)dz = \int_\omega \frac{iRe^{i\theta}d\theta}{R^2e^{i2\theta} + 1}$

$\lim_{R \to \infty}\int_\omega \frac{iRe^{i\theta}d\theta}{R^2e^{i2\theta}+1}≈ \frac{1}{R}\int_\omega \frac{id\theta}{e^{i\theta}}=0$

Is this correct? I thought Jordan's Lemma worked only for semicircles but this seems to give the same result.

we are left with:

$\int_0^\infty F(z)dz + \int_{i\infty}^0 F(z)dz - \frac{\pi}{2} = 0$

so i know that the integral along the imaginary axis must be 0 but i'm not sure how to prove it


text:
2017
Том 69
№ 9

# Nikmehr M. J.

Articles: 4
Article (English)

### Strongly Semicommutative Rings Relative to a Monoid

Ukr. Mat. Zh. - 2014. - 66, № 11. - pp. 1528–1539

For a monoid M, we introduce strongly M-semicommutative rings obtained as a generalization of strongly semicommutative rings and investigate their properties. We show that if G is a finitely generated Abelian group, then G is torsion free if and only if there exists a ring R with |R| ≥ 2 such that R is strongly G-semicommutative.

Article (English)

### Generalized Semicommutative and Skew Armendariz Ideals

Ukr. Mat. Zh. - 2014. - 66, № 9. - pp. 1213–1222

We generalize the concepts of semicommutative, skew Armendariz, Abelian, reduced, and symmetric left ideals and study the relationships between these concepts.

Article (English)

### Weak $\alpha$-skew Armendariz ideal

Ukr. Mat. Zh. - 2012. - 64, № 3. - pp. 404-414

We introduce the concept of weak $\alpha$-skew Armendariz ideals and investigate their properties. Moreover, we prove that $I$ is a weak $\alpha$-skew Armendariz ideal if and only if $I[x]$ is a weak $\alpha$-skew Armendariz ideal. As a consequence, we show that $R$ is a weak $\alpha$-skew Armendariz ring if and only if $R[x]$ is a weak $\alpha$-skew Armendariz ring.

Article (English)

### Properties of a certain product of submodules

Ukr. Mat. Zh. - 2011. - 63, № 4. - pp. 502-512

Let $R$ be a commutative ring with identity, $M$ an $R$-module and $K_1,..., K_n$ submodules of $M$. In this article, we construct an algebraic object, called product of $K_1,..., K_n$. We equipped this structure with appropriate operations to get an $R(M)$-module. It is shown that $R(M)$-module $M^n = M... M$ and $R$-module $M$ inherit some of the most important properties of each other. For example, we show that $M$ is a projective (flat) $R$-module if and only if $M^n$ is a projective (flat) $R(M)$-module.


text:
# Submission Creation Requests (archived)

+ 6 like - 0 dislike
10304 views

For users without direct submission privileges. Please submit a paper from the Submit Paper page.

closed as a duplicate of: Submission Creation Requests
closed Jul 14, 2015

+ 0 like - 0 dislike

[TEMPORARILY ON HOLD] (see comments and Q&A)

Hello,

## Bootstrapping Generations

Authors: Alejandro Rivero

A supersymmetric version of Chew's "democratic bootstrap" argument predicts the existence of three generations of particles, with one quark, of type "up", more massive that the other five.

It is a condensation of the arxiv papers http://arxiv.org/abs/hep-ph/0512065, http://arxiv.org/abs/0710.1526 and http://arxiv.org/abs/0910.4793.

EDIT: The point is, given a SUSY model, ask for "nuclear democracy" in the old sense of every particle can be built as a composite of others. The paper then studies the particular case of building the scalar sector from the particles in the fermionic sector.

answered Aug 28, 2014 by (260 points)
reshown Jul 14, 2015

I am afraid you will not get very kind review, as the paper misunderstands the nature of the bootstrap condition, and substitutes a strange new thing that should not be associated to this name. If you wish to get a kinder review, you could ask about the subject as a question on physics.stackexchange: "Does it make sense to postulate an equal number of quark states and quark-quark bound states?" I would prefer if we do not import this submission, because you will simply get hostile comments regarding the misunderstanding of the bootstrap, and regarding the incompatibility of the boson-fermion counting equality you postulate with the standard model, or any supersymmetric extension of it.

I am afraid that such question would get even more hostile comments because without reading of the paper the answer seems obviously "no".  And in any case,  I would hope that the review will also get some constructive comment. The result is interesting and I think that some effort should be done to save it.

But about the meaning of "democratic" and "bootstrap" in Chew's work, I agree that it could be useful to do a generic question. It seems that Chew keep changing the use of the word as the research evolved.

EDIT: in fact, the formulation of your question is already a hint to improve the paper. It could be useful to decompose the question in two separate objections: "has it sense to postulate that some elementary particles can be seen as composites of other ones?" and "has it sense to request the same number of bosonic and fermionic degrees of freedom in a theory"?

The question you are asking in the request is a really good one: is it possible for a supersymmetry to link fundamental and composite particles? To link constituents to bound-states? This question will be well received here for sure. Also, I don't know the answer. You should ask on Q&A, and if nobody knows for a long time, it might be an open question (I don't think it is open, but maybe) Your submission will needlessly generate negative reviews for what is essentially a preliminary effort to find an answer to a very nice question.

@arivero: We are still in beta, and we are trying to attract professionals. Papers that are too distant from professional interest will dissuade professionals from joining. I am asking you not to please not push the matter, your paper can get a review eventually if you insist, but it will be a negative one, and you will have a negative reputation, and then you will lose your voting rights.

When you ask a paper to go in reviews, it is expected to contain original results, while your paper contains only state counting which can be reproduced quickly by others. Such state-counting can be done in the body of a Q&A question, you can just write a sketch and people will understand what you mean.

If you had a real model of how to do what you want, it would be appropriate for reviews, but there is no full model in there, nor a sketch of one. This is why it would get negative reviews, and this would be a pity for such a nice question. If you stick to Q&A, you will get positive responses for sure.

I did not miss the logic of the paper--- your observation is original, but it is not useful absent a model which can do something with bound states and fundamental states.

For specific problems (there are infinitely many problems with such logic) There are misunderstandings there about the state counting: for example, you have 3 colors of quarks, but the bound states are only colorless, you have colorless bound states of three quarks, there are no bound states of neutrinos, etc, so that the actual method by which you get your "three generations" is completely incorrect.

But the general question is ok--- can you mix bound states and fundamental states with SUSY? That's a great question, and it will get upvotes. I'm just giving political advice here. Your paper will not be reviewed positively, but your general idea of mixing composite and elementary particles using supersymmetry is interesting.

Regarding the status of your paper, perhaps "In discussion" is better than frozen. At the end of the process, we would like to have a professional submission to review, and for you to get positive feedback. It's better than a superficial review which is just pointing out big gaps and ignoring the main idea.

Definitely I need to improve my communication skills. From the paper:

... is constituted by a string terminated in a pair of quarks.
With this imaginery, we can try a weaker version of the democratic bootstrap.

I read this (actually, I wrote this) as telling that each scalar composite contains two quarks. From this all the combinations follow. It is not the case that I am cherry picking the quarks. (EDIT: but is is true that I am implicitly assuming that for each combination we have only one scalar. I should be more explicit on this point).

With this ansatz I show that there is no solution. So then I relax, setting that only $r,s$ quarks can be attached to the string:

We postulate thus that only a subset of ”light quarks” are in the terminations of the string and then able to form composites.

(EDIT: ok, I think that the indication "a subset" can be the origin of the confusion. I do not choose randomly what quarks are allowed to play, but instead I explore all the space of possibilities: I set two variables s,r for the number of up-type and down-type quarks that are allowed to be, say, at the end of the string).

And then I show, by solving the equations for r,s and the total number N of generations, that there is a unique solution, s=2, r=3, N=3

The paper you describe is not my paper.

(EDIT: I will wrote and upload a v2 addressing the problems you mention).

The question you are asking in the request is a really good one: is it possible for a supersymmetry to link fundamental and composite particles? To link constituents to bound-states? This question will be well received here for sure

@Ron what is your definition of "well received"? A first comment about doing off-topic remarks does not seem a good reception to me.


text:
# Conformations and Mitochondrial Uncoupling Activity of Synthetic Emerimicin Fragments

Raj, Antony P and Das, Manoj K and Balaram, P (1988) Conformations and Mitochondrial Uncoupling Activity of Synthetic Emerimicin Fragments. In: Biopolymers, 27 (4). pp. 683-701.

 Preview
PDF
activity.pdf

Several amino terminal fragments of the emerimicins $(Ac-{Phe}^1-{Aib}^2-{Aib}^3-{Aib}4-{Va1}^5-{Gly}^6-{Leu}^7-{Aib}^8-{Aib}^9-{Hyp}^{10}-{Gln}^{11}-D-{Iva}^{12}-{Hyp}^{13}-Ala/{Aib}^{14}-{Phol}^{15})$ ranging in length from five to ten residues have been synthesized. Nuclear magnetic resonance studies have been carried out on the 1-5, 6-10, 1-6, 1-7, 1-8, 1-9, and 1-10 fragments. The number of solvent-shielded NH groups in $CD{Cl}_3$ solutions for 1-5, 1-6, 1-7, 1-8, 1-9, and 1-10 indicate that $3_{10}$-helical structures are favored in this solvent. In ${({CD}_3)}_2SO$, an additional NH group, assigned to Aib(3) NH is solvent exposed in the fragments longer than six residues, suggesting partial unfolding of the N-terminal \beta-turn or transition to an \alpha-helical conformation. The data for fragment 6-10 are consistent with a conformation having a single Leu-Aib \beta-turn. Infrared studies suggest an increase in the number of intramolecular hydrogen bonds with increasing peptide chain length. Appreciable mitochondrial uncoupling activity is observed for peptides with a chain length of at least seven residues. The order of efficiencies of the fragments is 1-7 < 1-8 - 1-10 < 1-9, with the decapeptide exhibiting anomalously low uncoupling activity.


text:
# Possibilities of the Number of Solutions of a Homogeneous System of Linear Equations

## Problem 14

Here is a very short true or false problem.

Select either True or False. Then click “Finish quiz” button.

You will be able to see an explanation of the solution by clicking “View questions” button.

Sponsored Links

## System of linear equations

Determine whether the following sentence is True or False.

Sponsored Links

### More from my site

#### You may also like...

This site uses Akismet to reduce spam. Learn how your comment data is processed.

##### Common Eigenvector of Two Matrices and Determinant of Commutator

Let $A$ and $B$ be $n\times n$ matrices. Suppose that these matrices have a common eigenvector $\mathbf{x}$. Show that $\det(AB-BA)=0$.

Close