text:
# reviews

On things about which I have an opinion

# Save your email! Avoid the Thunderbird 78 update

Saturday, June 19, 2021

History repeats itself as the TB devs learn nothing from the misery they created by auto-updating 60x users to 68 without providing any warning or option to avoid the update.  This is crappy user management.  On updates that will break an installed add-on, the user should be informed of what will be disable and asked if they want to proceed with the update, not silently forced to conform to a stripped-down, unproductive environment as if the user’s efforts at optimization were childish mistakes unworthy of consideration or notice.

The Thunderbird devs have increasingly adopted a “if you’re not doing it our way, you’re doing it wrong and we’re going fix your mistake whether you like it or not” attitude.  This is highly annoying because the org already alienated their add-on community by repeatedly breaking the interface models add-on developers relied on.

For a while add-on devs gamely played along dealing with reputational damage as idiotic and poorly planned actions by Thunderbird devs broke their code and left them to deal with user frustration and scrambled to fix problems they didn’t create.  Many, if not by now most, add-on developers finally had enough and abandoned ship.  This is tragic because without some of the critical modifications to Thunderbird provided by developers it is essentially unusable.

I eventually came to peace with the add-on-pocolypse between 60 and 68 as add on developers worked through it and very carefully set my TB 68 to not update ever again, even though 90a finally fixes the problem that 68 caused where it became impossible to display dates in ISO 8601 format, but that’s a whole ‘nother kettle of fish.

Still, despite trying to block it, I got a surprise update; if this keeps up, I’ll switch to Interlink Mail and News.

So if you, like I did, got force “upgraded” to 78 from a nicely customized 68, this is what worked for me to undo the damage: (If you weren’t surprise updated, then jump right down to preventing future surprises.)

• Uninstall thunderbird (something like # sudo apt remove thunderbird)
• Extract the tar file and copy it (sudo) to /usr/lib/thunderbird
sudo mv ~/downloads/thunderbird/ /usr/lib/thunderbird
• Create a desktop entry

# nano ~/.local/share/applications/tb68.desktop

[Desktop Entry]
Version=1.0
Type=Application
Name=Thunderbird-68
Icon=thunderbird
Exec="/usr/lib/thunderbird/thunderbird"
Comment=last TB version
Categories=Application;Network;Email;
Terminal=false
MimeType=x-scheme-handler/mailto;application/x-xpinstall;
StartupNotify=true
Actions=Compose;Contacts

• Prevent future updates (hopefully) by creating a no-update policy file:
# sudo nano /usr/lib/thunderbird/distribution/policies.json

{
"policies": {
"DisableAppUpdate": true
}
}

and then, just to be sure, break the update checker code:

# sudo mv /usr/lib/thunderbird/updater /usr/lib/thunderbird/no-updater
# sudo mv /usr/lib/thunderbird/updater.ini /usr/lib/thunderbird/no-updater.ini
• Start the freshly improved and downgraded to the last remotely usable version of Thunderbird with a special downgrade allowed option the first time from the command line:
# /usr/lib/thunderbird/thunderbird --allow-downgrade

If you were unlucky enough to launch TB 78 even once, your add-ons are screwed up now (thanks devs, Merry Christmas to you too).  Those that have a 78 compatible version will have been auto-updated to the 78 version which isn’t compatible with 68 (w00t w00t, you can see why the plugin devs quit in droves).  At least this time your incompatible add-ons weren’t auto-deleted like with 68.  Screen shot or otherwise capture a list of your disabled plugins, then remove the incompatible ones and add them back to the 68-compatible previous release.

• Restart one more time normally to re-enable the 68 compatible add-ons without 78 updates that the 78 launch disabled.

One more detail – if find your CardBook remote address books are gone, you need to rebuild your preferences.

• Back up your profile (good thing to do no matter what)
• Uninstall the CardBook plugin
• Quit TB
• In your profiles directory, delete all files that end with .sqlite (rm *.sqlite)
• Restart TB (the .sqlite files should be recreated)
• Reinstall the CardBook plugin.  Your address books should reappear.  (if not, the advice on the interwebs is to create a new profile and start over).

PHEW! just a few hours of lost time and you’ve fixed the misery the TB devs forced on you without asking.  How nice.  What thoughtful people.

[poll id=”2″]

Posted at 07:16:58 GMT-0700

Category: HowToLinuxNegativereviewsSecuritytechnology

# Compile and install Digikam on Ubuntu 20.04 Focal (21.10 too)

Friday, March 26, 2021

Digikam is an incredibly powerful media management tool that integrates a great collection of powerful media processing projects into a single, fairly nice and moderately intuitive user interface. The problem is that it make use of SO many projects and libraries that installation is quite fragile and most distributions are many years out of date – that is a typical sudo apt install digikam will yield version 4.5 while release is (as of this writing) 7.5.

In particular, this newer version has face detection that runs LOCALLY – not on Google or Facebook’s servers – meaning you don’t have to trade your personal photos and all the data implicit in them to a data broker to make use of such a useful tool.  Sure, Google once bought and then improved Picasa Desktop which gave you this function, but then they realized this was cutting into their data harvesting business and discontinued Picasa and tried to convince people to let them look at all their pictures with Google Photos.  We really, really need to make personal data a toxic asset, such an intolerable liability that any company that holds any personal data has negative value.  But until then, use FOSS software on your own hardware where ever possible.

You can compile the latest version on Ubuntu 20.04 Focal Fossa, though not exactly painlessly, or you can install the flatpak easily. I hate flatpaks with a passion, so I went through the exercise and found what appears to be stable success with the following procedure which yielded a fully featured digikam with zero dependency errors or warnings and all features enabled using MariaDB as a backend.

Updating Ubuntu from 20.04 to 21.10 (probably any other major update too) will (as typical) break a ton of stuff.  For “reasons” the updater uninstalls all sorts of things like MariaDB and many of the dependencies.  Generally, as libraries change versions, recompiling is required.  This is so easy with FreeBSD ports…

sudo apt update
sudo mysql_secure_installation

The secure options are all good, accept them unless you know better.

Start the server (if it isn’t)

sudo systemctl start mariadb.service

Do some really basic config:

sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

and set:

character-set-server = utf8mb4
collation-server = utf8mb4_general_ci
default_storage_engine = InnoDB

Switch to mariadb and create an admin user account and (I’d suggest) one for digikam.  It seems this has to be done before the first connect and can’t be fixed after.  You’ll probably want to use a different ‘user’ than I did, but feel free.

sudo mariadb
CREATE USER 'gessel'@'localhost' IDENTIFIED BY 'password';
GRANT ALL ON *.* TO 'gessel'@'localhost' IDENTIFIED BY 'password';
CREATE DATABASE digikam;
GRANT ALL PRIVILEGES ON digikam.* TO 'gessel'@'localhost';
FLUSH PRIVILEGES;

should correctly create the correct user – though check the instructions tab on the database connection options pane for any changes if you’re following these instructions for install of a later version. You will need the socket location to connect to the database so before exit; run:

mysqladmin -u admin -p version

Should yield something like:

Enter password:

Protocol version	10
Connection		Localhost via UNIX socket
UNIX socket		/var/run/mysqld/mysqld.sock
Uptime:			5 hours 26 min 6 sec

Threads: 29  Questions: 6322899  Slow queries: 0  Opens: 108  Flush tables: 1  Open tables: 74  Queries per second avg: 323.157

And note the value for UNIX socket, you’re going to need that later: /var/run/mysqld/mysqld.sock – yours might vary.

### Install digiKam Dependencies

• Updated to libx264-163 and libx265-199
• Version change from 7.2.0 to 7.3.0

• Installing on Ubuntu 21.10 “impish”
• Version change to 7.5.0 (note camelcase used for file name now, “digiKam” not “digikam“)
• Problem with libopencv-dev required selecting a # sudo aptitude install solution to get past a libilmbase-dev but it is not installable error.

Digikam has just a few dependencies.just a few... the below command should install the needed for 7.30 on Ubuntu 21.10. Any other version combination might be different.:

sudo aptitude install \
bison \
checkinstall \
devscripts \
doxygen \
extra-cmake-modules \
ffmpeg \
ffmpegthumbnailer \
flex \
graphviz \
help2man \
jasper \
libavcodec-dev \
libavdevice-dev \
libavfilter-dev \
libavformat-dev \
libavutil-dev \
libboost-dev \
libboost-graph-dev \
libeigen3-dev \
libexiv2-dev \
libgphoto2-dev \
libjasper-dev \
libjasper-runtime \
libjasper4 \
libjpeg-dev \
libkf5calendarcore-dev \
libkf5contacts-dev \
libkf5doctools-dev \
libkf5kipi-dev \
libkf5notifyconfig-dev \
libkf5sane-dev \
libkf5solid-dev \
libkf5xmlgui-dev \
liblcms2-dev \
liblensfun-dev \
liblqr-1-0-dev \
libmagick++-6.q16-dev \
libmagick++-6.q16hdri-dev \
libmagickcore-dev \
libmarble-dev \
libqt5opengl5-dev \
libqt5sql5-mysql \
libqt5svg5-dev \
libqt5webkit5-dev \
libqt5webview5 \
libqt5webview5-dev \
libqt5x11extras5-dev \
libqt5xmlpatterns5-dev \
libqtav-dev \
libqtwebkit-dev \
libswscale-dev \
libtiff-dev \
libusb-1.0-0-dev \
libx264-163 \
libx264-dev \
libx265-199 \
libx265-dev \
libxml2-dev \
libxslt1-dev \
marble \
pkg-kde-tools \
qtbase5-dev \
qtbase5-dev-tools \
qtmultimedia5-dev \
qtwebengine5-dev \
libopencv-dev \
qtwebengine5-dev-tools

### Compile Digikam

Switch to your projects directory (~/projects, say) and get the source, cross your fingers, and go to town. The make -j4 command will take a while to compile everything.  There are two basic mechanisms for getting the source code: wget the taball or git pull the repository.

wget https://download.kde.org/stable/digikam/7.5.0/digiKam-7.5.0.tar.xz
tar -xvf digiKam-7.5.0.tar.xz
cd digiKam-7.5.0.tar.xz

#### git pull the repository

Git uses branches/tags so check the pull down list of latest branches and tags at the top left, below the many, many branches is the tag list at https://invent.kde.org/graphics/digikam/-/tree/v7.5.0 , latest on top, and currently 7.5.0. This is currently a 1.4 GB git pull (!!).
There was an issue in the v7.3.0 tag that caused built to fail that was fixed in current, so building “stable” isn’t always the best choice for stability.

git clone -b v7.5.0 https://invent.kde.org/graphics/digikam.git
cd digikam

./bootstrap.linux
cd build
make -j4
sudo su
make install/fast

Compiling might take 15-30 minutes depending on CPU.  Adjust -jx to optimize build times, the normal rule of thumb is that x=# of cores or cores+1, YMMV, 4 is a reasonable number if you aren’t confident or interested in experimenting.

The ./bootstrap.linux result should be as below; if it indicates a something is missing then double check dependencies.  If you’ve never compiled anything before, you might need to install cmake and and some other basics not in the apt install list above:

-- ----------------------------------------------------------------------------------
--  digiKam 7.2.0 dependencies results   <https://www.digikam.org>
--
--  MySQL Database Support will be compiled.. YES (optional)
--  MySQL Internal Support will be compiled.. YES (optional)
--  DBUS Support will be compiled............ YES (optional)
--  App. Style Support will be compiled...... YES (optional)
--  QWebEngine Support will be compiled...... YES (optional)
--  libboostgraph found...................... YES
--  libexiv2 found........................... YES
--  libexpat found........................... YES
--  libjpeg found............................ YES
--  libkde found............................. YES
--  liblcms found............................ YES
--  libopencv found.......................... YES
--  libpng found............................. YES
--  libqt found.............................. YES
--  libtiff found............................ YES
--  bison found.............................. YES (optional)
--  doxygen found............................ YES (optional)
--  ccache found............................. YES (optional)
--  flex found............................... YES (optional)
--  libmagick++ found........................ YES (optional)
--  libeigen3 found.......................... YES (optional)
--  libgphoto2 found......................... YES (optional)
--  libjasper found.......................... YES (optional)
--  libkcalendarcore found................... YES (optional)
--  libkiconthemes found..................... YES (optional)
--  libkio found............................. YES (optional)
--  libknotifyconfig found................... YES (optional)
--  libksane found........................... YES (optional)
--  liblensfun found......................... YES (optional)
--  liblqr-1 found........................... YES (optional)
--  libmarble found.......................... YES (optional)
--  libqtav found............................ YES (optional)
--  libxml2 found............................ YES (optional)
--  libxslt found............................ YES (optional)
--  libx265 found............................ YES (optional)
--  OpenGL found............................. YES (optional)
--  libqtxmlpatterns found................... YES (optional)
--  digiKam can be compiled.................. YES
-- ----------------------------------------------------------------------------------

### Launch and configure Digikam

(if you’re still root, exit root before launching # digikam)

The Configuration options are pretty basic, but note that to configure the Digikam back end you’ll need to use that MariaDB socket value you got before and the user you created like so UNIX_SOCKET=/var/run/mysqld/mysqld.sock:

On the first run, it will download about 350mb of code for the face recognition engine.  Hey – maybe a bit heavy, but you’re not giving the Google or Apple free lookie looks at all your personal pictures.  Also, if all this is a bit much (and, Frankly, it is) I’d consider Digikam one of the few applications that makes the whole flatpak thing seem somewhat justified.  Maybe.

I recommend mysqltuner highly, then maybe check this out (or just leave it default, default works well).

Tuning a database is application and computer specific, there’s no one size fits any, certainly not all, and it may change as your database grows. There are far more expert and complete tuning guides available, but here’s what I do:

#### Pre-Tuning Data Collection

Tuning at the most basic involves instrumenting the database to log problems, running it for a while, then parsing the performance logs for useful hints. The mysqltuner.pl script is far more expert at than I’ll ever be, so I pretty much just trust it. You have to modify your mysqld.cnf file to enable performance data collection (which, BTW, slows down operation, so undo this later) which, for MariaDB, means adding a few lines:

sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf
# enable performance schema to allow optimization, but ironically hit performance, so disable after tuning.
# in the [mysqld] section insert
performance_schema=ON
performance-schema-instrument='stage/%=ON'
performance-schema-consumer-events-stages-current=ON
performance-schema-consumer-events-stages-history=ON
performance-schema-consumer-events-stages-history-long=ON

I rather like this guide’s helpful instructions for putting the script in /usr/local/sbin/ so it is in the execution path:

sudo wget https://raw.githubusercontent.com/major/MySQLTuner-perl/master/mysqltuner.pl -O /usr/local/sbin/mysqltuner.pl
sudo chmod 700 /usr/local/sbin/mysqltuner.pl
sudo mysqltuner.pl

Then restart with sudo service mariadb restart then go about your business with digikam – make sure you rack up some real hours to gather useful data on your performance. Things like ingesting a large collection should generate useful data. I’d suggest doing disk tuning first because that’s hardware not load dependent.

#### Disk tuning

Databases tend to hammer storage and SSDs, especially SLC/enterprise SSDs, massively improve DB performance over spinning disks – unless you have a massive array of really good rotating drives. I’m running this DB on one spinning disk, so performance is very MEH. MySQL and MariaDB make some assumptions about disk performance which is used to scale some pretty important parameters for write caching. You can meaningfully improve on the defaults by testing your disk with a great linux utility called “fio”.

sudo apt install fio
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=4G --readwrite=randrw --rwmixread=75

This will take a while and will give some very detailed information about the performance of your disk subsystem, the key parameters being average and max write IOPS. I typically create a # performance tuning section at the end of my [mysqld] section and before [embedded] and I’ll put these values in as, say: (your IOPS values will be different):

# performance tuning

innodb_io_capacity              = 170
innodb_io_capacity_max          = 286

and sudo service mariadb restart

#### Using mysqltuner.pl

After you’ve collected some data, there may be a list of tuning options.

sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

Mine currently look like this, but they’ll change as the database stabilizes and my usage patterns change.

# performance tuning

innodb_io_capacity              = 170
innodb_io_capacity_max          = 286

innodb_buffer_pool_size         = 4G
innodb_log_file_size            = 512M
innodb_buffer_pool_instances    = 4
skip_name_resolve               = 1
query_cache_size                = 0
query_cache_type                = 0
query_cache_limit               = 2M
max_connections                 = 175
join_buffer_size                = 4M
tmp_table_size                  = 24M
max_heap_table_size             = 24M
innodb_buffer_pool_size         = 4G
max_allowed_packet              = 128M

and

sudo service mariadb restart

Note max_allowed_packet = 128M comes from this guide. I trust it, but it isn’t a mysqltuner suggestion.

Posted at 17:11:21 GMT-0700

Category: HowToLinuxphotoPositivereviewstechnology

# Tagging MP3 Files with Puddletag on Linux Mint

Tuesday, March 23, 2021

A “fun” part of organizing an MP3 collection is harmonizing the tags so the datas work consistently with whatever management schema you prefer.  My preference is management by the file system—genre/artist/year/album/tracks works for me—but consistent metainformation is required and often disharmonious.  Finding metaharmony is a chore I find less taxing with a well structured tag editor and to my mind the ur-meta-tag manager is  MP3TAG.

The problem is that only works with that dead-end spyware riddled failing legacyware called “Windows.” Fortunately, in Linux-land we have puddletag, a very solid clone of MP3TAG.  The issues is that the version in repositories is (as of this writing) 1.20 and I couldn’t find a PPA for the latest, 2.0.1.  But compiling from source is super easy and works in both Linux Mint 19 and Ubuntu 20.04 (yay open source!):

1. Install pre-reqs to build (don’t worry, if they’re installed, they won’t be double installed)
2. get the tarball of the source code
3. expand it (into a reasonable directory, like ~/projects)
4. switch into that directory
5. run the python executable “puddletag” directly to verify it is working
6. install it
7. tell the desktop manager it’s there – and it should be in your window manager along with the rest of your applications.

sudo apt install python3-pyqt5 python3-pyqt5.qtsvg python3-pyparsing python3-mutagen python3-acoustid libchromaprint-dev libchromaprint-tools libchromaprint1
tar -xvf puddletag-2.0.1.tar.gz cd puddletag-2.0.1/
cd puddletag
./puddletag
sudo python3 setup.py install
sudo desktop-file-install puddletag.desktop

A nice feature is the configuration directory is portable and takes your complete customization with you – it is an extremely customizable program so you can generally configure it as fits your mental model.  Just copy the entire puddletag directory located at ~/.configure/puddletag.

Posted at 15:19:01 GMT-0700

Category: AudioHowToLinuxPositivereviewsuncategorized

# EZ rsync cheat sheet

Wednesday, December 30, 2020

Rsync is a great tool – incredibly powerful for synchronizing directories, copying over a network or over SSH, an awesome way to backup a mobile device back to a core network securely and other great functions.  it works better than just about anything else developed before or since, but is a command line UI that is easy to forget if you don’t use it for a while and Windows is a challenge.

This isn’t meant to be a comprehensive guide, they’re are lots of those, but a quick summary of what I find useful.

There’s one confusing thing that I have to check often to be sure it is going to do what I think it should – the trailing slash on the source.  It works like this:

A quick summary of useful command options (there are many, many) is:

-v, --verbose               increase verbosity
-r, --recursive             recursive (go into subdirectories)
-c, --checksum              skip based on checksum, not mod-time & size (slow, but accurate)
-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X) (weird with SMB/CIFS)
-z, --compress              compress file data during the transfer, should help over slow links
-n, --dry-run               trial run, don't move anything
-u, --update                only copy files that have different sizes and equal or later modification times (-c will enable checksum comparison)
--progress              show the sync progress during transfer
--exclude ".*"          exclude files starting with "."
--remove-source-files   after synced, empty the dir (like mv/merge)
--delete                any files in dest that aren't in source are deleted in destination (danger)
--info=progress2 --info=name0  This yields a pretty usable one line progress meter.

I do not recommend using compression (-z) on a LAN, it’ll probably slow you down. Over a slower (typically) WAN link it usually helps, but YMMV depending on link and CPU speed. Test it with that one line progress meter if it is a long enough sync to matter – it shows transfer rate a little like this:

1,770,984,121   2%  747.54kB/s   27:46:38  xfr#2159, ir-chk=1028/28648)

If the files really have to be accurately transferred, the checksum (-c) option is critical – every copy (or at least “move”) function should include this validation, especially before deleting the original.

Posted at 11:53:28 GMT-0700

Category: FreeBSDLinuxPositivereviews

# Dealing with Apple Branded HEIF .HEIC files on Linux

Saturday, August 22, 2020

Some of the coding tricks in H.265 have been incorporated into MPEG-H coding, an ISO standard introduced in 2017, which yields a roughly 2:1 coding efficiency gain over the venerable JPEG, which was introduced in 1992.  Remember that?  I do; I’m old.  I remember having a hardware NUBUS JPEG decoder card.   One of the reasons JPEG has lasted so long is that images have become a small storage burden (compared to 4k video, say) and that changing format standards is extremely annoying to everyone.

Apple has elected to make every rational person’s life difficult and put a little barbed wire around their high-fashion walled garden and do something a little special with their brand of a HEVC (h.265) profile for images.  Now normally seeing iOS user’s insta images of how fashionable they are isn’t really worth the effort, but now and then a useful correspondent joins the cult and forks over a ton of money to show off a logo and starts sending you stuff in their special proprietary format.  Annoying, but fixable.

Assuming you’re using an OS that is neither primarily spyware nor fashion forward, such as Linux Mint, you can install HEIF decode (including Apple Brand HEIC) with a few simple commands:

$sudo add-apt-repository ppa:jakar/qt-heif$ sudo apt update
\$ sudo apt install qt-heif-image-plugin

Once installed, various image viewers should be able to decode the images.  I rather like nomacs as a fairly tolerable replacement for Irfan Skiljan‘s still awesome irfanview.

Update: 2022-09-22

Jammy isn’t supported by the jakar PPA, but there are a few other options:

Easy, from Hritik Chaudhary in this post,

sudo apt install heif-gdk-pixbuf

should give gdk access, but not (it seems) qt.  You can use gpicview as an imageviewer with this library.

sudo apt install gpicview

Or build the qt-heic-image-plugin from source:

git clone --depth 1 https://github.com/novomesk/qt-heic-image-plugin.git
cd qt-heic-image-plugin
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make
sudo make install

This machine required ECM from extra cmake modules, which I hadn’t previously installed.

sudo apt install extra-cmake-modules

to successfully cmake.

Posted at 03:56:36 GMT-0700

Category: HowToLinuxphotoPositivereviewstechnology

# How to stick with a decent version of Firefox (pre-Quantum)

Friday, June 1, 2018

Firefox (52) remains my browser of choice – entirely because of plug-ins. When Firefox completely destroyed the UI/UX with “Australis,” a horrific UI change that basically made Firefox into a crappy clone of Chrome, the only thing that made Firefox usable was “Classic Theme Restorer.” Apparently, unsatisfied with the damage Mozilla had managed to wreak on their user-base with idiotic UI decisions, over the past year or so, a new version called “Quantum” (57) was rolled out that broke the functionality of almost every important plug-in.

This utterly disastrous and truly unforgivable transgression against the user-base was only slightly mitigated by sustaining 52-ESR, at least until the Sept of this year. After that, everyone who cares about having a decent alternative to Chrome will have to migrate to Waterfox.

In the mean time, one really important thing you have to remember to do if you stuck reinstalling your system on Linux (e.g. Linux Mint) is to immediately uninstall Firefox before using it even once.  Then change your install version to ESR and install. If you let Quantum run even once, it will mark all your good plugins as disabled and you need to reinstall them one at a time to get them working again.

sudo add-apt-repository ppa:jonathonf/firefox-esr
sudo apt-get update
sudo apt-get install firefox-esr

Well, that’s the end of Firefox….  Sad to see it go after all these years, but the new plugin concept has made Firefox a subordinate version of Chrome rather than a powerful, customizable tool.

I followed these fine directions and now have waterfox running.

echo 'deb https://web.archive.org/web/20190207094152/https://dl.bintray.com/hawkeye116477/waterfox-deb/ release main' | sudo tee /etc/apt/sources.list.d/waterfox.list
sudo apt-get update
sudo apt install waterfox

Just import your firefox preferences on starup and walla, you instant happiness with all the plugins that Quantum broke restored, including such absolute essentials as “Classic Theme Restorer” (which undoes the absolutely horrible UI changes that Mozilla adopted) and downthemall, privacy plugins, etc. If you’ve updated some plugins to be Quantum compatible, you’ll have to back those up (for me that’s things like FoxClocks and Noscript, which managed to hack together semi-viable Quantum compatible plugins after slogging through Mozilla’s buggy WebExtensions API and HTML5 quirks).

This whole translation is quite unfortunate. Waterfox is dependent on the Firefox code base, so this solution may have a finite lifespan, but for now it works and undoes the horror of Firefox 57+/Quantum.

Posted at 11:43:58 GMT-0700

Category: Neutralreviewstechnology

# Turn off windows update now!

Monday, March 14, 2016

If you haven’t already, turn off Windows update now.  Microsoft has recently started installing Windows 10 spyware without consent.  A good friend of mine had a bunch of systems at the company where he runs IT hacked by Microsoft over the weekend, which broke the certificate store for WPA-2 and thus their wifi connections.

To be clear, Windows 10 is spyware.  Microsoft has changed their business model from selling a product to selling data – your data – to whoever they want.  Windows 10 comes with a EULA that gives them the right to steal everything on your computer – your email, your private pictures, your home movies, your love letters, your medical records, your financial records – anything they want without telling you.  “If you’re not paying for the product, you are the product.

If this happens to you,  I suggest contacting your state attorney general and filing a complaint against Microsoft.  Hopefully a crushing class action suit or perhaps jail time for the executives that dreamed up this massive heist will help deter future corporate data thieves, though that’s certainly irrational optimism.

I wish I could recommend switching to Linux for everyone, but there’s a lot of software that still depends on Windows and a lot of users that will have a hard time migrating (developers: please stop developing for Microsoft).  Apple seems unequivocally better in refusing to act as key player in bringing about Total Information Awareness.  I’m not a huge fan of their walled garden and computers as overpriced fashion accessories approach, but it is far better than outright theft.  For those that are slightly computer savvy, there’s Linux Mint, which is quite usable and genuinely free.

These instructions might help prevent that disaster of an update being visited upon you (and possibly law enforcement visits to come after Microsoft starts sifting through all your datas and forwarding on whatever they find).  The latest reports suggest they aren’t enough, but it is the best I have found other than isolating your windows box from the internet completely.

Posted at 14:27:03 GMT-0700

Category: NegativePrivacySecuritytechnology

# Green Lacewings

Sunday, January 10, 2016

I noticed that my avocado tree was developing brown spots on the leaves, which were almost certainly the result of Persea mites.

So I looked up some possible cures, and it seemed like introducing a predator would be the best option and the least hassle.  I’d had good luck with introduced ladybugs a few years back, which formed a stable population that survived for many years after introduction.  For this pest, green lacewings are recommended.  I found a nearby insectary that could provide larvae on cards and they shipped them overnight.

The little guys look cute just waiting to hatch…

I hung he cards on the leaves of the tree after incubating them overnight in a warm room, and they should hatch sometime in the next day or two, as long as the ants don’t find them first…

Update 8 Sept 2016:

The green lacewings seem to have eaten all the mites.  It has been 9 months and there aren’t any signs of damage to this spring’s leaves.  Yay!

The new leaves that grew seem to be developing without any bites at all.  The old leaves that were too damaged have fallen off, but the surviving older leaves still show the scars of the mites.  Green lacewings seem to have done the trick.

Posted at 14:40:47 GMT-0700

Category: photoPositivereviews

# Signal Desktop: Probably a good thing

Tuesday, December 8, 2015

Signal is an easy to use chat tool that competes (effectively) with What’sApp or Viber. They’ve just released a desktop version which is being “preview released/buzz generating released.”  It is developed by a guy with some cred in the open source and crypto movement, Moxie Marlinspike.  I use it, but do not entirely trust it.

I’m not completely on board with Signal.  It is open source, and so in theory we can verify the code.  But there’s some history I find disquieting.  So while I recommend it as the best, easiest to use, (probably) most secure messaging tool available, I do so with some reservations.

• It originally handled encrypted SMS messages.  There is a long argument about why they broke SMS support on the mailing lists.  I find all of the arguments Whisper Systems made specious and unconvincing and cannot ignore the fact that the SMS tool sent messages through the local carrier (Asiacell, Korek, or Zain here).  Breaking that meant secure messages only go through Whisper Systems’ Google-managed servers where all metadata is captured and accessible to the USG. Since it was open source, that version has been forked and is still developed, I use the SMSSecure fork myself
• Signal has captured all the USG funding for messaging systems.  Alternatives are not getting funds.  This may make sense from a purely managerial point of view, but also creates a single point of infiltration.  It is far easier to compromise a single project if there aren’t competing projects.   Part of the strength of Open Source is only achieved when competing development teams are trying to one up each other and expose each other’s flaws (FreeBSD and OpenBSD for example).  In a monoculture, the checks and balances are weaker.
• Signal has grown more intimate with Google over time.  The desktop version sign up uses your “google ID” to get you in the queue.  Google is the largest commercial spy agency in the world, collecting more data on more people than any other organization except probably the NSA.  They’re currently an advertising company and make their money selling your data to advertisers, something they’re quite disingenuous about, but the data trove they’ve built is regularly mined by organizations with more nefarious aims than merely fleecing you.

What to do?  Well, I use signal.  I’m pretty confident the encryption is good, or at least as good as anything else available.  I know my metadata is being collected and shared, but until Jake convinces Moxie to use anonymous identifiers for accounts and message through Tor hidden nodes, you have to be very tech savvy to get around that and there’s no Civil Society grants going to any other messaging services using, for example, an open standard like a Jabber server on a hidden node with OTR.

For now, take a half step up the security ladder and stop using commercial faux security (or unverifiable security, which is the same thing) and give Signal a try.

Maybe at some later date I’ll write up an easy to follow guide on setting up your own jabber server as a tor hidden service and federating it so you can message securely, anonymously, and keep your data (meta and otherwise) on your own hardware in your own house, where it still has at least a little legal protection.

Posted at 10:21:22 GMT-0700

Category: PositivePrivacyreviewsSecuritytechnology

# Low Voltage LED Lighting

Monday, July 13, 2015

My kitchen has had halogen lighting for 20 years, from back when it was a slightly more efficient choice than incandescent lighting and had a pleasing, cooler (bluer, meaning the filament runs hotter) color temperature.

Progress has moved on and while fluorescent lights still have a lead in maximum luminous efficacy (lm/w), for example the GE Ecolux Watt-Miser puts out 111 lm/W, they’re less versatile than LEDs and installation is a hassle while low voltage LEDs are easy to install and look cool.

[wp_charts title=”Luminous efficacy of lighting types” type=”bar” align=”alignleft” margin=”5px 20px” datasets=”17.5,24,111,101.9,200″ labels=”incandescent,halogen,fluorescent,LED,LP Sodium” width=100% scaleFontColor=#FFF]

### System Design

The goal of this project was to add dimmable, pleasing light to the kitchen that I found aesthetically interesting.  I wanted a decent color rendering index (CRI), ease of installation, and at reasonable cost.  I’ve always liked the look of cable lighting and the flexibility of the individual, adjustable luminaires.

I couldn’t find much information on how variable output LEDs work and what can be used to drive them.  I have a pretty good collection of high quality power supplies, which I wanted to take advantage of, but wasn’t sure if I’d be able to effectively dim the bulbs from the documentation I found. So I did some tests.

### Test Configuration

I bought a few different 12V, Dimmable LEDs and set up a test configuration to verify operation and output with variable voltage and variable current.  The one bit of data I had was that using standard commercial controllers, the lowest output is typically stated to be around 70% of maximum output: that is the dimming range is pretty limited with standard (PWM/Transformer) controllers.  The results I found were much more encouraging, but revealed some quirks.

I used a laboratory-grade HP power supply with voltage and current control to drive the LEDs, decent multimeters to measure voltage and current, and an inexpensive luminance meter to measure LED output.

I measured 3 different LEDs I selected based on price and expected compatibility with the aesthetics of the project and because they looked like they’d have different internal drivers and covered a range of rated wattage.

### Test Results

These bulbs have internal LED controllers that do some sort of current regulation for the diodes that results in a weird voltage/current/output response.  Each bulb has a different turn-on voltage, then responds fairly predictably to increasing input voltage with increasing output, reaches the controller stabilizing voltage and runs very inefficiently until voltage gets over the rated voltage and then becomes increasingly efficient until, presumably, at some point the controller burns out.  I find that the bulbs all run more efficiently at 14V than at the rated 12V.

As a side note, to perform the data analysis, I used the excellent xongrid plugin for excel to perform Kriging interpolation (AKA Gaussian process regression) to fit the data sets to the graphing function’s capabilities.  The graphs are generated with WP-Charts and the table with TablePress.

#### Watts v. Volts

This chart shows the wattage consumed by each of the three LEDs as a function of input voltage, clearly demonstrating both that the power consumption function is non-linear and that power consumption in watts improves when driven over the rated 12V.  Watts are calculated as the product of the measured Volts * Amps.  Because of the current inversion that happens as the controllers come fully on-line, these LEDs can’t be properly controlled near full brightness with a current-controlled power supply, though it works well to provide continuous and fairly linear dimming at low outputs, once the voltage/current function changes slope, the current limiting controller in the power supply freaks out.

[wp_charts title=”wattvolts” type=”line” align=”alignright” margin=”5px 20px” datasets=”0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.015,0.017,0.024,0.035,0.048,0.108,0.279,0.594,1.040,1.603,2.292,3.100,4.069,4.865,4.801,4.559,4.511,4.494,4.478,4.457,4.438,4.415,4.391,4.366,4.348,4.330,4.311,4.291,4.271,4.250,4.228,4.206,4.186,4.166,4.145,4.124,4.104,4.083,4.063,4.041,4.017,3.993,3.971,3.955 next 0.051,0.086,0.146,0.293,0.669,1.209,1.879,2.738,3.466,4.259,5.530,6.700,6.995,6.972,6.524,6.000,5.803,5.673,5.560,5.457,5.379,5.307,5.239,5.176,5.127,5.082,5.039,5.000,4.970,4.944,4.919,4.896,4.878,4.862,4.846,4.831,4.817,4.802,4.788,4.772,4.752,4.731,4.710,4.689,4.657,4.631,4.621,4.654,4.845,5.066,5.274,5.419,5.293,5.066,4.817,4.589 next 0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.314,2.205,4.725,4.747,4.928,5.124,5.346,5.550,5.726,5.883,6.035,6.169,6.273,6.343,6.332,6.283,6.217,6.145,6.091,6.040,5.988,5.937,5.898,5.864,5.830,5.798,5.772,5.746,5.721,5.695,5.668,5.642,5.615,5.590,5.571,5.554,5.537,5.521,5.502,5.484,5.466,5.448,5.428,5.410,5.394,5.383″ labels=”2.25,,,3.00,,,,4.00,,,,5.00,,,,6.00,,,,7.00,,,,8.00,,,,9.00,,,,10.00,,,,11.00,,,,12.00,,,,13.00,,,,14.00,,,,15.00,,,,16.00″ width=100% scaleFontColor=#FFF fillopacity=”0.0″ colors=”#FF3300,#66FF33,#3399FF”]

4W LED  5W LED  7.5W LED

#### Lux v. Volts

This chart shows the lux output by each of the three LEDs as a function of input voltage, revealing the effect of the internal LED driver coming on line and regulating output, which complicates controlling brightness but protects the LEDs.  The 5W LEDs have a fairly gentle response slope and start a very low voltage (2V) so are a good choice for a linear power supply.  The 4W LEDs don’t begin to light up until just over 6V, and so are a good match for low-cost switch mode supplies that don’t go to zero.

[wp_charts title=”luxvolts” type=”line” align=”alignright” margin=”5px 20px” datasets=”0,0,0,0,0,0,0,0,0,0,0,0,2,4,15,34,57,172,484,996,1632,2360,3110,3870,4629,5220,5156,4956,4925,4920,4908,4890,4877,4861,4838,4813,4795,4778,4760,4741,4722,4703,4682,4662,4643,4623,4604,4583,4559,4534,4509,4483,4455,4426,4400,4380 next 7,42,103,255,663,1180,1731,2380,2859,3311,3848,4340,4599,4762,4789,4770,4778,4781,4778,4770,4772,4773,4772,4770,4771,4772,4771,4770,4771,4772,4771,4770,4768,4766,4763,4760,4762,4764,4764,4760,4743,4721,4699,4681,4676,4676,4678,4680,4680,4680,4680,4680,4679,4677,4677,4680 next 0,0,0,0,0,0,0,0,0,0,376,2490,4810,4976,5200,5421,5687,5940,6168,6372,6546,6700,6848,6969,7017,7030,7025,7011,7008,7006,7000,6991,6987,6982,6977,6970,6966,6961,6956,6950,6948,6946,6943,6940,6936,6931,6925,6920,6917,6915,6913,6910,6904,6899,6896,6898″ labels=”2.25,,,3.00,,,,4.00,,,,5.00,,,,6.00,,,,7.00,,,,8.00,,,,9.00,,,,10.00,,,,11.00,,,,12.00,,,,13.00,,,,14.00,,,,15.00,,,,16.00″ width=100% scaleFontColor=#FFF fillopacity=”0.0″ colors=”#FF3300,#66FF33,#3399FF”]

4W LED  5W LED  7.5W LED

#### Lux/W v. Volts

This chart shows the luminous efficiency (Lux/Watt, Lumen measurement is quite complicated) by each of the three LEDs as a function of input voltage, showing that overdriving the LEDs past the rated 12V can significantly improve efficiency.  There’s some risk it will overheat the controller at some point and result in failure.  I’ll update this post if my system starts to fry LEDs, but my guess is that 14V, which cuts the power load by 20% over 12V operation with the 7.5W lamps I selected, will not significantly impact operational lifetime.

[wp_charts title=”effvolts” type=”line” align=”alignright” margin=”5px 20px” datasets=”0,0,0,0,0,0,0,0,0,0,0,0,20,230,621,977,1200,1594,1732,1678,1570,1472,1357,1248,1151,1073,1072,1087,1092,1095,1096,1097,1099,1101,1102,1102,1103,1104,1104,1105,1106,1107,1108,1108,1109,1110,1111,1111,1111,1111,1110,1109,1108,1107,1107,1108 next 133,487,708,868,991,976,921,869,825,777,707,648,654,683,738,795,822,842,859,874,888,900,911,922,931,939,947,954,960,965,970,974,978,980,983,985,989,992,995,998,998,998,998,998,1004,1009,1011,1006,970,929,891,864,887,929,976,1020 next 0,0,0,0,0,0,0,0,0,0,1196,1129,1018,1048,1055,1058,1064,1070,1077,1083,1085,1086,1092,1099,1108,1119,1130,1141,1151,1160,1169,1178,1185,1191,1197,1202,1207,1211,1216,1220,1226,1231,1237,1242,1245,1248,1251,1254,1257,1261,1265,1268,1272,1275,1278,1281″ labels=”2.25,,,3.00,,,,4.00,,,,5.00,,,,6.00,,,,7.00,,,,8.00,,,,9.00,,,,10.00,,,,11.00,,,,12.00,,,,13.00,,,,14.00,,,,15.00,,,,16.00″ width=100% scaleFontColor=#FFF fillopacity=”0.0″ colors=”#FF3300,#66FF33,#3399FF”]

4W LED  5W LED  7.5W LED

### Total System Efficiency

The emitter efficiency is relatively objective, but the total system efficiency includes the power supply.  I used a Daiwa SS-330W switching power supply I happened to have in stock to drive the system, which cost less than a dimmable transformer and matching controller, and should be significantly higher quality.  The Daiwa doesn’t seem to be easily available any more, but something like this would work well for up to 5A total load and something like this would handle as many as 40 7.5W LEDs on a single control, though the minimum 9V output has to be matched to LEDs to get satisfactory dimming. It is important not to oversize the power supply too much as switch mode supplies are only really efficient as you get close to their rated output.  An oversized switchmode power supply can be extremely inefficient.

With the Daiwa, driving 13 7.5W LEDs, I measured 8.46A at 11.94V output or 101 Watts to brightly illuminate the entire kitchen, and providing far more light than 400W of total halogen lights.  I measured the input into the power supply at 0.940A at 121.3V or 114 Watts.  That means the power supply is 88.6% efficient at 12V, which is more or less as expected for a variable output supply.

Increasing the output voltage to 14.63 Volts lowered the output current to 5.35A or 78 Watts without lowering the brightness at the installation; I measured at 168 lux at both 12.0V at 14.6V. The input current at 14.63V dropped to 0.755A or 91.6 Watts, meaning the power supply is slightly less efficient at lower output currents (as is usually the case).

• Overdriving the 12V rated LEDs to 14.63V improves plug efficiency by 20%.

At the low end, the SS-330W’s minimum output is 4.88V, which yields 12 lux at the counter or a 14x dimming ratio to 7% of maximum illumination, a far better range than is reported for standard dimmer/transformer combinations.

### Raw Data:

LED_power_graph_data

(MS Excel file, you will need the xongrid plugin to update the data as rendered in the graphs)

Posted at 02:45:36 GMT-0700

Category: FabricationphotoPositivereviewstechnology


text:
298 articles – 2275 Notices  [english version]
 HAL : in2p3-00130903, version 1
 arXiv : astro-ph/0309368
 DOI : 10.1086/378560
 Astrophysical Journal 598 (2003) 102
 New Constraints on $\Omega_M$, $\Omega_\Lambda$, and w from an Independent Set of Eleven High-Redshift Supernovae Observed with HST
 SUPERNOVAE Collaboration(s)
 (2003)
 We report measurements of $\Omega_M$, $\Omega_\Lambda$, and w from eleven supernovae at z=0.36-0.86 with high-quality lightcurves measured using WFPC-2 on the HST. This is an independent set of high-redshift supernovae that confirms previous supernova evidence for an accelerating Universe. Combined with earlier Supernova Cosmology Project data, the new supernovae yield a flat-universe measurement of the mass density $\Omega_M=0.25^{+0.07}_{-0.06}$ (statistical) $\pm0.04$ (identified systematics), or equivalently, a cosmological constant of $\Omega_\Lambda=0.75^{+0.06}_{-0.07}$ (statistical) $\pm0.04$ (identified systematics). When the supernova results are combined with independent flat-universe measurements of $\Omega_M$ from CMB and galaxy redshift distortion data, they provide a measurement of $w=-1.05^{+0.15}_{-0.20}$ (statistical) $\pm0.09$ (identified systematic), if w is assumed to be constant in time. The new data offer greatly improved color measurements of the high-redshift supernovae, and hence improved host-galaxy extinction estimates. These extinction measurements show no anomalous negative E(B-V) at high redshift. The precision of the measurements is such that it is possible to perform a host-galaxy extinction correction directly for individual supernovae without any assumptions or priors on the parent E(B-V) distribution. Our cosmological fits using full extinction corrections confirm that dark energy is required with $P(\Omega_\Lambda>0)>0.99$, a result consistent with previous and current supernova analyses which rely upon the identification of a low-extinction subset or prior assumptions concerning the intrinsic extinction distribution.
 Thème(s) : Physique/Astrophysique/Cosmologie et astrophysique extra-galactiquePlanète et Univers/Astrophysique
 Lien vers le texte intégral : http://fr.arXiv.org/abs/astro-ph/0309368
 in2p3-00130903, version 1 http://hal.in2p3.fr/in2p3-00130903 oai:hal.in2p3.fr:in2p3-00130903 Contributeur : Dominique Girod <> Soumis le : Mercredi 14 Février 2007, 12:51:44 Dernière modification le : Mercredi 14 Février 2007, 13:21:35

format: plain


text:
# Prove that $a|00\rangle+b|01\rangle+c|10\rangle+d|11\rangle$ is separable iff $ad-bc=0$

I am having trouble rigorously proving that a two qubit state $$|\psi\rangle=a|00\rangle+b|01\rangle+c|10\rangle+d|11\rangle$$ is unentangled/separable if $$ad-bc=0$$

I currently have fairly little exposure to quantum information theory. I've heard 'Schmidt decomposition' mentioned as a possible solution. Alternately, this is problem 16b in this document which also gives the hint to apply a unitary matrix $$e^{i\phi}\begin{bmatrix} c& -a \\ a^* & c^* \end{bmatrix}$$ to one of the qubits. Any solution is appreciated but the preference is for one that uses less machinery, and in particular I'd be interested to see how to use the above hint to prove the result.

EDIT

To be clear, I know that separable $$\implies ad-bc=0$$ - this is not difficult to show. It is the other implication ($$ad-bc=0\implies$$ separable) that I am having trouble with.

• Have you tried applying the unitary? What state do you get out afterwards? Jan 28 at 16:17
• I don't know if I'm applying it correctly, but for instance $(U\otimes I) \begin{bmatrix} a\\b\\c\\d\end{bmatrix}$ gives $(0,b c e^{i \phi }-a d e^{i \phi },a e^{i \phi } a^*+c e^{i \phi } c^*,b e^{i \phi } a^*+d e^{i \phi } c^*)$ where $U$ is the matrix. Of course the 2nd entry in this vector = 0 by assumption. Jan 28 at 16:56
• In general I don't think that matrix is unitary for the $a, b, c, d$ since to be unitary $|a|^2 + |c|^2=1$ which would imply that in the state $d=b=0$, the solution presented below is valid regardless; but the premise of your question is flawed (unless we are talking about a different $a \text{and} c$ which appears to be so looking at the exercise sheet) Jan 28 at 17:14
• Yes, I noticed that problem too - hence why I said I wasn't sure if I was applying the hint correctly Jan 29 at 3:10
• As a continuation for the analogous question of whether a state in $\mathbb{C}^m \otimes \mathbb{C}^n$ is separable, it is also determinant-like conditions. Like here your answer is cut out by the polynomial equation $ad-bc=0$ which is a single 2 by 2 determinant. You can see a general proof that all these Segre varieties are determinantal and what equations replace $ad-bc=0$. Feb 3 at 20:02

Maybe this will help.

Suppose we have the state $$|\psi \rangle = a|00\rangle+b|01\rangle+c|10\rangle+d|11\rangle = \begin{pmatrix} a \\ b \\ c \\d \end{pmatrix}$$, then by definition, $$|\psi \rangle$$ is separable if and only if $$|\psi\rangle = |\phi_1\rangle \otimes |\phi_2\rangle$$ where $$|\phi_1\rangle, |\phi_2\rangle \in \mathbb{C}^2$$.

Now let $$|\phi_1 \rangle = \begin{pmatrix} x \\ y \end{pmatrix}$$ and $$|\phi_2 \rangle = \begin{pmatrix} u \\ v \end{pmatrix}$$ then we have that $$|\phi_1 \rangle \otimes |\phi_2 \rangle = \begin{pmatrix} xu\\ xv\\ yu\\yv \end{pmatrix}$$

this implies that if $$|\psi \rangle = |\phi_1\rangle \otimes |\phi_2\rangle$$ then $$\begin{pmatrix} a \\ b \\ c \\d \end{pmatrix} = \begin{pmatrix} xu\\ xv\\ yu\\yv \end{pmatrix}$$ then after a few algebra steps you should arrive at $$ad = bc$$ must be the neccesary condition for this to be true...

• Downvoter left no comment, so I was wondering what their reason may have been. The answer is a nice simple argument that avoids the use of machinery like e.g. the determinant (which is what I'd use). I think their issue may have been with direction of implication. The question asks for a proof that $|\psi\rangle$ is separable if $ad=bc$, but the answer proves that $ad=bc$ if $|\psi\rangle$ is separable. Anyway, +1 to compensate since downvote is inappropriate here. Jan 29 at 2:51
• hi, thanks for the response. The trouble I am having is not showing that $ad=bc$ is a necessary condition for separability, but rather that it is a sufficient one. Am I correct in thinking your answer shows that it is necessary rather than sufficient? Jan 29 at 3:07
• $ad = bc$ is sufficient to allow you to rewrite the vector $|\psi \rangle = |\phi_1\rangle \otimes |\phi_2 \rangle$ since you can just consider $\begin{pmatrix}a \\ b \end{pmatrix} \otimes \begin{pmatrix} 1 \\ c/a \end{pmatrix}$ or $\begin{pmatrix} a \\ b \end{pmatrix} \otimes \begin{pmatrix} 1 \\ d/b \end{pmatrix}$ or $\begin{pmatrix} c \\ d \end{pmatrix} \otimes \begin{pmatrix} a /c\\ 1 \end{pmatrix}$ or $\begin{pmatrix} c \\ d \end{pmatrix} \otimes \begin{pmatrix} b /c\\ 1 \end{pmatrix}$ . Note that since $|a|^2 + |b|^2 + |c|^2 + |d|^2 = 1$ this implies that not all of them can be 0. Jan 29 at 3:34
• Very nice! I think there is a typo in the first coordinate of the last vector: $b/c$ should be $b/d$ :-) Jan 29 at 3:38
• @AdamZalcman Yes. It should be $b/d$. Thanks! Can't seem to edit comment though. Jan 29 at 3:41

Here is a complete and elementary answer - the discussion in the comments was basically there.

Let $$|\psi\rangle=a|00\rangle + b|01\rangle + c|10\rangle + d|11\rangle$$ be given such that $$ad-bc=0$$. Additionally, we have $$\| \psi\|^2=|a|^2+|b|^2+|c|^2+|d|^2=1$$.

Now the latter condition implies that $$a,b,c,d$$ are not all zero. So WLOG assume $$a\neq 0$$.

Now we have a decomposition $$|\psi\rangle = |v\rangle\otimes |w\rangle= \left(a|0\rangle + c|1\rangle\right)\otimes\left(|0\rangle +\frac{b}{a}|1\rangle\right)=a|00\rangle+b|01\rangle+c|10\rangle+\frac{cb}{a}|11\rangle$$ where $$\frac{cb}{a}=d$$.

In this decomposition, $$\|v\|^2=|a|^2+|c|^2$$, $$\|w\|^2=1+\left|\frac{b}{a} \right|^2$$. So the vectors $$v,w$$ are not normalised in general. Hence introduce a rescaling parameter, say $$\lambda\in \mathbb{R}_{>0}$$, such that $$|\psi\rangle=\lambda |v\rangle \otimes \frac{1}{\lambda}|w\rangle =\left(\lambda a|0\rangle + \lambda c|1\rangle\right)\otimes\left(\frac{1}{\lambda}|0\rangle +\frac{b}{\lambda a}|1\rangle\right) .$$

Now note that \begin{align*} \|\lambda v\|^2 \|\frac{1}{\lambda }w\|^2 &=\|v\|^2\|w\|^2\\ &=(|a|^2+|c|^2)\left(1+\left|\frac{b}{a} \right|^2\right)\\ &=|a|^2+|b|^2+|c|^2+\left|\frac{cb}{a}\right|^2\\ &=|a|^2+|b|^2+|c|^2+|d|^2=1. \end{align*} So if we pick $$\lambda=\frac{1}{\|v\|}=\frac{1}{\sqrt{|a|^2+|c|^2}}$$, then $$\|\lambda v\|=1$$, and the above also gives $$\|\frac{1}{\lambda}w\|=1$$. This now gives a valid decomposition and shows $$|\psi\rangle$$ is separable.

(At a higher level, what we used in this last renormalisation step is actually the fact that for tensor products of Hilbert spaces, we have $$\langle v_1\otimes w_1| v_2\otimes w_2\rangle_{H\otimes K} =\langle v_1|v_2\rangle_H \cdot \langle w_1|w_2\rangle_K$$, where $$v_1,v_2 \in H$$, $$w_1,w_2\in K$$. In our case this gave $$\|\psi\|^2=\|v\|^2\|w\|^2$$. )

Analogous decompositions hold in the case that we assume $$b,c$$ or $$d$$ nonzero instead.

An easy way to see it is to notice that $$ad-bc$$ is the determinant of the matrix of coefficients of the state. This matrix is singular, and thus has rank $$1$$, if and only if it has zero determinant.

But the rank of this matrix is also equal to the number of nonzero coefficients in the Schmidt decomposition of the state.

It follows that the state has the form $$|\psi\rangle=|u\rangle|v\rangle$$ for some states $$|u\rangle$$ and $$|v\rangle$$ if and only if $$ad-bc=0$$. A pure two-qubit state is separable if and only it is a product state, hence the conclusion.


text:
Solved

# How to save outpuit from powershell into *.csv

Posted on 2011-09-08
395 Views
I have following powershell scripts, where it will scan server name from AllServers.txt, around 300 servers. Do you gusy knwo how to construct powershell to save the output into csv format ? in the same C:\ data folder ?

foreach ($svr in get-content "C:\data\AllServers.txt") {$con = "server=$svr;database=master;Integrated Security=sspi"$cmd = "SELECT SERVERPROPERTY('ProductVersion') AS Version, SERVERPROPERTY('ProductLevel') as SP"
$da = new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $con)$dt = new-object System.Data.DataTable
$da.fill($dt) | out-null
$svr$dt | Format-Table -autosize
}
0
Question by:motioneye

LVL 7

Assisted Solution

celazkon earned 166 total points
ID: 36501707
Generally, to redirect the output to file instead of scripting console, use the out-file cmdlet.
For exact usage and info about it, see:
http://technet.microsoft.com/en-us/library/ee176924.aspx
0

LVL 10

Accepted Solution

wls3 earned 167 total points
ID: 36502147
Unless you format the object to work with the Export-Csv cmdlet (Get-Help Export-Csv -Examples), the Out-File cmdlet (Get-Help Out-File -examples), mentioned above, will be your more effective approach.
0

LVL 69

Assisted Solution

Qlemo earned 167 total points
ID: 36502293
Set-Location "C:\temp\ee"
$cmd = "SELECT SERVERPROPERTY('ProductVersion') AS Version, SERVERPROPERTY('ProductLevel') as SP"$set = new-object system.data.dataset
foreach ($svr in get-content "AllServers.txt") {$constr = "server=$svr;database=master;Integrated Security=sspi" (new-object System.Data.SqlClient.SqlDataAdapter ($cmd, $constr)).Fill($set) | out-null
$obj =$set.Tables[0] | select-object @{n = 'Server'; e = {$svr}}, Version, SP }$obj | export-csv -NoTypeInformation "C:\temp\ee\AllServerVersions.csv"

Is one way to do that. \$obj would allow for further processing, if needed.
0

## Featured Post

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

### Suggested Solutions

Ever wondered why sometimes your SQL Server is slow or unresponsive with connections spiking up but by the time you go in, all is well? The following article will show you how to install and configure a SQL job that will send you email alerts includ…
A project that enables an administrator to perform actions within a user session context not just at the time of login but any time later on day(s) or week(s) later.
Familiarize people with the process of utilizing SQL Server functions from within Microsoft Access. Microsoft Access is a very powerful client/server development tool. One of the SQL Server objects that you can interact with from within Microsoft Ac…
Via a live example, show how to shrink a transaction log file down to a reasonable size.


text:
3 years ago

# Size expansions of mean field approximation: Transient and steady-state analysis

Nicolas Gast, Luca Bortolussi, Mirco Tribastone

Publication date: Available online 10 November 2018

Source: Performance Evaluation

Author(s): Nicolas Gast, Luca Bortolussi, Mirco Tribastone

##### Abstract

Mean field approximation is a powerful tool to study the performance of large stochastic systems that is known to be exact as the system’s size $N$ goes to infinity. Recently, it has been shown that, when one wants to compute expected performance metric in steady-state, this approximation can be made more accurate by adding a term $V∕N$ to the original approximation. This is called a refined mean field approximation in Nicolas Gast and Benny Van Houdt (2017).

In this paper, we improve this result in two directions. First, we show how to obtain the same result for the transient regime. Second, we provide a further refinement by expanding the term in $1∕N2$ (both for transient and steady-state regime). Our derivations are inspired by moment-closure approximation, a popular technique in theoretical biochemistry. We provide a number of examples that show: (1) that this new approximation is usable in practice for systems with up to a few tens of dimensions, and (2) that it accurately captures the transient and steady state behavior of such systems.

DOI: S0166531618302633

You might also like
Discover & Discuss Important Research

Keeping up-to-date with research can feel impossible, with papers being published faster than you'll ever be able to read them. That's where Researcher comes in: we're simplifying discovery and making important discussions happen. With over 19,000 sources, including peer-reviewed journals, preprints, blogs, universities, podcasts and Live events across 10 research areas, you'll never miss what's important to you. It's like social media, but better. Oh, and we should mention - it's free.

Researcher displays publicly available abstracts and doesn’t host any full article content. If the content is open access, we will direct clicks from the abstracts to the publisher website and display the PDF copy on our platform. Clicks to view the full text will be directed to the publisher website, where only users with subscriptions or access through their institution are able to view the full article.


text:
# Apache MXNet - Python API Module

Apache MXNet’s module API is like a FeedForward model and it is easier to compose similar to Torch module. It consists of following classes −

## BaseModule([logger])

It represents the base class of a module. A module can be thought of as computation component or computation machine. The job of a module is to execute forward and backward passes. It also updates parameters in a model.

### Methods

Following table shows the methods consisted in BaseModule class

This method will get states from all devices
Methods Definition
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
fit(train_data[, eval_data, eval_metric, …]) This method trains the module parameters.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
forward_backward(data_batch) It is a convenient function, as name implies, that calls both forward and backward.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
get_states([merge_multi_context])
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
iter_predict(eval_data[, num_batch, reset, …]) This method will iterate over predictions.
predict(eval_data[, num_batch, …]) It will run the prediction and collects the outputs as well.
prepare(data_batch[, sparse_row_id_fn]) The operator prepares the module for processing a given data batch.
save_params(fname) As name specifies, this function will save the model parameters to file.
score(eval_data, eval_metric[, num_batch, …]) It runs the prediction on eval_data and also evaluates the performance according to the given eval_metric.
set_params(arg_params, aux_params[, …]) This method will assign the parameter and aux state values.
set_states([states, value]) This method, as name implies, sets value for states.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It set up the buckets and binds the executor for the default bucket key. This method represents the binding for a BucketingModule.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will get the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will get outputs from the previous forward computation.
get_params() It gets the current parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
get_states([merge_multi_context]) This method will get states from all devices.
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
load(prefix, epoch[, sym_gen, …]) This method will create a model from the previously saved checkpoint.
load_dict([sym_dict, sym_gen, …]) This method will create a model from a dictionary (dict) mapping bucket_key to symbols. It also shares arg_params and aux_params.
prepare(data_batch[, sparse_row_id_fn]) The operator prepares the module for processing a given data batch.
save_checkpoint(prefix, epoch[, remove_amp_cast]) This method, as name implies, saves the current progress to the checkpoint for all buckets in BucketingModule. It is recommended to use mx.callback.module_checkpoint as epoch_end_callback to save during training.
set_params(arg_params, aux_params[,…]) As name specifies, this function will assign parameters and aux state values.
set_states([states, value]) This method, as name implies, sets value for states.
switch_bucket(bucket_key, data_shapes[, …]) It will switche to a different bucket.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

### Attributes

Following table shows the attributes consisted in the methods of BaseModule class −

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
symbol As name specified, this attribute gets the symbol associated with this module.

data_shapes: You can refer the link available at https://mxnet.apache.org for details. output_shapes: More

## BucketingModule(sym_gen[…])

It represents the Bucketingmodule class of a Module which helps to deal efficiently with varying length inputs.

### Methods

Following table shows the methods consisted in BucketingModule class

### Attributes

Following table shows the attributes consisted in the methods of BaseModule class

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
Symbol As name specified, this attribute gets the symbol associated with this module.

## Module(symbol[,data_names, label_names,…])

It represents a basic module that wrap a symbol.

### Methods

Following table shows the methods consisted in Module class

Methods Definition
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
borrow_optimizer(shared_module) As name implies, this method will borrow the optimizer from a shared module.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
get_states([merge_multi_context]) This method will get states from all devices
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
load(prefix, epoch[, sym_gen, …]) This method will create a model from the previously saved checkpoint.
load_optimizer_states(fname) This method will load an optimizer i.e. the updater state from a file.
prepare(data_batch[, sparse_row_id_fn]) The operator prepares the module for processing a given data batch.
reshape(data_shapes[, label_shapes]) This method, as name implies, reshape the module for new input shapes.
save_checkpoint(prefix, epoch[, …]) It saves the current progress to checkpoint.
save_optimizer_states(fname) This method saves the optimizer or the updater state to a file.
set_params(arg_params, aux_params[,…]) As name specifies, this function will assign parameters and aux state values.
set_states([states, value]) This method, as name implies, sets value for states.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

### Attributes

Following table shows the attributes consisted in the methods of Module class

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
label_names It consists of the list of names for labels required by this module.

data_shapes: Visit the link https://mxnet.apache.org/api/python/docs/api/module for further details.

output_shapes: The link given herewith https://mxnet.apache.org/api/python/docs/api/module/index.html will offer other important information.

## PythonLossModule([name,data_names,…])

The base of this class is mxnet.module.python_module.PythonModule. PythonLossModule class is a convenient module class which implements all or many of the module APIs as empty functions.

### Methods

Following table shows the methods consisted in PythonLossModule class:

Methods Definition
backward([out_grads]) As name implies this method implements the backward computation.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
install_monitor(mon) This method will install monitor on all executors.

## PythonModule([data_names,label_names…])

The base of this class is mxnet.module.base_module.BaseModule. PythonModule class also is a convenient module class which implements all or many of the module APIs as empty functions.

### Methods

Following table shows the methods consisted in PythonModule class −

Methods Definition
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

### Attributes

Following table shows the attributes consisted in the methods of PythonModule class −

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.

output_shapes − For more details, visit the link available at https://mxnet.apache.org

## SequentialModule([logger])

The base of this class is mxnet.module.base_module.BaseModule. SequentialModule class also is a container module that can chain more than two (multiple) modules together.

### Methods

Following table shows the methods consisted in SequentialModule class

Methods Definition
add(module, **kwargs) This is most important function of this class. It adds a module to the chain.
backward([out_grads]) As name implies this method implements the backward computation.
bind(data_shapes[, label_shapes, …]) It binds the symbols to construct executors and it is necessary before one can perform computation with the module.
forward(data_batch[, is_train]) As name implies this method implements the Forward computation. This method supports data batches with various shapes like different batch sizes or different image sizes.
get_input_grads([merge_multi_context]) This method will gets the gradients to the inputs which is computed in the previous backward computation.
get_outputs([merge_multi_context]) As name implies, this method will gets outputs of the previous forward computation.
get_params() It gets the parameters especially those which are potentially copies of the actual parameters used to do computation on the device.
init_optimizer([kvstore, optimizer, …]) This method installs and initialize the optimizers. It also initializes kvstore for distribute training.
init_params([initializer, arg_params, …]) As name implies, this method will initialize the parameters and auxiliary states.
install_monitor(mon) This method will install monitor on all executors.
update() This method updates the given parameters according to the installed optimizer. It also updates the gradients computed in the previous forward-backward batch.
update_metric(eval_metric, labels[, pre_sliced]) This method, as name implies, evaluates and accumulates the evaluation metric on outputs of the last forward computation.

### Attributes

Following table shows the attributes consisted in the methods of BaseModule class −

Attributes Definition
data_names It consists of the list of names for data required by this module.
data_shapes It consists of the list of (name, shape) pairs specifying the data inputs to this module.
label_shapes It shows the list of (name, shape) pairs specifying the label inputs to this module.
output_names It consists of the list of names for the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.
output_shapes It consists of the list of (name, shape) pairs specifying the outputs of this module.

data_shapes − The link given herewith https://mxnet.apache.org will help you in understanding the attribute in much detail.

### Implementation Examples

In the example below, we are going create a mxnet module.

import mxnet as mx
input_data = mx.symbol.Variable('input_data')
f_connected1 = mx.symbol.FullyConnected(data, name='f_connected1', num_hidden=128)
activation_1 = mx.symbol.Activation(f_connected1, name='relu1', act_type="relu")
f_connected2 = mx.symbol.FullyConnected(activation_1, name = 'f_connected2', num_hidden = 64)
activation_2 = mx.symbol.Activation(f_connected2, name='relu2',
act_type="relu")
f_connected3 = mx.symbol.FullyConnected(activation_2, name='fc3', num_hidden=10)
out = mx.symbol.SoftmaxOutput(f_connected3, name = 'softmax')
mod = mx.mod.Module(out)
print(out)

Output

The output is mentioned below −

<Symbol softmax>

Example

print(mod)

Output

The output is shown below −

<mxnet.module.module.Module object at 0x00000123A9892F28>

In this example below, we will be implementing forward computation

import mxnet as mx
from collections import namedtuple
Batch = namedtuple('Batch', ['data'])
data = mx.sym.Variable('data')
out = data * 2
mod = mx.mod.Module(symbol=out, label_names=None)
mod.bind(data_shapes=[('data', (1, 10))])
mod.init_params()
data1 = [mx.nd.ones((1, 10))]
mod.forward(Batch(data1))
print (mod.get_outputs()[0].asnumpy())

Output

When you execute the above code, you should see the following output −

[[2. 2. 2. 2. 2. 2. 2. 2. 2. 2.]]

Example

data2 = [mx.nd.ones((3, 5))]

mod.forward(Batch(data2))
print (mod.get_outputs()[0].asnumpy())

Output

Given below is the output of the code −

[[2. 2. 2. 2. 2.]
[2. 2. 2. 2. 2.]
[2. 2. 2. 2. 2.]]


text:
# Nurikabe: Sixes

This is a Nurikabe puzzle. The goal is to paint some cells black so that the resulting grid satisfies the rules1 of Nurikabe:

• Numbered cells are white.
• White cells are divided into regions, all of which contain exactly one number. The number indicates how many white cells there are in that region.
• Regions of white cells cannot be adjacent to one another, but they may touch at a corner.
• Black cells must all be orthogonally connected.
• There are no groups of black cells that form a $$2\times2$$ square anywhere in the grid.

1 Paraphrased from the original rules on Nikoli

## 1 Answer

First, any two numbered cells that are one space apart must be separated by a black cell, which gives us the top-right, and given the "all black cells must be connected" condition one straightforwardly (and stair-wise) gets to

Again, from here the continuation still follows from the same rule applied to the bottom-right, until

Now note that the orange square below cannot be white, it's too far away from any numbered cell, and therefore the red one nearby cannot be black, because of the 2x2 rule. Moreover, there is no way the red cell can be connected to another 6 than the one on its right, because the one on its right would not have 6 friends.

We therefore get this, right after a second application of the 2x2 rule:

From here, there is only one way to connect the black pieces while leaving enough space for the top 6 to have friends:

• You beat me to it. I've played quite a few Nurikabes, and this one was not very hard but good fun. I must confess that at the point in your explanation with the orange square, I just saw the rest of solution without reasoning it through exactly, just by knowing the black parts had to be connected together around the remaining islands. – Jaap Scherphuis Sep 25 '19 at 21:23
• @JaapScherphuis It was quite fun indeed! I forbid myself to "see" things, as I was scolded by Deusovi once for doing that :) – Arnaud Mortier Sep 25 '19 at 21:26


text:
# Recurrence relation, error in generating function - where did I go wrong?

This is the homework. If you are interested in precise formulation, it is as follows: write a recurrence relation and a generating function that would generate a sequence of trites (elements of a set $\{0, 1, 2\}$), where subsequences of 01 and 00 are not allowed.

I've recognized this as being the following recurrence relation: $f(n) = 2f(n - 1) + f(n - 2)$. I've built a model of it and tested, it looks like this is correct. Now, below is my effort at writing a generating function, which does not give me the expected result, but I cannot find the error.

\begin{align*} a_n = 2a_{n-1} + a_{n-2}\\ s^2 = 2s + 1\\ s^2 - 2s - 1 = 0\\ \end{align*}

\begin{align*} s = \frac{ 2 \pm \sqrt{ 4 + 4 } } { 2 }\\ s_0 = \frac{ 2 + 2 \sqrt{ 2 } } { 2 } = 1 + \sqrt{ 2 }\\ s_1 = \frac{ 2 - 2 \sqrt{ 2 } } { 2 } = 1 - \sqrt{ 2 }\\ \end{align*}

\begin{align*} a_n = \alpha s_0^n + \beta s_1^n\\ a_0 = \alpha + \beta = 1\\ a_1 = (1 + \sqrt{ 2 }) \alpha + (1 - \sqrt{ 2 }) \beta = 3\\ \alpha = 1 - \beta\\ (1 + \sqrt{ 2 }) (1 - \beta) + (1 - \sqrt{ 2 }) \beta = 3\\ (1 + \sqrt{ 2 }) - (1 + \sqrt{ 2 }) \beta + (1 - \sqrt{ 2 }) \beta = 3\\ \beta ((1 - \sqrt{ 2 }) - (1 + \sqrt{ 2 })) = 3 - (1 + \sqrt{ 2 })\\ \beta = \frac{ 3 - (1 + \sqrt{ 2 }) } { (1 - \sqrt{ 2 }) - (1 + \sqrt{ 2 }) }\\ \beta = \frac{ 3 - (1 + \sqrt{ 2 }) } { 1 - \sqrt{ 2 } - 1 - \sqrt{ 2 } }\\ \beta = \frac{ 2 - \sqrt{ 2 } } { -2 \sqrt{ 2 } }\\ \alpha = 1 - \frac{ 2 - \sqrt{ 2 } } { -2 \sqrt{ 2 } }\\ \alpha = \frac{ -2 \sqrt{ 2 } - 2 - \sqrt{ 2 } } { -2 \sqrt{ 2 } }\\ \alpha = \frac{ -3 \sqrt{ 2 } - 2 } { -2 \sqrt{ 2 } }\\ \end{align*}

\begin{align*} a_n = \frac{ -3 \sqrt{ 2 } - 2 } { -2 \sqrt{ 2 } } \times (1 + \sqrt{ 2 }) + \frac{ 2 - \sqrt{ 2 } } { -2 \sqrt{ 2 } } \times (1 - \sqrt{ 2 })\\ a_n = \frac{ (-3 \sqrt{ 2 } - 2)(1 + \sqrt{ 2 })^n + (2 - \sqrt{ 2 })(1 - \sqrt{ 2 })^n }{ -2 \sqrt{ 2 } }\\ a_n = \frac{ (-3 \sqrt{ 2 } - \sqrt{ 2 } \sqrt{ 2 })(1 + \sqrt{ 2 })^n + (\sqrt{ 2 } \sqrt{ 2 } - \sqrt{ 2 })(1 - \sqrt{ 2 })^n } { -2 \sqrt{ 2 } }\\ a_n = \frac{ \sqrt{ 2 } (-3 - \sqrt{ 2 })(1 + \sqrt{ 2 })^n + \sqrt{ 2 } (\sqrt{ 2 } - 1)(1 - \sqrt{ 2 })^n } { -2 \sqrt{ 2 } }\\ a_n = \frac{ \sqrt{ 2 } ((-3 - \sqrt{ 2 })(1 + \sqrt{ 2 })^n + (\sqrt{ 2 } - 1)(1 - \sqrt{ 2 })^n) } { -2 \sqrt{ 2 } }\\ a_n = \frac{ (-3 - \sqrt{ 2 })(1 + \sqrt{ 2 })^n + (\sqrt{ 2 } - 1)(1 - \sqrt{ 2 })^n } { -2 } \end{align*}

Sorry, my TeX-fu isn't strong enough to make this look good. Feel free to edit it to make ti look more comprehensible.

You may find the model to test my calculations here: http://pastebin.com/R1aRmeL7

-
A generating function is a function $a_0+a_1x+a_2x^2+\dots$. You don't seem to be heading in the direction of a generating function at all; rather, you are finding a formula for $a_n$, which is a different question. –  Gerry Myerson Jun 30 '12 at 11:05
There is a sign error in the lign where you express $\alpha$ as a single fraction (with denominator $-2\sqrt2$). As a result, the term $-3\sqrt2$ in the numerator should read $-\sqrt2$. –  Did Jun 30 '12 at 11:07
@GerryMyerson sorry, I'm translating the task from another language, that was a mistranslation on my side. Then I don't need the generating function but (... that other thing) which given $n$ returns the number of sequences of length $n$ it is possible to generate while not solving for $n-1, n-2, ... n-(n-1), 0$. –  wvxvw Jun 30 '12 at 11:33
@did Oh, ok... thanks! –  wvxvw Jun 30 '12 at 11:34
You seem to like minus signs, I don't. Example: $\alpha = 1 - \frac{ 2 - \sqrt{ 2 } } { -2 \sqrt{ 2 } }$ and the line that follows. It is better to automatically convert to $\alpha = 1 + \frac{ 2 - \sqrt{ 2 } } {2 \sqrt{ 2 } }$. Also, solving the system of linear equations by substitution is less structured than elimination. –  André Nicolas Jun 30 '12 at 14:01

You could have made things a little easier for yourself after you found $s_0=1+\sqrt2$ and $s_1=1-\sqrt2$, starting with finding $\alpha$ and $\beta$. You have $\alpha+\beta=1$ and $(1+\sqrt2)\alpha+(1-\sqrt2)\beta=3$. The latter can be written $(\alpha+\beta)+\sqrt2(\alpha-\beta)=3$, and since $\alpha+\beta=1$, this simplifies immediately to give you the system
\left\{\begin{align*} &\alpha+\beta=1\\ &\alpha-\beta=\sqrt2\;. \end{align*}\right.
Clearly $2\alpha=1+\sqrt2=s_0$ and $2\beta=1-\sqrt2=s_1$, so the general term is $$a_n=\frac12\left(s_0^{n+1}+s_1^{n+1}\right)\;,$$ or, if you prefer,
$$a_n=\frac12\left((1+\sqrt2)^{n+1}+(1-\sqrt2)^{n+1}\right)\;.$$


text:
# Implementing a Random Picker by Supplying an Outcome Table (Similar to a Drop Table)

Whilst trying to vary the draw color of some randomly positioned sprites, I really wanted to be able to just supple a table with 2 columns (colour and chance of picking) to some helper function that would do all calculations for me.

Ideally, I want to be able to do something like the follow (semi-pseudocode) :

var table = new object[]
{
Color.White, 0.2, // Chance: 20%
Color.Red, 0.2, // Chance: 20%
Color.Blue, 0.5, // Chance: 50%
Color.Black, 0, // Default, i.e. remaining chance (10%)
};

Color newColor = Pick<Color>.AtRandom(table); // Apologies if this is very wrong!

I'm fairly confident I could get something to work within the class - my issue is deciding on the correct way to supply the Pick class the chances for each possible selection. I don't want this to be specifically for Color - I want to be able to select between any type of struct / instance of a class. The last line to me doesn't seem 'possible' as in I am unsure whether or not you would require an instance of Pick or if this can be done using a static class.

The use of object[] seems like a hackerish way to implement this. It's more streamlined to use a Tuple however the chance table becomes extremely verbose when every item in the array would be along the lines of Tuple.Create<Color, double>(Color.Red, 0.2), etc.

What is the most sensible / expandable way to do this?

If you need to create an instance of the table anyway, you might as well build that into your Pick class. That way you can get type safety throughout, no hacky object[] business and boxing/unboxing.

eg.

public class PickTable<T>
{
List<T> values;
List<double> weights

public PickTable(int capacity)
{
values = new List<T>(capacity);
weights = new List<double>(capacity);
}

public void Add(T value, double weight)
{
// Might want to track total weight here, or do reweighting as necessary
// (eg. for 0 weight = remainder)
}

public T AtRandom()
{
// Random selection logic goes here
return values[selectedIndex];
}
}

Using it is then just as concise as your version:

var colorTable = new PickTable<Color>(4);

• Thanks - this will certainly get me started. I going to expand on it and have a .FromFile() method so I can write it in the form I want! – Joe Shanahan Dec 23 '13 at 14:17


text:
## tend

[see also: converge, approach]

Clearly, $F_n$ tends to zero as $n\to\infty$.

Observe that as $I$ becomes smaller, $d(I,f)$ tends to 1.

Go to the list of words starting with: a b c d e f g h i j k l m n o p q r s t u v w y z


text:
## How to set up a mythtv backend on a headless raspberry pi 2

First install the raspberry pi 2 debian Jessie distribution found here. This has packages optimised for the raspberry pi chipset, but is otherwise the usual Debian jessie. Set up the system in the usual way so you have a user account. Make sure you attach a decent sized USB hard disk for storing the mythtv recordings....

## Encoding H.264 to stream to PS3 with mythtv

This took me ages to figure out and I couldn't find any useful information. To rip a DVD and have it available to stream to my PS3 using mythtv backend, I used Handbrake on Linux. H.264 encoding gives by far the best quality per bit, but getting the Playstation to stream such things is a...

Chris and I have been discussing the correct way to transform likelihood functions with different parameterisations of a dataset. This is something that keeps coming up so I thought I'd take a few notes on it. We started off with a complex $N$-dimensional data vector, which I'll call , which has a covariance matrix...


text:
# Unprovable statements S where the only way to prove S is to assume S

Motivation: Incompleteness (and various independence statements) is about unprovable statements. One natural way to make an unprovable statement provable is to assume it as a new axiom. But this feels like cheating, so people often look for "natural" axioms to add that will imply their favorite unprovable statement. The question is whether there are any statements that are unprovable in a theory where essentially the only way to get a theory which proves the statement is to assume it.

Question: Let $T$ be a theory (e.g. ZFC). Say a statement $S$ is "minimally unprovable" in/over $T$ if $S$ is not provable from $T$ (and neither is its negation), and such that if $R$ is any statement that is provable from $T \cup \{S\}$ but not provable from $T$ alone, then there is a proof in $T$ that $R$ is equivalent to $S$.

1. Does every (sufficiently powerful?) theory have minimally unprovable statements?

2. Are there examples of a(n interesting) theory $T$ together with a minimally unprovable statement $S$?

-
Perhaps something similar to $PA+\lbrace Con^n(PA)|n \gt 0\rbrace$ as the theory, and $\bigwedge_n Con^n(PA)$ as a minimally unprovable statement? –  David Roberts Nov 8 '12 at 0:02
I'm puzzled by the connection between the question and the motivation. What would you call a statement $X$ with neither $X$ nor its negation provable from $T$ so that for any statement $Y$ if $T+Y$ proves $X$, then $T+X$ proves $Y$? Is this farther from your motivation than your notion of "minimally unprovable?" Your notion is that of a statement so weak it can only prove things equivalent to itself. I think the alternative notion is of a statement that is so weak that nothing strictly weaker can prove it. –  Matt Brin Nov 8 '12 at 1:00
Well, it pays to listen to the logician/set theorist, rather than me :-) I did think it unlikely, but that is easy to say after the fact. –  David Roberts Nov 8 '12 at 1:20
This is related in some way - mathoverflow.net/questions/26411/… –  François G. Dorais Nov 8 '12 at 2:17
See also the answer to this queston mathoverflow.net/questions/84074/…, which shares some features with the queston and answer here. –  Joel David Hamkins Nov 8 '12 at 2:38

The following is essentially Joel's answer and also essentially the last part of Francois's answer, but its "look and feel" seems different enough to make it worth pointing out. The main point is that, if $S$ is minimally unprovable over $T$ then $T\cup\{\neg S\}$ is consistent and complete. Consistency is just your requirement that $S$ is not provable from $T$.

To establish completeness, suppose $Y$ is a sentence that is not provable from $T\cup\{\neg S\}$; I'll show that $\neg Y$ must be provable from $T\cup\{\neg S\}$. The assumption means that $(\neg S)\to Y$ isn't provable from $T$, and therefore neither is its contrapositive $(\neg Y)\to S$. Therefore neither is the (propositionally equivalent) formula $(S\lor\neg Y)\to S$. Now apply the main clause in the definition of minimally unprovable, with $S\lor\neg Y$ in the role of $R$. We've just seen that the conclusion of that clause fails, so one of the hypotheses must fail. The first hypothesis says that $R$ is provable from $T\cup\{S\}$, which is clearly true because of the disjunct $S$ in $R$. So the second hypothesis must fail; that is, $R$ must be provable from $T$. But then, by propositional logic, $\neg Y$ is provable from $T\cup\{\neg S\}$, as claimed.

Now if $T$ is recursively axiomatizable and contains enough arithmetic, then $T\cup\{\neg S\}$ has the same properties and, by Rosser's improvement of Goedel's second incompleteness theorem, it cannot be both consistent and complete. Therefore, $S$ cannot be minimally unprovable over $T$.

-
Thank you for this very clear answer... –  Joël Nov 13 '12 at 18:46

Theorem. There are no minimally unprovable statements, over any computably axiomatizable theory $T$ interpreting basic arithmetic.

Proof. Suppose that $S$ is not provable in $T$, that $T+S$ is consistent, that $T$ is computably axiomatizable and that $T$ contains (an interpretation of) PA or some other sufficient arithmetic theory. Let $R$ be the assertion $$\text{if the Rosser sentence of }T+\neg S\text{ holds, then }S.$$ Note that $T+S$ proves $R$ easily. So $T+R$ is consistent. Meanwhile, $T$ does not prove $R$, since if it did, then $T+\neg S$ plus the Rosser sentence of $T+\neg S$ would prove $S$, contradicting Rosser's theorem that this theory is consistent. Finally, $T+R$ does not prove $S$, since it is consistent with $T+\neg S$ that the Rosser sentence of $T+\neg S$ fails, so it is consistent with $T+R$ that $S$ fails. QED

If you weaken the hypotheses, then you can have theories with minimally unprovable statements. For example, let $T$ be the theory of true arithmetic, plus the assertion that a new constant symbol $c$ is either $0$ or $1$. The assertion $c=0$ is not provable in $T$, but it is minimally unprovable, since if $T+R$ does not settle the value of $c$, then it has the same models as $T$.

-
$T$ needs to be recursively axiomatizable? –  François G. Dorais Nov 8 '12 at 2:17
Yes, you are right! I'll edit. –  Joel David Hamkins Nov 8 '12 at 2:26
I edited to fix. Also, if you drop the c.e. requirement, then there can be minimally nonprovable assertions. –  Joel David Hamkins Nov 8 '12 at 2:36

Here is a classical example of a theory that has minimal unprovable statements. Let $T$ be the theory of (nontrivial) dense linear orders: $$\exists x \exists y (x \lt y), \qquad \forall x (x \not\lt x),$$ $$\forall x \forall y(x \neq y \to x \lt y \lor y \lt x),$$ $$\forall x \forall y \forall z (x \lt y \land y \lt z \to x \lt z),$$ $$\forall x \forall y (x \lt y \to \exists z(x \lt z \land z \lt y)).$$ It is well known that $T$ has four completions obtained by adding any one of the four axioms $\min \land \max$, $\min \land \lnot\max$, $\lnot\min \land \max$, $\lnot \min \land \lnot \max$, where $\min$ is $$\exists x \forall y (x = y \lor x \lt y)$$ and $\max$ is $$\exists x \forall y (x = y \lor y \lt x).$$ These four sentences are maximally unprovable statements over $T$ and therefore their negations are all minimally unprovable. In other words, $\min \lor \max$, $\min \lor \lnot\max$, $\lnot\min \lor \max$, $\lnot \min \lor \lnot \max$ are all minimally unprovable sentences over $T$.

In general, any coatom in the Lindenbaum–Tarski algebra of an incomplete theory will be a minimally unprovable statement for that theory. Many common theories (including PA and ZFC) have atomless Lindenbaum–Tarski algebras.

-
I forgot to exclude the trivial linear order with just one point by adding a non-triviality clause to $T$... –  François G. Dorais Nov 8 '12 at 3:13
François: Thanks! Not being a logician, I didn't know about the Lindenbaum-Tarski algebra. But if I've understood correctly, atoms in the Lindenbaum-Tarski algebra of $T$ (if any exist) are exactly the minimally unprovable statements over $T$. –  Joshua Grochow Nov 8 '12 at 15:53
Josh and François, isn't it instead the co-atoms in the Lindenbaum algebra that are the minimally unprovable statements? After all, being lower in the Lindenbaum algebra means being a stronger assertion (as falsity is at the bottom). The assertions min$\wedge$max and so on in François's answer are actually atoms, and their negations are co-atoms. Right? –  Joel David Hamkins Nov 11 '12 at 16:27
(And one should also say that when the theory is complete, it has no independent assertions, but the Lindenbaum algebra still has an atom.) –  Joel David Hamkins Nov 11 '12 at 16:28
Absolutely, Joel! Fixing now... –  François G. Dorais Nov 13 '12 at 17:47


text:
Search
Question: Extreme log2 FC from DESeq2
0
12 months ago by
wbliu0
wbliu0 wrote:

Hi there,

I've run DESeq2 on an RNA-seq data sets, with the count data came from HTSeq count. The DESeq() call was all by default.  I got some very low log2 fold changes. Two genes  are posted below:

Data (normalized counts):

        1.R 2.R 3.R 4.R 5.R 6.R 7.R 8.R 1.NR 2.NR 3.NR 4.NR
Gene1    0  0  0.0   0  0  0   0   0   0     0   40.6  0.0
Gene2    0  0 256.6  0   0   0   0   0   0    0  122.2  1.6
5.NR    6.NR    7.NR  8.NR  9.NR
Gene1     130.1   94.3      0    0.0    0.0
Gene2     0.0    475.7      0    0.4    0.8

Result:

      baseMean log2FoldChange     lfcSE      stat       pvalue         padj
<numeric>      <numeric> <numeric> <numeric>    <numeric>    <numeric>
Gene1 15.589483      -22.87922  2.649630 -8.634875 5.879134e-18 9.567115e-14
Gene2   7.355824      -21.86646  2.792237 -7.831161 4.833841e-15 3.933055e-11

My question is, should they be kept as real hits? Since the data are rather sparse and have high within-group variation/dispersion, why were they not filtered out by the cook's distance or independent filtering?

versions:  DESeq2_1.18.1, R_3.4.2.

Any explanation will be greatly appreciated!

modified 12 months ago by Michael Love20k • written 12 months ago by wbliu0
0
12 months ago by
Michael Love20k
United States
Michael Love20k wrote:

It's hard to say with just two genes why you get a certain t-statistic. The first one seems clear that you have 3/9 samples with positive counts and outside the single digit range, compared to 9/9 with zero, so it makes sense that it ends up with a small p-value. The second one looks like not promising, but it's hard to say why it gets a small p-value. I have noticed that with data that has -- within a single condition -- many zeroes mixed with large-ish counts, like >200, that using the LRT produces more conservative behavior than the Wald test. The LRT for a two group comparison can be run with test="LRT", reduced~1 when you run DESeq().