Understanding the Unpacking Operators (* and **) in Python 3.x

The * operator unpack the arguments out of a list or tuple.
> args = [3, 6]
> list(range(*args))
[3, 4, 5]

As an example, when we have a list of three arguments, we can use the * operator inside a function call to unpack it into the three arguments:

def f(a,b,c):
    print('a={},b={},c={}'.format(a,b,c))

> z = ['I','like','Python']
> f(*z)
a=I,b=like,c=Python

> z = [['I','really'],'like','Python']
> f(*z)
a=['I', 'really'],b=like,c=Python

In Python 3 it is possible to use the operator * on the left side of an assignment, allowing to specify a “catch-all” name which will be assigned a list of all items not assigned to a “regular” name:

> a, *b, c = range(5)
> a
0
> c
4
> b
[1, 2, 3]

The ** operator can be used to unpack a dictionary of arguments as a collection of keyword arguments. Calling the same function f that we defined above:

 

> d = {'c':'Python','b':'like', 'a':'I'}
> f(**d)
a=I,b=like,c=Python

and when there is a missing argument in the dictionary (‘a’ in this example),  the following error message will be printed:


> d2 = {'c':'Python','b':'like'}
> f(**d2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: f() missing 1 required positional argument: 'a'

 
Tried with: Python 3.6.5

Installing Wand (0.4) and ImageMagick v6 on Mac (macOS High Sierra v 10.13.5)

wizard

ImageMagick® is used to create, edit, compose, or convert bitmap images. It can read and write images in a variety of formats (over 200) including PNG, JPEG, GIF, HEIC, TIFF, DPX, EXR, WebP, Postscript, PDF, and SVG. Use ImageMagick to resize, flip, mirror, rotate, distort, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and Bézier curves.

Wand is a ctypes-based simple ImageMagick binding for Python, so go through the step-by-step guide on how to install it.

Let’s start by installing ImageMagic:

brew install imagemagick@6

Next, create a symbolic link, with the following command (replace <your specific 6 version> with your specific version):

ln -s /usr/local/Cellar/imagemagick@6/<your specific 6 version>/lib/libMagickWand-6.Q16.dylib /usr/local/lib/libMagickWand.dylib

In my case, it was:

ln -s /usr/local/Cellar/imagemagick@6/6.9.10-0/lib/libMagickWand-6.Q16.dylib /usr/local/lib/libMagickWand.dylib

Let’s install Wand

pip3 install Wand

Now, let’s try to run the code

from wand.image import Image

with Image(filename=sourceFullPathFilename) as img:
img.save(filename=targetFilenameFull)

Unfortunately, I got the following error message:

wand.exceptions.DelegateError: FailedToExecuteCommand `’gs’ -sstdout=%stderr -dQUIET -dSAFER -dBATCH -dNOPAUSE -dNOPROMPT -dMaxBitmap=500000000 -dAlignToPixels=0 -dGridFitTT=2 ‘-sDEVICE=pngalpha’ -dTextAlphaBits=4 -dGraphicsAlphaBits=4 ‘-r72x72’ ‘-sOutputFile=/var/folders/n7/9xyh2rj14qvf3hrmr7g9b4gm0000gp/T/magick-31607l23fY21KEi6b%d’ ‘-f/var/folders/n7/9xyh2rj14qvf3hrmr7g9b4gm0000gp/T/magick-31607_nNNZjiBBusp’ ‘-f/var/folders/n7/9xyh2rj14qvf3hrmr7g9b4gm0000gp/T/magick-31607Zfemn9tWrdiY” (1) @ error/pdf.c/InvokePDFDelegate/292
Exception ignored in: <bound method Resource.__del__ of <wand.image.Image: (empty)>>

It seems that ghostscript is not installed by default, so let’s install it:

brew install ghostscript

Now we will need to create a soft link to /usr/bin, but /usr/bin/ in OS X 10.11+ is protected.

Just follow these steps:

1. Reboot to Recovery Mode. Reboot and hold “Cmd + R” after start sound.
2. In Recovery Mode go to Utilities -> Terminal.
3. Run: csrutil disable
4. Reboot in Normal Mode.
5. Do the “sudo ln -s /usr/local/bin/gs /usr/bin/gs” in terminal.
6. Do the 1 and 2 step. In terminal enable back csrutil by run: csrutil enable

(based on this)

Now it works – Enjoy!

 

Install MongoDB Community Edition and PyMongo on OS X

  • Install Homebew, a free and open-source software package management system that simplifies the installation of software on Apple’s macOS operating system.

/usr/bin/ruby -e “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)

  • Ensure that you’re running the newest version of Homebrew and that it has the newest list of formulae available from the main repository

brew update

  • To install the MongoDB binaries, issue the following command in a system shell:

brew install mongodb

  • Create a data directory (-p create nested directories, but only if they don’t exist already)

mkdir -p ./data/db

  • Before running mongodb for the first time, ensure that the user account running mongodb has read and write permissions for the directory

sudo chmod 765 data

  • Run MongoDB

mongod –dbpath data/db

  • To stop MongoDB, press Control+C in the terminal where the mongo instance is running

Install PyMongo

pip install pymongo

  • In a Python interactive shell:

import pymongo

from pymongo import MongoClient

RoboMongo

  • Create a Connection

client = MongoClient()

  • Access Database Objects

MongoDB creates new databases implicitly upon their first use.

db = client.test

  • Query for All Documents in a Collection

cursor = db.restaurants.find()

for document in cursor: print(document)

  • Query by a Top Level Field

cursor = db.restaurants.find({“borough”: “Manhattan”})

for document in cursor: print(document)

  • Query by a Field in an Embedded Document

cursor = db.restaurants.find({“address.zipcode”: “10075”})

for document in cursor: print(document)

  • Query by a Field in an Array

cursor = db.restaurants.find({“grades.grade”: “B”})

for document in cursor: print(document)

 

  • Insert a Document

Insert a document into a collection named restaurants. The operation will create the collection if the collection does not currently exist.

result = db.restaurants.insert_one(

{

“address”: {            “street”: “2 Avenue”,            “zipcode”: “10075”,            “building”: “1480”,            “coord”: [-73.9557413, 40.7720266]        },

“borough”: “Manhattan”,

“cuisine”: “Italian”,

“grades”: [

{                “date”: datetime.strptime(“2014-10-01”, “%Y-%m-%d“),                “grade”: “A”,                “score”: 11            },

{                “date”: datetime.strptime(“2014-01-16”, “%Y-%m-%d“),                “grade”: “B”,                “score”: 17            }        ],

“name”: “Vella”,

“restaurant_id”: “41704620”

})

result.inserted_id

 

How To Find The Lag That Results In Maximum Cross-Correlation [R]

I have two time series and I want to find the lag that results in maximum correlation between the two time series. The basic problem we’re considering is the description and modeling of the relationship between these two time series.

In signal processing, cross-correlation is a measure of similarity of two series as a function of the lag of one relative to the other. This is also known as a sliding dot product or sliding inner-product.

For discrete functions, the cross-correlation is defined as:

corr

In the relationship between two time series (yt and xt), the series yt may be related to past lags of the x-series.  The sample cross correlation function (CCF) is helpful for identifying lags of the x-variable that might be useful predictors of yt.

In R, the sample CCF is defined as the set of sample correlations between xt+h and yt for h = 0, ±1, ±2, ±3, and so on.

A negative value for h is a correlation between the x-variable at a time before t and the y-variable at time t.   For instance, consider h = −2.  The CCF value would give the correlation between xt-2 and yt.

For example, let’s start with the first series, y1:

x <- seq(0,2*pi,pi/100)
length(x)
# [1] 201

y1 <- sin(x)
plot(x,y1,type="l", col = "green")

ser1

Adding series y2, with a shift of pi/2:

y2 <- sin(x+pi/2)
lines(x,y2,type="l",col="red")

ser2

Applying the cross correlation function (cff)

cv <- ccf(x = y1, y = y2, lag.max = 100, type = c("correlation"),plot = TRUE)

corr

The maximal correlation is calculated at a positive shift of the y1 series:

cor = cv$acf[,,1]
lag = cv$lag[,,1]
res = data.frame(cor,lag)
res_max = res[which.max(res$cor),]$lag
res_max
# [1] 44

Which means that maximal correlation between series y1 and series y2 is calculated between y1t+44 and y2t

corr-2

 

 

Android Studio Tips and Tricks

Editor Actions

If you’re unfamiliar with using Android Studio and the IntelliJ IDEA interface, this page provides some tips to help you get started with some of the most common tasks and productivity enhancements.

[table id=1 /]

It is always useful to visit Android Studio -> Preferences -> Keymap for the full list of shortcuts

Keymap

Tick Android Studio -> Preferences -> Editor -> General -> Appearance -> Show line number 

 

Modify the Shell Path in OSX to Access adb Through Terminal

The shell path for a user in OSX is a set of locations in the filing system whereby the user can use certain applications, commands and programs without the need to specify the full path to that command or program in the Terminal. This will work in all OSX operating systems.

You can find out whats in your path by launching Terminal in Applications/Utilities and entering:

echo $PATH

Adding in a Permanent Location

To make the new path stick permanently you need to create a .bash_profile file in your home directory and set the path there. This file control various Terminal environment preferences including the path.

nano  ~/.bash_profile

Create the .bash_profile file with a command line editor called nano

export PATH="/Users/ofir/Library/Android/sdk/platform-tools:$PATH"

Add in the above line which declares the new location /Users/ofir/Library/Android/sdk/platform-tools as well as the original path declared as $PATH.

So now when the Terminal is relaunched or a new window made and you check the the path by

echo $PATH

You will get the new path at the front followed by the default path locations, all the time

/Users/ofir/Library/Android/sdk/platform-tools:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/git/bin

 

Bluetooth Debugging on Android Wear

I’ve just bought a new ASUS ZenWatch and I’m working on a small application to extract the sensors data from the watch.

This post ( http://developer.android.com/training/wearables/apps/bt-debugging.html ) is describing the basic steps required to setup a device for debugging and to setup a debugging session. I’ve listed below some additional steps to help with the troubleshooting if things doesn’t work as planned.

Android Debug Bridge (adb)

Android Debug Bridge (adb) is a versatile command line tool that lets you communicate with an emulator instance or connected Android-powered device. It is a client-server program that includes three components:

  • A client, which runs on your development machine. You can invoke a client from a shell by issuing an adb command. Other Android tools such as DDMS also create adb clients.
  • A server, which runs as a background process on your development machine. The server manages communication between the client and the adb daemon running on an emulator or device.
  • A daemon, which runs as a background process on each emulator or device instance.

Setup Devices for Debugging


  1. Enable USB debugging on the handheld:
    • Open the Settings app and scroll to the bottom.
    • If it doesn’t have a Developer Options setting, tap About Phone (or About Tablet), scroll to the bottom, and tap the build number 7 times.
    • Go back and tap Developer Options.
    • Enable USB debugging.
  2. Enable Bluetooth debugging on the wearable:
    1. Tap the home screen twice to bring up the Wear menu.
    2. Scroll to the bottom and tap Settings.
    3. Scroll to the bottom. If there’s no Developer Options item, tap About, and then tap the build number 7 times.
    4. Tap the Developer Options item.
    5. Enable Debug over Bluetooth.

Set Up a Debugging Session


  • On the handheld, open the Android Wear companion app.
  • Tap the menu on the top right and select Settings.
  • Enable Debugging over Bluetooth. You should see a tiny status summary appear under the option:
Host: disconnected
Target: connected
  • Connect the handheld to your machine over USB.
  • In the Android Studio open the Terminal (Alt+F12 Windows) and run:
    adb forward tcp:4444 localabstract:/adb-hub
    adb connect localhost:4444

    Note: You can use any available port that you have access to.

    • adb forward forwards socket connections from a specified local port to a specified remote port on the emulator/device instance. Map a socket through the phone that is paired to your Wear device.
    • adb connect will connect to the device
  • A message will appear on the phone to allow Wear Debugging, press OK

WearDebugger

  • In the Android Wear companion app, you should see the status change to:
Host: connected
Target: connected

Well, this is the happy flow.

While debugging I’ve encountered some issues, especially after disconnecting the phone from the USB cable.

First, print a list of all attached emulator/device instances:

adb devices
List of devices attached
LGD855bxxxxx  device

In this case you can see only the phone and not the watch, so you will need to execute

adb forward tcp:4444 localabstract:/adb-hub
adb connect localhost:4444

In other cases you may get the error message

error: more than one device/emulator

You may need to terminates the adb server process:

adb kill-server

Now you can execute the adb forward and adb connect commands

adb forward tcp:4444 localabstract:/adb-hub
adb connect localhost:4444

If you get an error message

error: device unauthorized.

Try revoking the permissions on the device – Developer options -> Revoke USB debugging authorizations. Then plug the device in and accept it again.

 

Implementation of the BlockingQueue Interface That Works Properly When Accessed via Multiple Threads

A BlockingQueue is typically used to have on thread produce objects, which another thread consumes. Here is a diagram that illustrates this principle:

blocking-queue
A BlockingQueue with one thread putting into it, and another thread taking from it.

The producing thread will keep producing new objects and insert them into the queue, until the queue reaches some upper bound on what it can contain. It’s limit, in other words. If the blocking queue reaches its upper limit, the producing thread is blocked while trying to insert the new object. It remains blocked until a consuming thread takes an object out of the queue.

The consuming thread keeps taking objects out of the blocking queue, and processes them. If the consuming thread tries to take an object out of an empty queue, the consuming thread is blocked until a producing thread puts an object into the queue.

BlockingQueue methods come in four forms, with different ways of handling operations that cannot be satisfied immediately, but may be satisfied at some point in the future: one throws an exception, the second returns a special value (either null or false, depending on the operation), the third blocks the current thread indefinitely until the operation can succeed, and the fourth blocks for only a given maximum time limit before giving up. These methods are summarized in the following table:

Throws exception Special value Blocks Times out
Insert add(e) offer(e) put(e) offer(e, time, unit)
Remove remove() poll() take() poll(time, unit)
Examine element() peek() not applicable not applicable

The code below defines an implementation of the BlockingQueue interface that works properly when accessed via multiple threads since it’s synchronized properly

class SimpleBlockingQueue<E> implements BlockingQueue<E> {
    /**
     * The queue consists of a List of E's.
     */
    final private List<E> mList;

Add a new E to the end of the queue, blocking if necessary for space to become available

 public void put(E e) throws InterruptedException {
        synchronized(this) {
            if (e == null)
                throw new NullPointerException();

            // Wait until the queue is not full.
            while (isFull()) {
                // System.out.println("BLOCKING ON PUT()");
                wait();
            }

            // Add e to the ArrayList.
            mList.add(e);
            
            // Notify that the queue may have changed state, e.g., "no
            // longer empty".
            notifyAll();
        }
    }

Remove the E at the front of the queue, blocking until there’s something in the queue

public E take() throws InterruptedException {
        synchronized(this) {
            // Wait until the queue is not empty.
            while (mList.isEmpty()) {
                // System.out.println("BLOCKING ON TAKE()");
                wait();
            }

            final E e = mList.remove(0);
        
            // Notify that the queue may have changed state, e.g., "no
            // longer full".
            notifyAll();
            return e;
        }
    }

 

Using Java Thread.join() as a simple barrier synchronizer [Android]

Starts a Thread for each element in the List of input Strings and uses Thread.join() to wait for all the Threads to finish. This implementation doesn’t require any Java synchronization mechanisms other than what’s provided by Thread

 private volatile List<String> mInput = null;

/**
* The array of words to find.
*/
final String[] mWordsToFind;

/**
* The List of worker Threads that were created.
*/
private List mWorkerThreads;

Launch process for each and every string

// This List holds Threads so they can be joined when their processing is done.
mWorkerThreads = new LinkedList();

// Create and start a Thread for each element in the
// mInput.
for (int i = 0; i < mInput.size(); ++i) {
    // Each Thread performs the processing designated by
    // the processInput() method of the worker Runnable.
    Thread t = new Thread(makeTask(i));

    // Add to the List of Threads to join.
    mWorkerThreads.add(t);

    // Start the Thread to process its input in the background.
    t.start();
}

Barrier synchronization: using thread.join() to wait for the completion of all the other threads

// Barrier synchronization.
for (Thread thread : mWorkerThreads)
   try 
   {
      thread.join();
   } 
      catch (InterruptedException e) 
   {
      printDebugging("join() interrupted");
   }

If t is a Thread object whose thread is currently executing,

t.join();

causes the current thread to pause execution until t‘s thread terminates.

Overloads of join allow the programmer to specify a waiting period.

Like sleep, join responds to an interrupt by exiting with an InterruptedException.

Based on https://github.com/douglascraigschmidt/LiveLessons/blob/master/ThreadJoinTest/src/ThreadJoinTest.java

Using The Device’s Accelerometer And Magnometer To Orient A Compass [Android]

Initialization

private String TAG = "SensorCompass";

// Sensors & SensorManager
private Sensor accelerometer;
private Sensor magnetometer;
private SensorManager mSensorManager;

// Storage for Sensor readings
private float[] mGravity = null;
private float[] mGeomagnetic = null;

// Rotation around the Z axis
private double mRotationInDegress;

OnCreate

protected void onCreate(Bundle savedInstanceState) {

		// ...

		// Get a reference to the SensorManager
		mSensorManager = (SensorManager) getSystemService(SENSOR_SERVICE);

		// Get a reference to the accelerometer
		accelerometer = mSensorManager
				.getDefaultSensor(Sensor.TYPE_ACCELEROMETER);

		// Get a reference to the magnetometer
		magnetometer = mSensorManager
				.getDefaultSensor(Sensor.TYPE_MAGNETIC_FIELD);

		// Exit unless both sensors are available
		if (null == accelerometer || null == magnetometer)
			finish();

	}

Register this class as a listener for the accelerometer events and for magnetometer events:

@Override
	protected void onResume() {
		super.onResume();


		// Register for sensor updates

		mSensorManager.registerListener(this, accelerometer,
				SensorManager.SENSOR_DELAY_NORMAL);

		mSensorManager.registerListener(this, magnetometer,
				SensorManager.SENSOR_DELAY_NORMAL);
	}

The onPause method unregister this class as a listener for all sesnors

@Override
protected void onPause() {
	super.onPause();

	// Unregister all sensors
	mSensorManager.unregisterListener(this);
}

The onSensorChanged method process the incoming sensors’ events

@Override
public void onSensorChanged(SensorEvent event) {

	// Acquire accelerometer event data
	
	if (event.sensor.getType() == Sensor.TYPE_ACCELEROMETER) {

		mGravity = new float[3];
		System.arraycopy(event.values, 0, mGravity, 0, 3);

	} 
	
	// Acquire magnetometer event data
	
	else if (event.sensor.getType() == Sensor.TYPE_MAGNETIC_FIELD) {

		mGeomagnetic = new float[3];
		System.arraycopy(event.values, 0, mGeomagnetic, 0, 3);

	}

	// If we have readings from both sensors then
	// use the readings to compute the device's orientation
	// and then update the display.

	if (mGravity != null && mGeomagnetic != null) {

		float rotationMatrix[] = new float[9];

		// Users the accelerometer and magnetometer readings
		// to compute the device's rotation with respect to
		// a real world coordinate system

		boolean success = SensorManager.getRotationMatrix(rotationMatrix,
				null, mGravity, mGeomagnetic);

		if (success) {

			float orientationMatrix[] = new float[3];

			// Returns the device's orientation given
			// the rotationMatrix

			SensorManager.getOrientation(rotationMatrix, orientationMatrix);

			// Get the rotation, measured in radians, around the Z-axis
			// Note: This assumes the device is held flat and parallel
			// to the ground

			float rotationInRadians = orientationMatrix[0];

			// Convert from radians to degrees
			mRotationInDegress = Math.toDegrees(rotationInRadians);

			// Request redraw
			mCompassArrow.invalidate();

			// Reset sensor event data arrays
			mGravity = mGeomagnetic = null;

		}
	}

}