Thursday, October 27, 2016 when deepdiff is too much

Working with Python for the past 3 months coming from Java has been interesting (grossly understated).

Anyhow, I found myself needing a way to find the stuff (not just difference) between two dictionaries and a quick google led me to deepdiff by Sep Dehpour. It was good but found it a tad overkill for what I was doing. So another round of google and stackoverflow allowed me to cobble together this:

class SimpleDictDiffer(object):
    Calculate the difference between two dictionaries as:
    (1) items added
    (2) items removed
    (3) keys same in both but changed values
    (4) keys same in both and unchanged values

    def __init__(self, first_dict, second_dict):
        self.current_dict, self.past_dict = first_dict, second_dict
        self.set_current, self.set_past = set(first_dict.keys()), set(second_dict.keys())
        self.intersect = self.set_current.intersection(self.set_past)

    def added(self):
        return self.set_current - self.intersect

    def removed(self):
        return self.set_past - self.intersect

    def changed(self):
        return set(o for o in self.intersect if self.past_dict[o] != self.current_dict[o])

    def unchanged(self):
        return set(o for o in self.intersect if self.past_dict[o] == self.current_dict[o])

A github gist for this is also available.

Thursday, September 22, 2016

Flask on Windows and the case of the Pip install failing

At work I have a Mac and back home I have a Linux (Fedora) and Windows 10 machines. I started a Flask project on the Mac and then put it in a Github repo which I then cloned on my Windows 10 machine. And this is where I encountered the problem of pip failing to install the modules. I tried two variations to the call:
  1. > pip install -r requirements.txt
  2. > python -m pip install -r requirements.txt
This is with Python 3 and both will spit out a "Permission Denied" error relating to your appdata/Temp folder. The contents of my requirements.txt is pretty tame.

Here's a partial.


It fails right on the get module. It say something like it can access the temp folder on your Windows machine which put me on a wild goose chase because the error isn't fucking related to permissions. I figured out the fix after watching the temp folder while pip attempted to install the modules.

Apparently, the Windows pip version does something really weird with the and/or egginfo where it attempts a build on a empty setup folder for the get module causing the error. The fix is to just change the version. In this case for get==0.0.0 to get==0.0.20.

The pip install command should go off with no problems now.

Tuesday, August 30, 2016

Python zipfiles with the cloud is technologic

Working with zip files in Python is straight forward.

Write it

s3connection = self.get_s3_connection()
bucket = s3connection.get_bucket(aws_bucket_name)
k = Key(bucket)
k.key =


Cut it, Paste it, Save it

import shutil
shutil.copyfile(self.src_zipfile, self.temp_zipfile)

Load it

zf = zipfile.ZipFile(zip_file)   # turn file into a zipfile object
zf.extractall(path=unpack_path)  # extract all files to path
files = glob.glob(os.path.join(unpack_path, 'list_*.txt'))  #glob glob
for files in files:
    # to stuff to the files here

Check it

# In a test case
zf = zipfile.ZipFile(zip_file)
files = glob.glob(os.path.join(unpack_path, 'list_*.txt'))

Quick - rewrite it

with open(some_file, 'r+') as file:
    #do file rewrite here

Now here's daftfunk - technologic