Thursday, September 22, 2016

Flask on Windows and the case of the Pip install failing

At work I have a Mac and back home I have a Linux (Fedora) and Windows 10 machines. I started a Flask project on the Mac and then put it in a Github repo which I then cloned on my Windows 10 machine. And this is where I encountered the problem of pip failing to install the modules. I tried two variations to the call:
  1. > pip install -r requirements.txt
  2. > python -m pip install -r requirements.txt
This is with Python 3 and both will spit out a "Permission Denied" error relating to your appdata/Temp folder. The contents of my requirements.txt is pretty tame.

Here's a partial.

Flask==0.11.1
Flask-RESTful==0.3.5
Flask-Webpack==0.1.0
get==0.0.0
itsdangerous==0.24

It fails right on the get module. It say something like it can access the temp folder on your Windows machine which put me on a wild goose chase because the error isn't fucking related to permissions. I figured out the fix after watching the temp folder while pip attempted to install the modules.

Apparently, the Windows pip version does something really weird with the setup.py and/or egginfo where it attempts a build on a empty setup folder for the get module causing the error. The fix is to just change the version. In this case for get==0.0.0 to get==0.0.20.

The pip install command should go off with no problems now.


Tuesday, August 30, 2016

Python zipfiles with the cloud is technologic

Working with zip files in Python is straight forward.

Write it

s3connection = self.get_s3_connection()
bucket = s3connection.get_bucket(aws_bucket_name)
k = Key(bucket)
k.key = key.name

k.get_contents_to_filename(file_path)

Cut it, Paste it, Save it

import shutil
shutil.copyfile(self.src_zipfile, self.temp_zipfile)

Load it

zf = zipfile.ZipFile(zip_file)   # turn file into a zipfile object
zf.extractall(path=unpack_path)  # extract all files to path
files = glob.glob(os.path.join(unpack_path, 'list_*.txt'))  #glob glob
for files in files:
    # to stuff to the files here

Check it

# In a test case
zf = zipfile.ZipFile(zip_file)
zf.extractall(path=unpack_path)
files = glob.glob(os.path.join(unpack_path, 'list_*.txt'))
self.assert(len(files),expected_file_count)


Quick - rewrite it

with open(some_file, 'r+') as file:
    #do file rewrite here
    file.seek(0)
    file.write('Quick!')
    file.truncate()

Now here's daftfunk - technologic

Wednesday, August 3, 2016

Tidbits on Python's virtualenv

Python being a mess that it is - *cough* Python 2 vs 3 *cough* - has a thing for virtual dev environments.

Virtual dev enviroments or virtualenv solves the mess by creating isolated dev environments. Each environment then can have it's own Python and lib versions, dependencies and even permissions and settings. Virtualenv is complimented by virtualenvwrapper.

This is all well and good, unfortunately, virtualenv + wrapper are terminal (command line) driven and I often forget the basic commands. Hence, this blog post:

Basic assumption here is that you've setup and configured virtualenv with virtualenvwrapper.

1. Make a new virtual environment with a specific version of Python: use the -p flag

mkvirutalenv -p [path-to-python] [project name]

example: mkvirtualenv -p /usr/bin/python project1

2. List all existing virtual environments; workon command without arguments also works

lsvirtualenv -b

-b flag = brief mode, disables verbose output

3. Delete or remove an existing virtual environment; only deletes environments found in the WORKON_HOME path

rmvirtualenv [env_name]

Virtual environment must be deactivated before removing.

4. Deactivate current active virtual environment

deactivate

Also:

- Command ref for virtualenvwrapper
- Ref for virtualenv