Saturday, December 2, 2017

Dealing with global web assets on Django

Dealing with global web assets on Django, my thoughts.

Working with Django, often your default mode of thought is apps. App are like these small self-contain programs that have their own urls, views, templates and the web assets that come with templates. Web assets, I mean those css (or Sass or Less files), images and maybe some JavaScript.

But when you start thinking long term, as far as templates go, apps will prove unmanageable. You'll start thinking of breaking then into smaller reusable units while leveraging template inheritance. Which leads us to the dilemma, how do we handle the global web assets?

My first idea was just put it on the /static folder. It works but not a good idea because a standard django .gitignore will exclude the /static folder.  Another thing, /static folder is where the django-admin collectstatic command puts all files it gets from the your dependency modules. It's not uncommon to have hundreds of MB inside this folder - hence, you don't commit them into repos.

So, I had to figure out a way that allowed to me get around the .gitignore rule (which I won't edit - they are standard for a reason) with the assets. What I ended up with is

1. Putting the global, shared web assets in an /assets folder
2. Inside the /assets folder are subfolders for css, js, img, font
3. Change my django configuration to handle it.


STATIC_ROOT = str(ROOT_DIR('static'))

STATICFILES_DIRS = [
    ('projectA1', str(ROOT_DIR('assets'))),
]

STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
]


The ROOT_DIR function is just a helper function that prints out the path to static relative to the root directory.

So anyway, an example usage then is like .. href="{% static "projectA1/css/style.css" %}" which is pretty nice. Details are here.

Monday, November 6, 2017

Git aware prompts for Windows just like in Bash

At work, I use a bash terminal. It's git aware so it looks like this:

My bash prmpt

It shows the current branch and color codes the status of the repo among other things. If you want the settings, you can get the gist for it.

At home, I work with Windows. I know I can get a bash shell running on Windows but I wondered if there's a same thing for the Windows terminal.

The first thing I learn is that Windows has two terminals, the ordinary cmd.exe thing and a much more powerful thing called Powershell. Powershell can do what cmd can do and much more. Powershell is more of a platform rather than just a window to issue commands.

Which then lead me to posh-git. At default settings, it just changes your prompt to be git-aware. For example:

C:\Users\jaypax\projects\posh-git [master+0 ~1 -0 !]>

It's not exactly like my bash prompt but it's close. I might be even convinced it's better with a little tweaking.

I got to say it's pretty nice but don't take my word for it. Take it for a spin.

Monday, October 16, 2017

Doing listviews with empty lists and pullrefresh in Nativescript-ng

This was a pain in the butt to do right and I had to do a "bit" of trial and error to get this right.

All I wanted do was show a "centered" bold, large text on the screen when a list is empty and the actual list view when there's data on the list with both having support for pull to refresh action.

Googling this well probably lead you to Telerik's doc on the RadListView for pullToRefresh which I call shenanigans on because I couldn't make it to work. So we move to next idea which is Brad Martin's pullToRefresh nativescript plugin.

I did get Martin's nativescript plugin to work on my first attempt sans a bug that was a bit odd. I'm not even sure if the bug can be replicated on another machine. Anyhow, my first attempt's code looked like this:

 <PullToRefresh (refresh)="refreshList($event)" row="1">  
     <GridLayout horizontalAlignment="center" verticalAlignment="center"   
           rows="*" columns="*" *ngIf="emptyList()">  
       <Label horizontalAlignment="center" verticalAlignment="center" row="1" column="1" class="h2"   
           text="No Orders Available."></Label>  
     </GridLayout>   
     <ListView [items]="ordersList"   
          (itemTap)="onOrderTap($event)" class="list-group">  
       <ng-template let-order="item" let-odd="odd" let-even="even">  
         <GridLayout [class.odd]="odd" [class.even]="even" columns="auto, *" rows="auto">  
           <Label col="0" class="iconbkg" text=""></Label>  
           <avatar [avatardata]="order.avatar_prop"></avatar>  
           <StackLayout col="1" class="formMessage">  
             <GridLayout columns="auto, 100, *" rows="auto">  
               <Label class="h3 text-left" col="0" [text]="'Order #' + order.number"></Label>  
               <Label class="h3 text-right" col="1" [text]="order.date_placed | date:'MMMM dd yyyy'"></Label>  
             </GridLayout>                            
             <Label class="body" [text]="'City: ' + order.shipping_address.line4 + ', ' + order.shipping_address.state"></Label>  
             <Label class="body" [text]="order.lines.length + ' Item(s)'"></Label>  
           </StackLayout>  
         </GridLayout>  
       </ng-template>  
     </ListView>  
   </PullToRefresh>  


If you refer to the inner Gridlayout with the *ngIf element. The idea was to show this gridLayout when the list is empty. Don't bother with the ListView component because it will not show anything if [items] is empty.

The problem was when I do a pull to refresh, it was causing a crash in the android emulator saying that it was referencing a null view of some sort.

I eventually got it to do want I wanted. I did two things:

1. Move the empty list gridLayout into its own pullToRefresh container. Like so:

<PullToRefresh (refresh)="refreshList($event)" row="1" col="1" *ngIf="emptyList()">  
     <GridLayout horizontalAlignment="center" verticalAlignment="center" rows="*" columns="*">  
       <Label horizontalAlignment="center" verticalAlignment="center" row="1" column="1" class="h2"   
           text="No Orders Available."></Label>  
     </GridLayout>      
   </PullToRefresh>  

   <PullToRefresh (refresh)="refreshList($event)" row="1">  
     <ListView [items]="ordersList"   
          (itemTap)="onOrderTap($event)" class="list-group">
.....


2. I removed the ChangeDetectionStrategy in the @component declaration of the view class.

 With these two changes, the error no longer happened and it works perfectly.

Thursday, September 14, 2017

VS Code with code ligatures

Code ligatures are these cool things I just found out about recently that replace text to symbols. For example: '!=' turns into '!='. That pretty cool.

Now, if you're not seeing the equal symbol with a slash across it, that just means you haven't installed correct font for code ligatures - FiraCode. Click here for install instructions.

With the font installed, time to activate this for VS Code, you'll have to modify your settings.json - either your users or your workspaces settings.json.

FYI, most editors and IDEs are supported except for a few; most notable are Eclipse and Sublime which are not supported yet. Huh....

To open settings.json, from the File menu choose Prefereneces, Settings or use the keyboard shortcut Ctrl +,. Add these lines and restart VS Code.

    "editor.fontFamily": "Fira Code",
    "editor.fontSize": 14,
    "editor.fontLigatures": true

You might have to wrap the "Fira Code" font with extra apostrophes like
"'Fira Code'".
if it doesn't work.

Read up more about code ligatures.

Wednesday, August 30, 2017

Quick Letter Avatars for List in NativeScript

Letter Avatars are these elements often associated with Gmail - especially on it's Android client. So, being a programmer lacking in design skills, I decided to copy them for a Nativescript app I'm working on that had lists.

My approach to this was just to use a Label element and CSS. It's hacky but it works and in my world if it works it's not stupid.

So, we start on the component template:

 <GridLayout rows="*">  
  <ListView [items]="ordersList" (itemTap)="onOrderTap($event)" class="list-group">  
   <ng-template let-order="item" let-odd="odd" let-even="even">  
     <GridLayout [class.odd]="odd" [class.even]="even" columns="auto, *, auto">  
       <Label class="lavatar" text="VP" style="background-color: goldenrod; color: white;"></Label>  
       <StackLayout col="1" class="item-text">  
         <Label [text]="'#' + order.number"></Label>  
         <Label [text]="'Order by ' + order.guest_email"></Label>  
         <Label [text]="order.date_placed | date:'MMMM dd yyyy'"></Label>  
       </StackLayout>  
     </GridLayout>  
   </ng-template>  
  </ListView>  
 </GridLayout>   

The important thing here is the Label with the class "lavatar" which as the following rules:

.lavatar{
    text-align: center;
    font-size: 20;
    font-weight: bold;
    padding: 18;
    border-radius: 40;
    border-color: salmon;
    border-width: 0.5;
    width: 64px;
    height: 64px;
}

The border-radius is what's actually rounds the element.

Unfortunately this rule only works on "known" dimensions - width and height. For a more dynamic solution, I think you'll have to keep looking.

Settings the colors is based on the fact that CSS rules have the "cascade rule" so the final style rule applied is what's on the element. That's why the Label has a style attribute despite knowing it's in bad taste.

 Oh well.

Friday, August 11, 2017

NativeScript and that "status 200 url null" error

This error confused me while developing with NativeScript trying to do a rest api call. Without digging thru the error object, you only get: "Response with status: 200  for URL: null" for the error text. Not exactly useful.

I then had to run "tns debug android" and dig thru the error object by doing adding a "json()" method call to it. I then got a more verbose error message. Something like:

"Error: java.net.ConnectException: failed to connect to localhost/127.0.0.1 (port 8000): connect failed: ECONNREFUSED (Connection refused)"

The chrome devtools also was cryptic only saying that the rest api call "Stalled".

Ok. So I then visited StackOverflow looking for a solution. No dice. Until, I realized that I'm doing development using an android emulator which is basically a virtual machine. And being a virtual machine calling 'localhost' or 127.0.0.1 is calling itself. The address is loopback address of the vm NOT my machine. No wonder the rest api call was failing. Fortunately, the android emulator networking is setup to work with localhost development.

The solution is to just change the rest api call from 'http://localhost/api' to http://10.0.2.2/api'.

Now it works.

For the curious, my rest api server is a Django Rest Framework app running locally.





Tuesday, July 11, 2017

Splitting 40 character tokens or keys for readability with dashes

A good bit programming is about mangling text.

Python 3 has a lot of fun language features you can use to process text. Here is short Python 3 script that will split a 40 character length token (or text) into 6 segments. Of course, how many character per segment depends on the text length.


token = 'ad6d9c4e3fe09cfd24afdd62cc3705be02545272'

# double // so we don't have to do an int cast: int(len(token)/6) - Python 3 feature
chunks, chunk_size = len(token), len(token) // 6

keyed_token = [ token[i:i+chunk_size] for i in range(0, chunks, chunk_size)]

print('-'.join(keyed_token))


It should output: ad6d9c-4e3fe0-9cfd24-afdd62-cc3705-be0254-5272

Done!

Friday, June 30, 2017

Fixing CERTIFICATE_VERIFY_FAILED error on Macs with Python3

There's this annoying thing where Requests, a very popular Python library for working the internet doesn't work because of SSL certificates not being trusted.

This whole thing apparently is caused by OSX and Python3 having no certificates and can't validate SSL connections! This information is sort of hidden away in this itty bitty readme file on the certifi module which then pointed me to the readme file in python3. What. The. Fuck.

There, you'll know that for Python 3.6 on OSX, it requires a post-install step, which installs the certifi package of certificates. This is documented in the ReadMe, which you should find at /Applications/Python\ 3.6/ReadMe.rtf

The ReadMe will have you run this post-install script, which just installs certifi: 

/Applications/Python\ 3.6/Install\ Certificates.command

I should read the release notes for Python3 more closely.  

Wednesday, June 7, 2017

Dealing with outdated or abandoned pypi modules with VCS install

So I was using a small, simple but working Paypal module which unfortunately was forgotten or abandon by the original author. It was working on my Django 1.5 site up until I needed to upgrade to Django 1.10 because of the other modules. Also for security and stability reasons but that's another story for another time.

Fortunately, this Paypal module wasn't "super" broken. The errors were just a couple of lines in the templates and a function named patterns() from django.url.conf - urls.py.

This leads into a couple of solutions:

1. I could copy-pasta the code and turn it into a internal django app with the fixes.

2. I could fork it, fix and then setup by own pypi server.

3. I could find out if someone fixed this already and take it from there.

Of course, I'd pick #3 - Programmer. So, lazy. Look it up.

I was lucky to found out that someone has fixed all the issues and got it running on Django 1.10. All that's left is to fork his repo and install it into my virtual environment.

It's simple because pip already supports installing modules from a VCS.

$ pip install git+https://github.com/killertilapia/fixed-forked-project

If you need install a certain branch do this instead

$ pip install git+https://github.com/killertilapia/fixed-forked-project@branch

This also works with other VCS like Mercurial (hg+https), Bazaar (bzr+https) or for-fuck-sakes why! - SVN (svn+https).

Wednesday, May 31, 2017

Windows Python 3.6, VS2017 and the case of the missing basetsd.h

So you decided to join the Python 3 bandwagon on Windows with your new shiny Visual Studio. And then you start work on a fresh virtual environment and then you do a pip install or something. Sometimes, you'll end up with a compile error just like this:
WTF is this missing 'basetsd.h' file
You'll google this and end up with one of many stackoverflow topics stating you must install Visual Studio 2015 or some SDK.

Fuck that and save your time. You can do better because a programmer named Dimitri Janczak found a solution that works and I can confirm it works. 

The solution is as follows:

1. Install or update setuptools (> 34.0). Do note that setuptools is sort of tied with your version of Pip. So you might have to do: python -m pip install -U pip setuptools

2. The next step is to pick the correct command-line environment. What?! I know right, Visual Studio comes with multiple command prompts. 


Select the appropriate Native Tools Command prompt. Navigate to the project folder. Activate the virtual environment and do what you did last before you encountered the compile error.

If you goes to plan then you should get a successful result. 

Thursday, May 11, 2017

Using git push to deploy code to DigitalOcean

There'll be times using a continue integration server or stack is a big pain in the butt. Sometimes we just want to type git push from the terminal and the live app is updated. This is that tutorial although specific to DigitalOcean. I'll bet it will roughly work the same for other cloud services.

Assumptions
  • You have a Digital Ocean server (or droplet) running and configured
  • You have the ssh to said droplet working
So ssh to the droplet and then navigate to /var/www (or anywhere else you want to put your code) and make two directories:

$ mkdir liveapp.com && mkdir appcode.git
$ cd appcode.git

Inside the appcode.git folder we will create a "bare" git repository: git init --bare
A bare git repository doesn't contain our code but rather the internals of the .git folder. The whole point of this is to access the hooks folder within.

The hooks folder have scripts that can be run at certain steps within the git repo's lifecycle. We want a hook script to run after we "receive" a push to this repo. Hence we create a post-receive file inside the hooks folder.

$ vi post-receive

# Type this in

#!/bin/sh
git --work-tree=/var/www/liveapp.com --git-dir=/var/www/appcode.git checkout -f

# save and exit
# make the post-receive hook executable

$ chmod +x post-receive

There! We are done for the server side. We move to the our local machines.

All we actually need is need is to add a remote target in your working git repo. Assuming you added the droplet in your local ssh config then the command should be look like:

$ git remote add droplet ssh://user@ip-of-droplet/var/www/appcode.git

# add a new commit - this will not work even if you existing commits
# you must have a new commit
$ git add .
$ git commit -m "new commit"

# push to repo - could be github or bitbucket
$ git push

# push to droplet
$ git push droplet master

If all goes to plan, then our code will show in our new "git work tree" which is the liveapp.com folder inside our droplet.

A couple more thoughts:
  1. It's fairly easy to setup a sort of staging or testing folder/area with this.
  2. The post-receive script can do more things like a "restart app" or send mail to admin for notification. It's just a bash script.

Friday, May 5, 2017

Ghetto Python development with VS Code

Not everyone can afford Pycharm or Sublime or you just don't like the new Pycharm terms/license then this could be a alternative setup with Visual Studio Code.
  1. Install Visual Studio Code. It's has all the "f"s: fast, free, flexible, fowerful.
    Don't forget to install Python also.

    Tip: Install Python 3 unless you're maintaining legacy Python code then install Python 2.
  2. Pick our plugins for Python. You can install plugins or extensions from the within VS Code; Just click on the "Extensions" button on the side menu or you can download them from the "marketplace" and install them manually. Go here for details.

    Here's our list of extensions.
    1. donjayamanne.python - our primary plugin so we can work with Python. It's got almost all the things we'd expect to make for a good Python IDE; Intellisense, Code formatting, refactoring, debugging, and linting.

      EDIT: I think this has been rolled into the official Python plugin by Microsoft.

      A word on linting though, The plugin doesn't come with it. You'll have to install your linting module like say Pylint or Flake 8 separately via pip. 

    2. Indent-raindow - Trust me, you'll need this. It makes seeing the code indents easier. 

    3. Jinja - This is an optional plugin. You'll probably only need this if you're working with a lot of Jinja templates.

      Don't forget to explore the marketplace for other customisations like themes.
  3. Start coding. Don't forget to make a virtual environment because VSCode does support them on a per project bases via a settings JSON file. Same with debugging.
  4. Push to a repo. VSCode already has a git client built-in so push your code to popular repos like Github, Gitlab or Bitbucket.
Code away!

Saturday, March 25, 2017

Visual Studio 2017 Xamarin stuck if you didn't install the SDK and NDK

I ran into this bug when I was upgrading from Visual Studio 2015 to 2017. I included the Xamarin option But I didn't include the Android SDK and NDK. I removed them from the install list. I did this because I already have both installed.

The "hang" occurs then you try opening the Xamarin in the Options window. Visual Studio will stop working here so you can't set the paths for Xamarin.

So I went digging through the Xamarin logs in the AppData folder. I know where the log folder is from a previous issue with Xamarin. Xamarin is a good idea but it's buggy as fuck. So navigate to %USERPROFILE%\AppData\Local\Xamarin\Logs\15.0\ folder and see which is the latest log and open it with your favorite text editor.

Once we confirm, if it's really the missing SDK and NDK paths, we then have to add them to Window's Registry.
  1. Open regedit
  2. Navigate to Computer\HKEY_CURRENT_USER\SOFTWARE\Xamarin\VisualStudio\15.0\Android
  3. Right-click and add new String Values
    • AndroidNdkDirectory - set it to the full path to where you install the NDK 
    • AndroidSdkDirectory - set it to the full path to where you install the SDK
  4. Close and restart the computer
And we should have fixed the issue.




Monday, March 6, 2017

Dealing with SSL HandShake failure with Python Requests (_ssl.c:590)

I was working with Requests trying to talk to a REST api when I encountered this SSL HandShake error. It was fairly easy to replicate because it happens every time I tried talking to a HTTPS endpoint or resource. It didn't matter if it was a GET or POST action. I suspect it'd be the same for other actions like PATCH or DELETE.

So what's the fix? Uninstall-install then maybe an update or upgrade.
  1. I updated Python to 2.7.11. You might not need to do this if your up-to-date. 
  2. I uninstalled requests. - $ pip uninstall requests
  3. I uninstalled urllib3. - $ pip uninstall urllib3
  4. Installed crypto dependencies or libs. This is where it gets hairy.

    First I had to update my machine's openSSL library. This could range from easy to hair pulling hard. Thankfully, I had brew on my machine so it was relatively easy.

    Anyhow you'll want to see this from the terminal after the update.

    $ openssl version
    OpenSSL 0.9.8zh 14 Jan 2016
    

    After you'll need to run this command:

    $ pip install pyopenssl ndg-httpsclient pyasn1
    

     This installs the rest of the crypto dependencies.
  5. I installed urllib3 back. - $ pip install urllib3
    I also installed urllib3[secure]. - $ pip install urllib3[secure]
    You might not need the secure version but I covering all the bases.
  6.  Install requests back. - $ pip install requests
    Same deal with urllib3, you might also want to install request with security.

    $ pip install request[security]
Why this works? I suspect that there's an installation-time check for either urllib3 (or requests) for the crypto stuff that keeps things from working without the uninstall-reinstall.





Tuesday, January 24, 2017

Doing query param request in Django

Typically, I want my API endpoint url to be flat like this:

mydomain.com/shit_api/v1/person/123

That url is pretty straight forward, it's trying to get a person with an ID of 123 from shit_api. But that's not always clear cut.

I've been working on a particular API that needed a pair of optional filters that we're a pain to work out in a regular expression. And this is where I found about Django's QueryDict object. With it I can write out API urls with params:

mydomain.com/shit_api/v1/something/?filter1=value1&filter2=value2

And on the urls and views, I can handle it as such:

#urls.py
urlpatterns = [
   url(r'^parts/$', SomethingView.as_view()), name='Something'),
]
...

#views.py
class SomethingView(View):
   def get(self, request, *args, *kwargs):
      fiter1_value = request.GET.get('filter1', 'default')
      fiter2_value = request.GET.get('filter2', 'default')

      #do more here

      return Response('some response')

The thing here is that request.GET.get() method having a default value. This make it that having it on the URL it isn't a problem. So call like:

mydomain.com/shit_api/v1/something/

Will still work.

Bonus since I don't have to deal with a convoluted regular expression. Yehey!

Friday, January 13, 2017

Tricks with Django URLs and knowing you suck doing RE

RE or regular expressions and I don't really see eye to eye. But as a programmer, I have to work with them. So it's awkward like your ex works in the same office as you. But lucky for us, we just stick to a couple of tricks to make it a bit palatable.

Numbers in URL

You'll need to use a \d+ in the pattern.

ex: (r’^status/(\d+)$’, ‘myapp.backbone_view’)

This would work on /api/status/1

Dealing with spaces

I found two ways to do this but then found out that they are the same: [\w|\W]+ AND .+

Yes that's a dot.

ex: (r’^status/([\w|\W]+)$’, ‘myapp.backbone_view’) OR
(r’^status/(.+)$’, ‘myapp.backbone_view’)

This would work on /api/status/dead on arrival

The browser will just replace the spaces with %20 and it should work.