Cleaning up your global NPM packages

Sooner or later when you're working with any web app, you'll have to deal with nodejs because frontend requirements will force you to do so. And sooner or later, you'll have a mess of npm packages in your global space. You'll want to clean it up but when you issue the command npm list you get a whole mess of packages with their dependencies.

However there's a useful trick you can do that will make npm only list the top level packages without their dependencies.

$ npm list -g --depth 0

Here's a sample result:

The trick is the --depth 0 parameter. It suppresses the listing of a package's dependencies.

We can now tame our unruly global npm packages.

Ha!

Django lazily evals URL patterns so I thought the "include" function was broken

Little known things sometimes kick your ass and you'll need to call a friend.

Until today, I didn't realise that Django lazily evaluates it's URLs. I discovered this when the code me and my team were working on had a cascading URL include. So we had something like:

# main urls.py
urlpattern = [ 
   url(r'^path/', include('app1.urls'),
]

# 2nd url - app1's url.py
urlpattern = [
   url(r'^app1/', include('app1.subapp.urls'),
]

# 3rd url - subapp for app1 urls.py
urlpattern = [
   url(r'^subapp/$, actual_view.as_view()),
]

Problem started when I assumed that the full URL path will show in the debug page on the browser. I was looking for /path/app1/subapp/. I only saw the /path listing.

This is where I thought that Django was not registering the rest. It never dawned on me that by just doing "localhost:8000/path/" will allow me to see the rest of the URL pathing. Django just shows the same or next urls not the urls 2 levels in.

Thank you Eric (@riclags) for pointing out that fact. 

I'm a dumbass sometimes.

Doing listviews with empty lists and pullrefresh in Nativescript-ng

This was a pain in the butt to do right and I had to do a "bit" of trial and error to get this right.

All I wanted do was show a "centered" bold, large text on the screen when a list is empty and the actual list view when there's data on the list with both having support for pull to refresh action.

Googling this well probably lead you to Telerik's doc on the RadListView for pullToRefresh which I call shenanigans on because I couldn't make it to work. So we move to next idea which is Brad Martin's pullToRefresh nativescript plugin.

I did get Martin's nativescript plugin to work on my first attempt sans a bug that was a bit odd. I'm not even sure if the bug can be replicated on another machine. Anyhow, my first attempt's code looked like this:

 <PullToRefresh (refresh)="refreshList($event)" row="1">  
     <GridLayout horizontalAlignment="center" verticalAlignment="center"   
           rows="*" columns="*" *ngIf="emptyList()">  
       <Label horizontalAlignment="center" verticalAlignment="center" row="1" column="1" class="h2"   
           text="No Orders Available."></Label>  
     </GridLayout>   
     <ListView [items]="ordersList"   
          (itemTap)="onOrderTap($event)" class="list-group">  
       <ng-template let-order="item" let-odd="odd" let-even="even">  
         <GridLayout [class.odd]="odd" [class.even]="even" columns="auto, *" rows="auto">  
           <Label col="0" class="iconbkg" text=""></Label>  
           <avatar [avatardata]="order.avatar_prop"></avatar>  
           <StackLayout col="1" class="formMessage">  
             <GridLayout columns="auto, 100, *" rows="auto">  
               <Label class="h3 text-left" col="0" [text]="'Order #' + order.number"></Label>  
               <Label class="h3 text-right" col="1" [text]="order.date_placed | date:'MMMM dd yyyy'"></Label>  
             </GridLayout>                            
             <Label class="body" [text]="'City: ' + order.shipping_address.line4 + ', ' + order.shipping_address.state"></Label>  
             <Label class="body" [text]="order.lines.length + ' Item(s)'"></Label>  
           </StackLayout>  
         </GridLayout>  
       </ng-template>  
     </ListView>  
   </PullToRefresh>  

If you refer to the inner Gridlayout with the *ngIf element. The idea was to show this gridLayout when the list is empty. Don't bother with the ListView component because it will not show anything if [items] is empty.

The problem was when I do a pull to refresh, it was causing a crash in the android emulator saying that it was referencing a null view of some sort.

I eventually got it to do want I wanted. I did two things:

1. Move the empty list gridLayout into its own pullToRefresh container. Like so:

<PullToRefresh (refresh)="refreshList($event)" row="1" col="1" *ngIf="emptyList()">  
     <GridLayout horizontalAlignment="center" verticalAlignment="center" rows="*" columns="*">  
       <Label horizontalAlignment="center" verticalAlignment="center" row="1" column="1" class="h2"   
           text="No Orders Available."></Label>  
     </GridLayout>      
   </PullToRefresh>  

   <PullToRefresh (refresh)="refreshList($event)" row="1">  
     <ListView [items]="ordersList"   
          (itemTap)="onOrderTap($event)" class="list-group">
.....

2. I removed the ChangeDetectionStrategy in the @component declaration of the view class.

 With these two changes, the error no longer happened and it works perfectly.

Fixing CERTIFICATE_VERIFY_FAILED error on Macs with Python3

There's this annoying thing where Requests, a very popular Python library for working the internet doesn't work because of SSL certificates not being trusted.

This whole thing apparently is caused by OSX and Python3 having no certificates and can't validate SSL connections! This information is sort of hidden away in this itty bitty readme file on the certifi module which then pointed me to the readme file in python3. What. The. Fuck.

There, you'll know that for Python 3.6 on OSX, it requires a post-install step, which installs the certifi package of certificates. This is documented in the ReadMe, which you should find at /Applications/Python\ 3.6/ReadMe.rtf

The ReadMe will have you run this post-install script, which just installs certifi: 

/Applications/Python\ 3.6/Install\ Certificates.command

I should read the release notes for Python3 more closely.  

Tricks with Django URLs and knowing you suck doing RE

RE or regular expressions and I don't really see eye to eye. But as a programmer, I have to work with them. So it's awkward like your ex works in the same office as you. But lucky for us, we just stick to a couple of tricks to make it a bit palatable.

Numbers in URL

You'll need to use a \d+ in the pattern.

ex: (r’^status/(\d+)$’, ‘myapp.backbone_view’)

This would work on /api/status/1

Dealing with spaces

I found two ways to do this but then found out that they are the same: [\w|\W]+ AND .+

Yes that's a dot.

ex: (r’^status/([\w|\W]+)$’, ‘myapp.backbone_view’) OR
(r’^status/(.+)$’, ‘myapp.backbone_view’)

This would work on /api/status/dead on arrival

The browser will just replace the spaces with %20 and it should work.

Facebook not playing nice with Twitter in an Ionic app

This bug I encountered when my Ionic was logged in via Facebook. It caused my Twitter routine not to retrieve user timelines. My Twitter routine depends on a Application only login to get an Oauth token.

The symptom of my problem was when doing a call to api.twitter.com/oauth2/token and api.twitter.com/1.1/statuses/user_timeline.json, Angular's $http wasn't sending the correct header values and I kept getting invalid request (code 99) error from twitter.

POST https://api.twitter.com/oauth2/token HTTP/1.1
Authorization: Basic [base64 code here]
Content-Type: application/x-www-form-urlencoded;charset=UTF-8

The thing to focus here is the Authorization value. Instead of a base64 encoded value for Twitter, Facebook overwrites it or I can't overwrite it via a $http config object.

With the bug understood, I found the solution was to use a $httpProvider interceptor to modify the header before sending it to the server.

function AuthHeaderInterceptor($rootScope, $q, $location, localStorageService) {
    // Function to handle a request
    var request = function (config) {

        var expression = /(https?:\/\/(.+?\.)?api.twitter.com\/1.1\/statuses\/user_timeline.json(\/[A-Za-z0-9\-\._~:\/\?#\[\]@!$&'\(\)\*\+,;\=]*)?)/g;
        var regex = new RegExp(expression);
        var test_url = config.url;

        if(test_url.match(regex)){           
            var authToken = localStorageService.get('twitter_token'); // get token from local storage
            config.headers.Authorization = 'Bearer ' + authToken;     // Force overwrite the Authorization header
        }

        // Return our config
        return config;
    };

    // Function to handle errors
    var responseError = function (rejection) {
        // Return our rejected promise
        return $q.reject(rejection);
    };

    return {
        request: request,
        responseError: responseError
    };
}

All that's left is to set this to my app config and I was good to go.

Python and the MySQL driver hole I found myself

So I'm a Python developer now and Django to boot. To those who read this blog would have noticed that I primary do Java and AngularJS but I've used Python sparingly so this change ain't so bad.

But first off, this Python 2 and 3 is freakin' annoying but it still works out nicely with virtual environments. Then I walked into Python's mess of MySQL drivers. Working with Django with a MySQL backend, there's little or no mention of needing a MySQL driver, if a newbie walked into this error it take them a chuck of time to figure it out. And if they figure it out, will then stumble into the myriad of options (mysql-connector-python, PyMySQL, etc.) which lead me to the hole I mention on the title.

The whole start was just because I picked PyMySQL as my driver which a pure Python implementation of a MySQL connector. My Python script kept failing on me until I found that I need to install the damn thing:

try:
    import pymysql

    pymysql.install_as_MySQLdb()
except ImportError:
    pass

Before I run my __main__ function.

Phew.

Well, I can't back out now. I'm a Python dev now anyway.

Ionic, Satellizer, Facebook and that "Given URL is not allowed" error

You can have your Ionic mobile application use Facebook authentication. You can do it the hard way - i.e. do it yourself via $http calls - or go the easy route via Satellizer. Being the lazy bastard that I am, I'll be using Satellizer.

Satellizer can be setup quickly, do bower install, add the needed JavaScript bits to your index.html and reference it in you Ionic app.

angular.module('meAwesomeIonicApp', ['ionic', 'ngCordova', 'satellizer', 'ngAnimate']).config(...)

From here you'll need to go to Facebook Developer and register your app. You'll then add the FB application appId to your satellizer settings. It should look something like:

    var commonConfig = {
        popupOptions: {
            location: 'no',
            toolbar: 'yes',
            width: window.screen.width,
            height: window.screen.height
        }
    };

    if (ionic.Platform.isIOS() || ionic.Platform.isAndroid()) {
        commonConfig.redirectUri = 'http://localhost/';
        $authProvider.platform = 'mobile'
    }

    $authProvider.facebook(angular.extend({}, commonConfig, {
        clientId: 'YOUR FB APP ID HERE',
        url: 'http://localhost:3000/auth/facebook',
        responseType: 'token'
    }));

This moves us to the controllers. In the controllers we have access to a $auth service which is provided by Satellizer. The $auth service then provides a authenticate(string) function. So we have:

 
$auth.authenticate(provider)
        .then(function() {n
             // Success login
        })
        .catch(function(response) {
             // Error in login
        });
};

You can easily add this to a ng-click handler. And this is where we encounter the "Given URL is not allowed" error. What's happening is that when we call $auth.authenticate(), it will try to open a FB login page based on the url value we configured in the $authProvider.facebook() call instead we get the error page instead of the login form.

Fortunately, for me the fix was easy. I just didn't configure the settings in the FB developer app page correctly. It isn't enough to just configure the Basic Section in the Settings page. You need to open the Advance Section and also configure the Valid OAuth redirect URIs values also. So if you add the http://localhost value in the textfield, it should fix the "Given URL is not allowed" error.

Angular Charts and the "undefined" draw error on single series line charts

I've been recently working with Ionic and my app needed a chart. Also, been listening to way too much Tricot and Toe.

This chart requirement lead me to angular-chart which is based on the excellent chart.js library. My use case was simple. I needed a line chart to visualize various "single" series data. And this is where I encountered this "undefined" draw (and update) functions.

For context, this how my data looked.

{
   id: 0,
   labels: ['13:00', '13:15', '13:30', '13:45', '14:00', '14:15', '14:30', '14:45'],
   series: ['Stat #1'],
   data: [28, 30, 29, 32, 70, 79, 89, 98 ]
}

The important part here is the data array. This was the one causing the problem after I traced it to a closed issue in github. To fix this was to just turn the flat data array into a 2-dimensional array and poof the error is gone.

{
   id: 0,
   labels: ['13:00', '13:15', '13:30', '13:45', '14:00', '14:15', '14:30', '14:45'],
   series: ['Stat #1'],
   data: [[28, 30, 29, 32, 70, 79, 89, 98 ]]
}

I can now see my chart.

Tapestry5, JPA 2 and Hibernate

The documentation for Integrating JPA found on the Tapestry5 website was lacking.

1. It doesn't tell you that you have add the Tapestry-jpa dependency in you project POM
2. It uses EclipseLink
3. Explicitly tell you to use JPA 2.0 NOT JPA 2.1;

So, I had a few problems to fix after reading and following the docs. With much tinkering with my Tapestry5 (5.4-beta-35) project, I figured out:

1. To add the Tapestry-jpa dependency
   <dependency> <groupId>org.apache.tapestry</groupId> <artifactId>tapestry-jpa</artifactId> <version>5.4-beta-35</version> </dependency>
    
   
2. Hibernate 4.2 instead of EclipseLink
   <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-entitymanager</artifactId> <version>4.2.20.Final</version> </dependency>
   
3. Edit the persistence.xml header so it reads version=2.0 instead of 2.1. This is because my Netbeans IDE can generate the persistence.xml file but defaults to 2.1 version instead of 2.0.
   

And if you get a javax.persistence.PersistenceException: Unable to build EntityManagerFactory error during the build, you probably didn't add a hibernate.dialect property in your persistence.xml configs.

Tapestry5, JPA, Netbeans and a FilerException

Tapestry5.4 supports JPA out of the box which is nice. Also Netbeans supports JPA, doubly nice. Now if you're using 2 or more Persistence Units, you have probably encountered a FilerException when you try to compile or run your project. This shouldn't be a problem if you're using only single Persistence Unit. I suspect the fix would be the same if you encounter the same exception.

java.lang.RuntimeException: javax.annotation.processing.FilerException: Attempt to recreate a file for type {myclass}

The {myclass} here is always an entity class.

I think, a secondary symptom of this is if all your entity classes are all in a single package. This should be the case if you're doing a standard Tapestry5 web application.

The fix to this is to edit the Persistence Unit and add a  property and explicitly declare the entity classes on the Persistence Unit.

<persistence-unit name="PayrollPU" transaction-type="RESOURCE_LOCAL">  
   <provider>org.hibernate.ejb.HibernatePersistence</provider>  
   <class>cu.ictso.miscapps.entities.Departments</class>  
   <class>cu.ictso.miscapps.entities.Employee</class>  
   <class>cu.ictso.miscapps.entities.Groups</class>  
   <class>cu.ictso.miscapps.entities.ViewAttendance</class>  
   <exclude-unlisted-classes>true</exclude-unlisted-classes>  
   <properties>  
    <property name="javax.persistence.jdbc.url" value="jdbc:sqlserver://localhost;databaseName=DTR-Payroll2;Integrated Security=false"/>  
    <property name="javax.persistence.jdbc.user" value="PayMaster"/>  
    <property name="javax.persistence.jdbc.driver" value="com.microsoft.sqlserver.jdbc.SQLServerDriver"/>  
    <property name="javax.persistence.jdbc.password" value="123456"/>  
    <property name="javax.persistence.schema-generation.database.action" value="create"/>  
   </properties>  
 </persistence-unit>  

Here's an example Persistence Unit.

When we need something more than Firebug

Us web developers in our quest to build something unique will eventually come upon a wall that looks insurmountable with our current set of debugging tools like Firebug and Chrome's or IE's developer tools. Sometimes you just need something that can work at the HTTP protocol. This is where Fiddler2 comes in. Fiddler is a web debugging proxy for any browser, system or platform. Which makes it quite handy if you're doing AJAX stuff on a lot of different platforms from Java to Node.js.

What's really special with Fiddler is the composer feature which allows to make AJAX verb (GET, DELETE, PUT, HEAD, etc.) calls to your server side API. It then allows you to look at the response headers, cookies, the raw results and a bunch of other values to help you debugged that code.

Use this with Mozilla Firefox to get the most mileage out of it since it does integrate into it quite nicely. Chrome doesn't do it because of that new rule that you can't install a plugin that's not found in the Chrome store.

It also has a nice set of add-ons which is just icing on cake. I like StresStimulus add-on which allows you to do load testing with your web apps.

Another thing is this thing is free and will run on Linux boxes (needs Mono though).

Add this to your toolbox.

Debugging Javascript: Blackbird, JSLint & dumpProps

JavaScript can be difficult to debug despite the tooling we got - IDE, Firebug, etc.

A naive way of doing debugging in JavaScript is to use alert() function. Unfortunately, alert falls short in this respect. Enter Blackbird. Blackbird is sort of a JavaScript console that is better than using alert(). As their website says, "You might never use alert() again."


Blackbird is attractive and useful but not might fall short in some debug scenarios. All too often, I have had a need to dump the insides of a JavaScript object - its properties. Fortunately, plain JavaScript can do this. This script can be found at the www.breakingpar.com site and it works nicely with Blackbird (with a few modifications).

function dumpProps(obj, parent) {  
   // Go through all the properties of the passed-in object   
   for (var i in obj) {  
    // if a parent (2nd parameter) was passed in, then use that to   
    // build the message. Message includes i (the object's property name)   
    // then the object's property value on a new line   
    if (parent) { var msg = parent + "." + i + "\n" + obj[i]; } else { var msg = i + "\n" + obj[i]; }  
    // Display the message. If the user clicks "OK", then continue. If they   
    // click "CANCEL" then quit this level of recursion   
    if (!confirm(msg)) { return; }  
    // If this property (i) is an object, then recursively process the object   
    if (typeof obj[i] == "object") {   
      if (parent) { dumpProps(obj[i], parent + "." + i); } else { dumpProps(obj[i], i); }  
    }  
   }  
 }  

When you get your script working as expected you might want to run it through JSLint. JSLint is a code analysis tool following the coding rules as discussed by Douglas Crockford's book, JavaScript: The good parts. Essentially, JSLint looks for problems in JavaScript program. It is a code quality tool.

I hope that these stuff makes debugging JavaScript a little more pleasant.