Thursday, May 11, 2017

Using git push to deploy code to DigitalOcean

There'll be times using a continue integration server or stack is a big pain in the butt. Sometimes we just want to type git push from the terminal and the live app is updated. This is that tutorial although specific to DigitalOcean. I'll bet it will roughly work the same for other cloud services.

Assumptions
  • You have a Digital Ocean server (or droplet) running and configured
  • You have the ssh to said droplet working
So ssh to the droplet and then navigate to /var/www (or anywhere else you want to put your code) and make two directories:

$ mkdir liveapp.com && mkdir appcode.git
$ cd appcode.git

Inside the appcode.git folder we will create a "bare" git repository: git init --bare
A bare git repository doesn't contain our code but rather the internals of the .git folder. The whole point of this is to access the hooks folder within.

The hooks folder have scripts that can be run at certain steps within the git repo's lifecycle. We want a hook script to run after we "receive" a push to this repo. Hence we create a post-receive file inside the hooks folder.

$ vi post-receive

# Type this in

#!/bin/sh
git --work-tree=/var/www/liveapp.com --git-dir=/var/www/appcode.git checkout -f

# save and exit
# make the post-receive hook executable

$ chmod +x post-receive

There! We are done for the server side. We move to the our local machines.

All we actually need is need is to add a remote target in your working git repo. Assuming you added the droplet in your local ssh config then the command should be look like:

$ git remote add droplet ssh://user@ip-of-droplet/var/www/appcode.git

# add a new commit - this will not work even if you existing commits
# you must have a new commit
$ git add .
$ git commit -m "new commit"

# push to repo - could be github or bitbucket
$ git push

# push to droplet
$ git push droplet master

If all goes to plan, then our code will show in our new "git work tree" which is the liveapp.com folder inside our droplet.

A couple more thoughts:
  1. It's fairly easy to setup a sort of staging or testing folder/area with this.
  2. The post-receive script can do more things like a "restart app" or send mail to admin for notification. It's just a bash script.

Friday, May 5, 2017

Ghetto Python development with VS Code

Not everyone can afford Pycharm or Sublime or you just don't like the new Pycharm terms/license then this could be a alternative setup with Visual Studio Code.
  1. Install Visual Studio Code. It's has all the "f"s: fast, free, flexible, fowerful.
    Don't forget to install Python also.

    Tip: Install Python 3 unless you're maintaining legacy Python code then install Python 2
  2. Pick our plugins for Python. You can install plugins or extensions from the within VS Code; Just click on the "Extensions" button on the side menu or you can download them from the "marketplace" and install them manually. Go here for details.

    Here's our list of extensions.
    1. donjayamanne.python - our primary plugin so we can work with Python. It's got almost all the things we'd expect to make for a good Python IDE; Intellisense, Code formatting, refactoring, debugging, and linting.

      A word on linting though, The plugin doesn't come with it. You'll have to install your linting module like say Pylint or Flake 8 separately via pip. 

    2. Indent-raindow - Trust me, you'll need this. It makes seeing the code indents easier. 

    3. Jinja - This is an optional plugin. You'll probably only need this if you're working with a lot of Jinja templates.

      Don't forget to explore the marketplace for other customisations like themes.
  3. Start coding. Don't forget to make a virtual environment because VSCode does support them on a per project bases via a settings JSON file. Same with debugging.
  4. Push to a repo. VSCode already has a git client built-in so push your code to popular repos like Github, Gitlab or Bitbucket.
Code away!

Saturday, March 25, 2017

Visual Studio 2017 Xamarin stuck if you didn't install the SDK and NDK

I ran into this bug when I was upgrading from Visual Studio 2015 to 2017. I included the Xamarin option But I didn't include the Android SDK and NDK. I removed them from the install list. I did this because I already have both installed.

The "hang" occurs then you try opening the Xamarin in the Options window. Visual Studio will stop working here so you can't set the paths for Xamarin.

So I went digging through the Xamarin logs in the AppData folder. I know where the log folder is from a previous issue with Xamarin. Xamarin is a good idea but it's buggy as fuck. So navigate to %USERPROFILE%\AppData\Local\Xamarin\Logs\15.0\ folder and see which is the latest log and open it with your favorite text editor.

Once we confirm, if it's really the missing SDK and NDK paths, we then have to add them to Window's Registry.
  1. Open regedit
  2. Navigate to Computer\HKEY_CURRENT_USER\SOFTWARE\Xamarin\VisualStudio\15.0\Android
  3. Right-click and add new String Values
    • AndroidNdkDirectory - set it to the full path to where you install the NDK 
    • AndroidSdkDirectory - set it to the full path to where you install the SDK
  4. Close and restart the computer
And we should have fixed the issue.




Monday, March 6, 2017

Dealing with SSL HandShake failure with Python Requests (_ssl.c:590)

I was working with Requests trying to talk to a REST api when I encountered this SSL HandShake error. It was fairly easy to replicate because it happens every time I tried talking to a HTTPS endpoint or resource. It didn't matter if it was a GET or POST action. I suspect it'd be the same for other actions like PATCH or DELETE.

So what's the fix? Uninstall-install then maybe an update or upgrade.
  1. I updated Python to 2.7.11. You might not need to do this if your up-to-date. 
  2. I uninstalled requests. - $ pip uninstall requests
  3. I uninstalled urllib3. - $ pip uninstall urllib3
  4. Installed crypto dependencies or libs. This is where it gets hairy.

    First I had to update my machine's openSSL library. This could range from easy to hair pulling hard. Thankfully, I had brew on my machine so it was relatively easy.

    Anyhow you'll want to see this from the terminal after the update.

    $ openssl version
    OpenSSL 0.9.8zh 14 Jan 2016
    

    After you'll need to run this command:

    $ pip install pyopenssl ndg-httpsclient pyasn1
    

     This installs the rest of the crypto dependencies.
  5. I installed urllib3 back. - $ pip install urllib3
    I also installed urllib3[secure]. - $ pip install urllib3[secure]
    You might not need the secure version but I covering all the bases.
  6.  Install requests back. - $ pip install requests
    Same deal with urllib3, you might also want to install request with security.

    $ pip install request[security]
Why this works? I suspect that there's an installation-time check for either urllib3 (or requests) for the crypto stuff that keeps things from working without the uninstall-reinstall.





Tuesday, January 24, 2017

Doing query param request in Django

Typically, I want my API endpoint url to be flat like this:

mydomain.com/shit_api/v1/person/123

That url is pretty straight forward, it's trying to get a person with an ID of 123 from shit_api. But that's not always clear cut.

I've been working on a particular API that needed a pair of optional filters that we're a pain to work out in a regular expression. And this is where I found about Django's QueryDict object. With it I can write out API urls with params:

mydomain.com/shit_api/v1/something/?filter1=value1&filter2=value2

And on the urls and views, I can handle it as such:

#urls.py
urlpatterns = [
   url(r'^parts/$', SomethingView.as_view()), name='Something'),
]
...

#views.py
class SomethingView(View):
   def get(self, request, *args, *kwargs):
      fiter1_value = request.GET.get('filter1', 'default')
      fiter2_value = request.GET.get('filter2', 'default')

      #do more here

      return Response('some response')

The thing here is that request.GET.get() method having a default value. This make it that having it on the URL it isn't a problem. So call like:

mydomain.com/shit_api/v1/something/

Will still work.

Bonus since I don't have to deal with a convoluted regular expression. Yehey!

Friday, January 13, 2017

Tricks with Django URLs and knowing you suck doing RE

RE or regular expressions and I don't really see eye to eye. But as a programmer, I have to work with them. So it's awkward like your ex works in the same office as you. But lucky for us, we just stick to a couple of tricks to make it a bit palatable.

Numbers in URL

You'll need to use a \d+ in the pattern.

ex: (r’^status/(\d+)$’, ‘myapp.backbone_view’)

This would work on /api/status/1

Dealing with spaces

I found two ways to do this but then found out that they are the same: [\w|\W]+ AND .+

Yes that's a dot.

ex: (r’^status/([\w|\W]+)$’, ‘myapp.backbone_view’) OR
(r’^status/(.+)$’, ‘myapp.backbone_view’)

This would work on /api/status/dead on arrival

The browser will just replace the spaces with %20 and it should work.