Django tests on an Azure Pipeline

Source: https://gist.github.com/killertilapia/c9e7635807e596f20a16123ceed9c48d

Running Django tests on Azure is a mix bag of confusing documentation and outdated examples. When We did this I can't shake the feeling that Python was a second class citizen in Azure. 

Stuff that bugged me OR caught me flat-footed

1. Environment Variables. I get that having many options supported for declaring variables is cool but I wish Azure followed one of Python's tenets and that's "There should be one— and preferably only one —obvious way to do it.!". 

Another thing that bugged me was how Azure allows declaring variables for the pipeline: why allow a `.` (period) in the declaration then explain that gets replacement with `_` (underscore) and the rest of the text capitalized under certain conditions. 

Why couldn't we just stayed with YML? I never did like idea of "scripts within scripts". Again, Python's tenets: "There should be one— and preferably only one —obvious way to do it.!". 

2. vmImage version. This isn't really an Azure issue but more of a Python test issue. In order to run the tests I wanted: 

python manage.py test --testrunner xmlrunner.extra.djangotestrunner.XMLTestRunner --no-input --settings=settings.testing

I had to create an environment for the test to run on. So step before this was to install my environment prerequisites which is directly influenced by the vmImage version. At first, I was using: ubuntu-16.08 which was causing a Django migration error due to unsupported feature. We quickly figured out that the sqlite version that came with 16.08 was older. We also had to bump up the python version to 3.8. 

If you're basic and prefer to use a UI then Azure is probably your thing. But an old school guy like me who prefers automated scripts, Azure's documentation and philosophy to cloud is an inconvenience. 

Fixing that CosmosDB Error=2: The index path corresponding to the specified order-by item is excluded.

This bug needs three things:

1. You're using Azure CosmosDB (I know, I don't like it too)
2. Have a Mongoose query with a sort option against a ..
3. Field that inside a sub document.

The query option in question is `{ sort: req.query.order || '-metaData.inserted_at' }`. The metaData.inserted_at field is just a date field. MetaData is just a plain object that has a couple of date fields tracking updates, deletes and such. So when you submit the query it spits out the Error=2 response.

CAUTION: Azure CosmosDB has a emulator isn't really helpful here. It will probably point you in a different direction. In my case, I was able to replicate the error and found a fix in where I 'unchecked' the Provision Throughput option in creating the database. That didn't solve the problem on the server.

In fixing this, you have two options:

1. No sorting in your code. 
2. Create the index

I went with option #2. A bit of a hassle. I tried CosmosDB as if it's a MongoDB equivalent.

db.getCollection('collectionName').getIndexes();

db.getCollection('collectionName').createIndex({'metaData.inserted_at':-1});

// shorter version
db.collectionName.createIndex({'metaData.inserted_at':-1});

It will be pain if you created an index that's wrong because you have to delete and create it again.

References:

• https://docs.mongodb.com/manual/tutorial/manage-indexes/#modify-an-index
• https://docs.microsoft.com/en-us/azure/cosmos-db/index-overview

Azure Cloud, an annoying mess of random errors & warnings

Context: We recently moved to Azure from AWS. We did this for business reasons. It's been about 2 months since we moved.

This would have not been a big deal if Azure wasn't a buggy P.O.S mess.

A few examples:

1. Random errors when completing a merge request. What the fuck is Status Code 0: error error?

Status code 0: error error? #WTF

2. You use Git flow? Well MS says 'fuck you'. You can't complete Pull Request from the command line. Doing a git flow feature finish from a CLI, Azure will mark your PR as abandoned unlike the other services. AND THEY EVEN HAVE THE BALLS TO CALL IT A FEATURE!! F!*@*#!@#(*!!!!

3. Random Status code: 200 errors when using the Azure CLI to do stuff. But when you do the same thing with the SLOW AS ASS web app console, no error. If you're a devops guys, do you feel safe?

4. RESOLVING MERGE CONFLICTS IS ASS BACKWARDS!! If a PR will be marked as conflict, you have to pull the PR branch and attempt local merge to see the conflict. You will not find the conflicts in the File viewer.  WHAT. THE. FUCK.

The take away is unless the cost savings is significant, really significant, move to Azure otherwise don't. Personally, I'd rather pour gasoline over my laptop and setting it on fire rather than use Azure for my projects EVEN IT WAS FREE.

Azure Cloud, an annoying mess of random errors & warning and things not working as expected.


Edit. Ironically, the day when I put up this post, I ran into #4.

My summer 2015 MEAN project: Dormmetita

Dormmetita. That's the name of the app we are building.

I asked a question over at the CDO-ITG FB page a couple of weeks ago what would the community like to read or do over the summer. I had a couple of options for Java, Node and Python. The overall winner though was nodejs. So for the summer I'm doing a MEAN tutorial to build an old app that a couple of guys "brainstormed" about a year back named Dormmetita. Right, +Romar Mayer Micabalo, +Paul Michael Labis, +Raven Duran?

Technically it's "Dorm me, tita." which is an app to find and post dorm rooms, apartments for students localized to my city, Cagayan de Oro which is a university town. We might throw in reviews but let's keep it simple for now. We can make up stuff as we go.

 I've broken down the tutorial in these topics:

1. Setting up 
2. Node is up: hello world
3. Coding the REST services (Part 1) - The Model
4. Finish the REST (Part 2) with Fiddler debugging and testing
5. Bootstrapping the view parts 
6. Add Angular to the page
7. Deploy to Azure, because free is nice

So bookmark this post to follow.