Wednesday, December 21, 2022

Dumping and Restoring a DB from host into a Docker container or back

Assuming you have the sql (ie. backup.sql) file on hand:

To create a backup of a mysql DB (ie. dumping it):

$ docker exec CONTAINER_NAME /usr/bin/mysqldump -u root --password=root DATABASE > backup.sql

To restore a mysql dump (for example):

$ cat backup.sql | docker exec -i CONTAINER_NAME /usr/bin/mysql -u root --password=root DATABASE 

Add the --force or -f flag to ignore errors

$ cat backup.sql | docker exec -i CONTAINER_NAME /usr/bin/mysql -u root --password=root  --force DATABASE 

I tested this using the mysql 8.0 image. 


ref: https://stackoverflow.com/questions/46579381/how-to-restore-mysql-dump-from-host-to-docker-container


Thursday, November 17, 2022

Fixing bad video files with ffmpeg

FFmpeg is an open-source collection of tools for processing video and audio files. It allows to convert video and audio files from one format into another, resize videos, stream audio and video, and perform various other actions with media files.

FFmpeg is quite easy to interact with Python via bindings but it's also a CLI which what I used to fixed bad video. 

The simplest command we can do:

ffmpeg -err_detect ignore_err -i video.mkv -c copy video_fixed.mkv

A more involved command:

ffmpeg -vcodec mpeg4 -b:v 7561k -qscale:v 2 -acodec aac -ac 2 -async 1 -strict experimental ./video_fixed.mp4 -threads 0 -i damage_file.mp4

This one re-encodes while trying to fix the video. This one also take longer to run.


Thursday, October 6, 2022

Django utils and misleading timezones

Date and time are difficult things to get right in web programming. It has many factors affecting it like the server time zone, the users' time zone, a serverless function calling from a different time zone, you set `USE_TZ` to true in your django settings, etc. 

I had an interesting problem where my django site had to use `US/Arizona` as the default time zone. At first glance, using `timezone.now()` was enough to account for the time zone usage.

The interesting part come when you drill down into the timezone class; especially the now() method. The 2nd bullet says that now() will always return times in UTC REGARDLESS of the value of TIME_ZONE. So you get values that are 1 day ahead or behind depending on your time zone. Mine, was 1 day ahead:


I was expecting that the now() method would return a `2022-10-05` date (you can ignore the time component). Apparently, the now() method must be converted to the correct local date via the localdate() method. 


There's also a localtime() method if you need to get the correct local time for the time zone.


Wednesday, August 17, 2022

Flutter + Python = Flet

Flet is this pretty cool framework that allows someone to build an app on web, mobile and desktop quickly. It uses the Flutter framework for the UI stuff to get it to look good and work on any platform. 

It says that its language agnostic but it currently only supports Python. Does that make it a Python framework?

Anyway, I've been playing around with it and I like it. 

Things that I like:

1. Good looking apps created quickly

2. Everything is in Python even the UI; quite similar how Flutter does it, in fact. 

3. Next-to-nothing code change to target different platforms; so, to get a desktop app to run on a web browser is a just a named function parameter

flet.app(target=main) # run it as a desktop
flet.app(target=main, view=flet.WEB_BROWSER) # run it on a web browser/web app

Things that's missing:

1. No hot reload (but it's on the roadmap so that's something to look forward to)

2. Not all of the Flutter widgets are available like the Hero widget


Overall, I'm having a grand old time trying out Flet. 

Tuesday, June 21, 2022

A guide to better google-fu in 8 forms

Google is a powerful tool but 98% of the people on the Internet suck at it. Google search can do more than just typing in a word and pressing enter.

Here's the 8 tips:

  1. "Quotation Marks"

    Put quotes around search terms to let you search exactly for that word.

    Example: "Sara Duterte"

    Gives you all Sarah Duterte search results without just "Sarah" or just "Duterte".

  2. - Dashes

    Use dashes to exclude a term from your search results. The word before the dash will be included.

    Example: multithreading -java

    You want to know about multithreading but exclude anything with java. 

  3. OR

    Put "OR" between each term to combine search results. A vertical bar | also works.

    Example: marathon OR race, marathon | race

  4. site:

    Use "site:" to search within a specific website or domain.

    Example: Nuclear Energy site:en.wikipedia.com

    Would give you results about Nuclear Energy but limited to en.wikipedia.com.

  5. $

    If you are searching for a price.

    Example: Video card $600

  6. .. (two periods)

    Search within a range of numbers. 

    Example: SUV 2010..2012, motherboard $100..$140

  7. location:

    Use "location:" to get results related to a particular location

    Example: Sara Duterte location:cebu



  8. Use @ in front of a word to search social media. 

    Example: @pythonconf

    Similarly, you can also use "#" in front of the word to search hashtags in social media. 

    Example: #pyconph

Tuesday, June 14, 2022

Enforce row count limits on ManyToManyField rows in Django models

I had a requirement where ModelA needs to limit how many ModelBs it can have. They are linked via a ManyToManyField using a through model.

It looks like this:

 class ModelA(models.Model):  
   name = models.Charfield(max_length=255)  
   b_limit = models.PositiveSmallIntegerField(help_text="Max number of Bs we can have")  
   model_b_set = models.ManyToManyFields('ModelB', through="ABJoinTable")  

 class ModelB(models.Model):  
   some_field = models.Charfield(max_length=255)    
   model_a_set = models.ManyToManyFields('ModelA', through="ABJoinTable")  

 class ABJoinTable(models.Model):   
   model_a = models.ForeignKey("ModelA", on_delete=models.CASCADE)  
   model_b = models.ForeignKey("ModelB", on_delete=models.CASCADE)  
   is_suspended = models.BooleanField(default=False)  

The through model just contains extra information is NOT the key to enforcing the rule. Remember, we want to limit how many ModelB rows ModelA can have. How many ModelB's ModelA can have it controlled by the "b_limit" field. 

The solution is to use the m2m_changed signal

from django.db.models.signals import m2m_changed  
from django.core.exceptions import ValidationError

def enforce_modelA_B_limit(sender, **kwargs):  
   modelA = kwargs['instance']  
   if modelA.model_b_set.count() >= modelA.b_limit:  
    raise ValidationError("ModelA has too many ModelBs")  

m2m_changed.connect(enforce_modelA_B_limit, sender=ModelA.model_b_set.through)  

You can place this wherever a signal function is valid. I just placed mine in the same place where the models are declared.

Why a signal?

If your gut feeling was to override the `save()` or `clean()` methods then those won't work. You cannot check the count for the ManyToManyField until you have save the primary model - ModelA. You'll get an exception saying something like: `ModelA needs to have a value for field "id" before this many-to-many relationship can be used.`.

Testing Tip

In writing a test case for this, you can use the `self.assertRaises()` context to catch the error. It's bad practice if you do a `try-catch` block to see if the rule works. The test case would look like this:

 def test_modelA_B_limit_rule(self):  
   # we assume that a setUpTestData() method exist  
   # we also assume that ModelA is limited to 1 ModelB so adding a second ModelB should throw an
   # exception
   sample_b = ModelB.objects.create(some_field="test")  
   modelA_instance = ModelA.objects.get(pk=1)  
   with self.assertRaises(ValidationError):  
     modelA_instance.model_b_set.add(sample_b) # should throw  

There's nothing to change if you have Django Rest Framework project. Your API will throw a 400 error if they violate the limit.

Refs: https://stackoverflow.com/questions/20203806/limit-maximum-choices-of-manytomanyfield

Thursday, June 9, 2022

Fixing a Django drf-yasg "serializer_class required" AssertionError

Sometimes you have an API view (ie. GenericAPIView) class without a serializer. This causes the drf-yasg generator to throw an AssertionError exception saying class should either include a 'serializer_class' attribute, or override the 'get_serializer_class()' method.

You can fix this by adding the following to the view class:

 def get_serializer(self, *args, **kwargs):  
     pass  
 def get_serializer_class(self):  
     pass  

This should pass the drf-yasg serializer check.


Reference