Showing posts with label Database. Show all posts
Showing posts with label Database. Show all posts

Wednesday, December 21, 2022

Dumping and Restoring a DB from host into a Docker container or back

Assuming you have the sql (ie. backup.sql) file on hand:

To create a backup of a mysql DB (ie. dumping it):

$ docker exec CONTAINER_NAME /usr/bin/mysqldump -u root --password=root DATABASE > backup.sql

To restore a mysql dump (for example):

$ cat backup.sql | docker exec -i CONTAINER_NAME /usr/bin/mysql -u root --password=root DATABASE 

Add the --force or -f flag to ignore errors

$ cat backup.sql | docker exec -i CONTAINER_NAME /usr/bin/mysql -u root --password=root  --force DATABASE 

I tested this using the mysql 8.0 image. 


ref: https://stackoverflow.com/questions/46579381/how-to-restore-mysql-dump-from-host-to-docker-container


Friday, June 10, 2016

Python and the MySQL driver hole I found myself

So I'm a Python developer now and Django to boot. To those who read this blog would have noticed that I primary do Java and AngularJS but I've used Python sparingly so this change ain't so bad.

But first off, this Python 2 and 3 is freakin' annoying but it still works out nicely with virtual environments. Then I walked into Python's mess of MySQL drivers. Working with Django with a MySQL backend, there's little or no mention of needing a MySQL driver, if a newbie walked into this error it take them a chuck of time to figure it out. And if they figure it out, will then stumble into the myriad of options (mysql-connector-python, PyMySQL, etc.) which lead me to the hole I mention on the title.

The whole start was just because I picked PyMySQL as my driver which a pure Python implementation of a MySQL connector. My Python script kept failing on me until I found that I need to install the damn thing:

try:
    import pymysql

    pymysql.install_as_MySQLdb()
except ImportError:
    pass

Before I run my __main__ function.

Phew.

Well, I can't back out now. I'm a Python dev now anyway.

Wednesday, January 13, 2016

Reset Auto Increment ID in SQL Server

Been asked this way too many times and I google it every time. So during development, when you need to reset the ID for your auto increment column:

DBCC CHECKIDENT (mytable, RESEED, 0)

This command reseeds "mytable" to start at 1. Be careful that we don't have records higher than the seed value you are setting. You'll break the table.

Also:

  • http://stackoverflow.com/questions/510121/reset-autoincrement-in-sql-server-after-delete

Monday, October 12, 2015

Tapestry5, JPA 2 and Hibernate

The documentation for Integrating JPA found on the Tapestry5 website was lacking.
  1. It doesn't tell you that you have add the Tapestry-jpa dependency in you project POM
  2. It uses EclipseLink
  3. Explicitly tell you to use JPA 2.0 NOT JPA 2.1;
So, I had a few problems to fix after reading and following the docs. With much tinkering with my Tapestry5 (5.4-beta-35) project, I figured out:
  1. To add the Tapestry-jpa dependency
    <dependency> <groupId>org.apache.tapestry</groupId> <artifactId>tapestry-jpa</artifactId> <version>5.4-beta-35</version> </dependency>
        

  2. Hibernate 4.2 instead of EclipseLink
    <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-entitymanager</artifactId> <version>4.2.20.Final</version> </dependency>
  3. Edit the persistence.xml header so it reads version=2.0 instead of 2.1. This is because my Netbeans IDE can generate the persistence.xml file but defaults to 2.1 version instead of 2.0.
And if you get a javax.persistence.PersistenceExceptionUnable to build EntityManagerFactory error during the build, you probably didn't add a hibernate.dialect property in your persistence.xml configs.

Thursday, October 8, 2015

Tapestry5, JPA, Netbeans and a FilerException

Tapestry5.4 supports JPA out of the box which is nice. Also Netbeans supports JPA, doubly nice. Now if you're using 2 or more Persistence Units, you have probably encountered a FilerException when you try to compile or run your project. This shouldn't be a problem if you're using only single Persistence Unit. I suspect the fix would be the same if you encounter the same exception.

java.lang.RuntimeException: javax.annotation.processing.FilerException: Attempt to recreate a file for type {myclass}

The {myclass} here is always an entity class.

I think, a secondary symptom of this is if all your entity classes are all in a single package. This should be the case if you're doing a standard Tapestry5 web application.

The fix to this is to edit the Persistence Unit and add a  property and explicitly declare the entity classes on the Persistence Unit.

<persistence-unit name="PayrollPU" transaction-type="RESOURCE_LOCAL">  
   <provider>org.hibernate.ejb.HibernatePersistence</provider>  
   <class>cu.ictso.miscapps.entities.Departments</class>  
   <class>cu.ictso.miscapps.entities.Employee</class>  
   <class>cu.ictso.miscapps.entities.Groups</class>  
   <class>cu.ictso.miscapps.entities.ViewAttendance</class>  
   <exclude-unlisted-classes>true</exclude-unlisted-classes>  
   <properties>  
    <property name="javax.persistence.jdbc.url" value="jdbc:sqlserver://localhost;databaseName=DTR-Payroll2;Integrated Security=false"/>  
    <property name="javax.persistence.jdbc.user" value="PayMaster"/>  
    <property name="javax.persistence.jdbc.driver" value="com.microsoft.sqlserver.jdbc.SQLServerDriver"/>  
    <property name="javax.persistence.jdbc.password" value="123456"/>  
    <property name="javax.persistence.schema-generation.database.action" value="create"/>  
   </properties>  
 </persistence-unit>  

Here's an example Persistence Unit.


Thursday, November 20, 2014

Why u no format SQL SQL Server Management Studio

Microsoft probably just loves developers but not DBAs because I find it amazing that the SQL Server Management Studio (SSMS) doesn't have a format SQL thing. It has the other stuff like Upper or lower case text, indent, undent, comment and uncomment BUT no freakin' format SQL. That is bullshit.

Of course, you could argue that I should buy some of those 1000 dollar plugins like Redgate SQL Prompt but that seems excessive. I just need to format SQL. That's it.

So after googling a bit on this I was able to find PoorSQL which a not only a website that formats SQL but is also a plugin in for SSMS and Visual Studio. Amazingly, it's also a open-source and free.


Tuesday, January 21, 2014

Since ElementaryOS, I needed an alternative to SQL Server Manager

I've had ElementaryOS running on my office PC since November last year and I've enjoyed it immensely. There's nothing like trolling Mac users thinking their desktop environment is unique.

But since I'm on, by all accounts, a linux box how do I access the database and let's face it, most days in an office we devs don't write a lick of code but instead doing queries, mining data for reports. Special kind of reports. Sounds familiar? And this is where I needed an alternative to SQL Server Manager since my company is using SQL SERVER. Enter DBeaver.

DBeaver is billed as a free universal database manager. Although, it doesn't have all the bells and whistles of SQL Server Manager it's got enough that I get my job done and it's free. Let me repeat that: Free.

To install this, we're gonna go the apt-get route:

$ wget -c http://dbeaver.jkiss.org/files/dbeaver_2.3.5_amd64.deb
$ sudo dpkg -i dbeaver_2.3.5_amd64.deb
$ sudo apt-get install -f

I'm install the 64-bit version.

For the purist, you could always download the tarball and install it from the source.

$ wget -c http://dbeaver.jkiss.org/files/dbeaver-2.3.5-linux.gtk.x86_64.zip
$ tar xvf dbeaver-2.3.5-linux.gtk.x86_64.zip
$ make install

After the install what's left is to download the drivers which you can do in dbeaver.

Friday, July 13, 2012

Install local jars with maven

I don't know why it took me a long time to write about this.

There are a lot of Java jars that isn't on the maven repository but you still need or like to manage these jars via Maven. To install a jar into your local Maven repo:

 
mvn install:install-file -Dfile={file} -DgroupId={groupname} -DartifactId={artifactname} -Dversion={versionnumber} -Dpackaging=jar
 
For example, you want to add Microsoft MSSQL jdbc driver into your local Maven repo then the procedure would be:
  1. Download and extract the file.
  2. Open the command line and type in:
  3.  
    mvn install:install-file -Dfile=sqljdbc4.jar -DgroupId=com.microsoft.sqlserver -DartifactId=mssqljdbc4 -Dversion=4.0 -Dpackaging=jar
     
    
You can then add it to your project with:
<dependency>  
<groupId>com.microsoft.sqlserver</groupId>  
<artifactId>mssqljdbc4</artifactId>  
<version>4.0</version>  
</dependency>

Sunday, March 4, 2012

Quirky stuff with JNDI and MSSQL

My last post talked about getting Glassfish and MSSQL connected. By doing this, we effectively hand over the management of the our database connection to the application container. One less thing to deal with in code.

Now, we try to get some data and see if it works. I decided to try out JDBC first. I could just jump to Hibernate but remember Hibernate just glosses over the details. If JDBC works then I know that Hibernate will work. So my code started as:
        ....
        try{
            InitialContext ctx = new InitialContext();
            DataSource ds = (DataSource)ctx.lookup("MSSQL2008");
            
            Connection conn = ds.getConnection();
            Statement stmt = conn.createStatement();
            
            ResultSet rs = stmt.executeQuery("SELECT * FROM [TestDB].[dbo].[Users]");
            String temp = rs.getString("email");
            
            stmt.close();
            conn.close();
           
            return temp;
        }catch(Exception e){
            e.printStackTrace();
        }
That didn't go off without errors. I got a "SQLServerException: The result set has no current row." error. Apparently, the Microsoft JDBC driver I was using didn't move the cursor to the first row of the result set. Feature or bug?
To fix this we forcibly move the cursor to the first row by adding the line: rs.absolute(1).
The code then will look like:

        ....
        try{
            InitialContext ctx = new InitialContext();
            DataSource ds = (DataSource)ctx.lookup("MSSQL2008");
            
            Connection conn = ds.getConnection();
            Statement stmt = conn.createStatement();
            
            ResultSet rs = stmt.executeQuery("SELECT * FROM [TestDB].[dbo].[Users]");
            rs.absolute(1);
            String temp = rs.getString("email");
            
            stmt.close();
            conn.close();
           
            return temp;
            
        }catch(Exception e){
            e.printStackTrace();
        }
Save, compile and redeploy and.....another error. Typical. This time the error I got was: SQLServerException: The requested operation is not supported on forward only result sets. Can you get what's the error? So we need to NOT create a forward only result set. This is done by adding the line: createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY). 
The final code is then:
        ....
        try{
            InitialContext ctx = new InitialContext();
            DataSource ds = (DataSource)ctx.lookup("MSSQL2008");
            
            Connection conn = ds.getConnection();
            Statement stmt = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
            
            ResultSet rs = stmt.executeQuery("SELECT * FROM [TestDB].[dbo].[Users]");
            rs.absolute(1);
            String temp = rs.getString("email");
            
            stmt.close();
            conn.close();
           
            return temp;
            
        }catch(Exception e){
            e.printStackTrace();
        }
All should be OK now. Now to do some real damage. *evil grin*

Thursday, December 22, 2011

6 Simple types of Inverted Queries

Whatever kind of database you are working on - NoSql, ORM, relational, or OO you will still need to query for data.

There are two ways in which data can be queried. The first way is for each entity to have a identifying or unique attribute. This identifying or unique attribute then can be used to select the entity or entities. The second way is an inversion of the first and is designed so that the entity identifiers associate with a given attribute may be obtained. The first way is useful for answering the question: What are the properties of a given entity? The second is useful for answering: What entities have a given property?

Assuming that we have entities E; attributes, A; and their attribute values, V. We have these 6 types:

Form Type of Query Example
Common attribute inquiry How much did Saleman No. 15 earn last month?
Which entities have the given value? Which salesman or men earned more that 20,000 last month?
List all attributes having a given set of values for a given entity Which month's earnings for Salesman No. 15 have exceeded 20,000?
Request for all information about a given entity List all the information stored about Saleman No. 15
List the value of a given attribute for every entity List last month's earnings for every salesman
List all entity attributes having a given set of values For every salesman, list every month when this earnings exceeded 20,000
You can mix and match the six types to create very complex queries. You could for example ask the question: Which salesman earned more that 20,000 in May OR June OR July?