ETL refers to "Extract, Transform, and Load". By this point, the concept of ETL is well-established within the data industry and there are a number of enterprise-proven "ETL Tools" available to assist organizations with the movement and transformation of data. An ETL Tool is simply software that's designed to help organizations move and transform data [...]
Databricks is now available in both AWS and Azure so it’s getting a lot of buzz! Let’s discuss 5 things you should know about Databricks before diving in. 1. Databricks is a managed Spark-based service for working with data in a cluster Databricks is an enhanced version of Spark and is touted by the Databricks [...]
While the data warehouse isn’t a new concept, many are left wondering – what is a data warehouse? And why do I need one? I'm sad to tell you that when we say "data warehouse", we aren't referring to a physical building that stores data! Datalere’s official definition of a data warehouse is the following [...]
With the Automating Database Deployment: Part 3 Running RoundhousE post we walked through running a RoundhousE deployment for the first time to create the database and several database objects. Script naming importance was talked about to help maintain the correct execution order. The tables were empty so a second deployment was run with new scripts added to [...]
Now that we have a basic understanding of RoundhousE database deployment from the first two parts of this series let’s get to the good stuff. If you haven't read the first articles you might take some time to go back through them. Here are links to those first two. Automating Database Deployment: Part 1 Introduction [...]
Remember from the introduction that RoundhousE is a command line driven database deployment engine. This article of the series will discuss the deployment flow and the RoundhousE environment, looking at the default behavior and how to modify it using command line switches. When a RoundhousE deployment is run the first step is a check to [...]
Database Deployment in DevOps One of the many obstacles to database development and DevOps is how to deploy databases consistently and how to automate the process. Database deployments have historically consisted of last minute treasure hunts to find or build scripts of the changes that have been made. Inevitably that leads to missed changes, no [...]
AWS EC2 Linux AMI with pyodbc, psqlODBC, Microsoft ODBC Drivers Context: Using pyodbc within a small python application to pull tracking data from an Azure SQL Server DB, do some work, and then store the results into AWS RDS PostreSQL DB. Problem: Default AWS EC2 Linux AMI 2017.09.1 does not come with unixODBC 2.3.x which [...]