Augmenting In-House Efforts in ETL Architecture and Development

A state agency needed a data warehouse to meet their needs. They attempted to develop the solution in-house, but quickly exhausted their team’s skill set. A second effort by a state-contracted consulting firm also failed to develop a solution that met the client’s needs and was easy for the client to maintain. We architected and developed an Extract, Transform and Load (ETL) framework to load complex data from source systems into a data warehouse. The solution provided core auditing and logging that the State staff could use to monitor the health of the ETL processing. Regular training and knowledge transfer sessions with the agency’s staff meant the client came away from the project with thorough knowledge of the ETL patterns and development processes used.