RSQL is a command-line client for Redshift. Unlike the psql command-line, RSQL has control flow commands (IF, ELSE, GOTO, etc.) that are useful for ETL jobs. I want to run RSQL in a Fargate container so I call it from Step Functions ETL workflow. Overall this was fairly straight forward, but I’ll document it anyway.

Setup

In my use case, I am converting hundreds of Teradata BTEQ scripts to RSQL using the Schema Conversion Tool (SCT). These scripts have interdependencies and must be in the same the folder. Therefore, I copied them all to an EFS volume. If the scripts were stand alone, I would prefer to put them on S3.

I need to run a load test against Redshift. However, rather than repeatedly running a few sample queries, I want to replay all queries from the audit log over a period of time. I had never tried to do this in JMeter so it took a little hacking to get it working. I’m running this against Redshift, but the logic should apply to any workload. For example, you could use your web server logs to replay HTTP requests.