I have a few Raspberry Pis around the house doing various tasks and wanted to automate Docker image builds using AWS CodeBuild. I assumed I could build on the Graviton ARM instances and run on the Pi. It works with the raspbian/stretch base image, but the Raspbian images are not actively maintained. When I switched to alpine base image, I started getting this error on the Pi. standard_init_linux.go:207: exec user process caused “exec format error”

RSQL is a command-line client for Redshift. Unlike the psql command-line, RSQL has control flow commands (IF, ELSE, GOTO, etc.) that are useful for ETL jobs. I want to run RSQL in a Fargate container so I call it from Step Functions ETL workflow. Overall this was fairly straight forward, but I’ll document it anyway. Setup In my use case, I am converting hundreds of Teradata BTEQ scripts to RSQL using the Schema Conversion Tool (SCT).

AWS Amplify is a framework to accelerate web and mobile application development. I needed to build an API that would return a binary object. Specifically, it allows me to download a PDF file. I could not find an example so I am documenting it here. In the end it was crazy simple. You just need to add a single line "BinaryMediaTypes": ["application/pdf"] to the AWS::ApiGateway::RestApi resource in the CFN template created by Amplify.

I published a post this summer with a few simple demo applications I use when configuring AWS infrastructure. I needed something similar for a serverless application on AWS. In other words, a Lambda function sitting behind an API Gateway or Application Load Balancer (ALB). I tend to use this simple Node.js function. It will return whatever it received as input. This is useful when you are debugging the infrastructure. For example, configuring Cognito with API Gateway or OIDC with ALB.

Every Halloween my kids and I build some kind of decoration to scare everyone. The past few years we have been evolving a pneumatic wolf head that pops up and scares you. Then, it takes a picture of you looking silly. It was based on a RaspberryPi and AWS IoT. This year I wanted to move to Arduino, but I could not find instructions for connecting to AWS IoT from the Arduino.

AWS Directory Services allows you to join AWS resources to Microsoft Active Directory. This includes Amazon Relational Database Service (RDS), Amazon FSx, Amazon Workspaces, Amazon Appstream 2.0, Amazon Connect, Amazon QuickSight, Amazon WorkDocs, Amazon WorkMail, and of course Amazon Elastic Compute Cloud (EC2) Windows instances. In addition, AWS recently announced the ability to Seamlessly Domain Join Linux EC2 Instances. As I modernize .NET applications by moving to .NET 5 and Linux, I can continue to leverage Active Directory for credential management.

I need to run a load test against Redshift. However, rather than repeatedly running a few sample queries, I want to replay all queries from the audit log over a period of time. I had never tried to do this in JMeter so it took a little hacking to get it working. I’m running this against Redshift, but the logic should apply to any workload. For example, you could use your web server logs to replay HTTP requests.

I have been working on multi-tenant OpenSearch (a.k.a. Open Distro for ElasticSearch) project. This article (https://www.elastic.co/blog/found-multi-tenancy) (from 2015) outlines a few isolation models and the issues you are likely to hit with each. Below I look at two options: one index per tenant and multiple tenants per index with document level security – a feature of OpenSearch. Provisioning I created a simple single-node Elasticsearch cluster on a t3.small. This is really small but I want to put pressure on the system and see how it scales.

In this final post I’ll list a few additional optimizations for reducing the first invocation times. See part one and two for more details. Burst CPU According to this video Lambda functions get additional CPU during the initialization phase. That extra CPU can help with JIT compilation but .NET lazy compiles code as it encounters it. Therefore, the function handler is not compiled until it’s executed, which is after the CPU is constrained.

I’m abandoning the multi-cloud blog hosting model that I was using in favor of AWS Amplify to simplify TLS configuration. But I thought I should document the old approach a little further in case I ever go back to it. The build pipeline for my blog fails every once in a while. For example, there was an issue with the Azure CLI earlier this month. Each time that happens it takes me a few minutes to remember how the pipeline works.

In part one, I looked at what happens the first time an ASP.NET application is invoked in Lambda. When we left off, we had a roughly 3 second initial response time. In this post I’ll focus on the initialization phase and part three will focus on invocation. ReadyToRun As I mentioned earlier, one cause of long cold start times in .NET is Just In Time (JIT) compilation. As each .NET assembly is loaded, the runtime converts the Intermediate Language (IL) into machine code for the specific platform it is running on.

The ability to host an ASP.NET project in AWS Lambda is a great way to get started with serverless. However, cold starts can result in a slow first invocation of the ASP.NET function. In this post I’ll set up a typical, albeit simple, application to gather benchmarks. In part two and three, I will explore a few options to speed up initialization and invocation respectively. Background A Lambda function is fundamentally different from a traditional application.

In the post, I will show you how to build a Linux container in Visual Studio running on a EC2 Windows Instance. The AWS Toolkit for Visual Studio allows you to deploy your project to Elastic Container Service (ECS) Fargate and recently as a container image to AWS Lambda among other options. In both of these cases, you must build a Linux container from Visual Studio or the dotnet command line.

I recently upgraded my home network from the Ubiquiti EdgeRouter to the UniFi Security Gateway (USG). Similar to the EdgeRouter, the USG supports most common configuration tasks from the web UI, but advanced configuration is only available from the command line. While you can configure a VPN tunnel to AWS from the UI, it does not allow you to configure redundancy or Border Gateway Protocol (BGP). With everyone quarantined – working and learning from home – I have been struggling to find time to hack the router.

Cloud Storage and Trailing Slashes Shortly after configuring this site to be served simultaneously from AWS, Azure and GCP, I realize I had a bug. Occasionaly the images were not loading. Ironically this was only happening on the Multi-Cloud Blogging post. After some investigation, I found this caused by how various providers handle a URI without a trailing slash. Specifically Azure. The Issue When I render the footer of this blog, I include the name of the cloud provider that served the page.

I spent some time over Thanksgiving moving my blog from Blogger to Hugo. I have been hosting my site in an Amazon S3 bucket with an automated build in AWS CodeBuild. That has been running well for the past month and I have worked out most of the kinks. So, I decided to make my blog Multi-Cloud and host it on AWS, Azure, and GCP while load balancing traffic across the three platforms.

I have been moving my blog to Hugo over the holiday weekend. I am working in a Cloud9 instance. Cloud9 allows you to preview an application running in the Cloud9 instance by proxying the connection through the Cloud9 service. The URL for the proxy uses the following format. https://CLOUD9_ENV_ID.vfs.cloud9.AWS_REGION.amazonaws.com/ The problem is that Hugo renders fully qualified URLs that include the baseURL found in the config file. I could update the config file, but I know I am going to accidentally check it in that way.

I have been working on a project to deploy Elastic Kubernetes Service (EKS) at an Academic Medical Center. They want to deploy a private cluster that does not have internet acess. EKS supports this, but DNS resolution can be tricky. There is an AWS blog post that explains how do it. Ultimately, we need an inbound R53 resolver ENI in the EKS VPC. When you configure EKS with a private endpoint it configures DNS to only respond to requests from within the VPC.

Chalice is a Python serverless microframework for AWS that enables you to quickly create and deploy applications that use Amazon API Gateway and AWS Lambda. In this blog post, I discuss how to create unit tests for Chalice. I’ll use Chalice local mode to execute these tests without provisioning API Gateway and Lambda resources. Creating a new project Let’s begin by creating a new Chalice project using the chalice command line.

Early in my career, I learned an important lesson: backup is easy, but restore is hard. Too often we take our backup and recovery for granted. We assume that if the backup completed successful, the restore will work when we need it. Anyone who has been through a disaster recovery exercise, whether simulated or real, knows this is seldom the case. In this post I discuss creating consistent backups of Windows Servers using the Volume Shadow Copy Service (VSS) and Elastic Block Store (EBS) snapshots.

I added a new Cloud Watch Logs Trace Listener to the .Net API for AWS. The API team plans to add support for Log4Net, but in the meantime I have been using this. https://github.com/brianjbeach/aws-dotnet-trace-listener

I have been wanting to configure a VPN Connection from AWS to my house, but my cheap Netgear router does not support IPSec. So, I picked up an old Cisco 871 router that does. I didn’t want to sacrifice the speed (it supports 802.11ac while the 871 is an old 802.11g device) and features of my Netgear router, so I put the 871 behind the Netgear and modified the VPN configuration for NAT traversal.

Cloud-Init is a set of Python scripts used to configure Linux instances when they boot in AWS. Cloud-Init is included on Ubuntu and Amazon Linux AMIs. You can think of a Cloud Init script as a bare-bones Configuration Management solution like Chef or Puppet. A Cloud-Init script is passed as user data. If you have ever passed a shell script as user data, it was Cloud-Init that queried the meta-data service and executed the script.

In my last post I used the awslogs daemon to push tcpdump events to AWS CloudWatch logs.  At the time it felt silly to use a file on disk and a daemon to push events from an interactive session.  Well I had some time to dig and I found a much cleaner way to do it without the daemon. It turns out that CloudWatch logs is implemented as a plugin to the AWS CLI.

I was recently debugging an issue with a fleet of Apache web servers.  I needed to watch for some low level network events we felt might be causing an issue (TCP resets, etc.).  I thought CloudWatch Logs would be a cool, albeit unnecessary, solution. NOTE: I found a much cleaner way to do this presented here. The awslogs package/daemon can be configured to upload any log file.  Just add a new configuration block to /etc/awslogs/awslogs.

In the last two posts (part 1, part 2) in this series we used PowerShell to gleam information from our monthly AWS billing report.  While you can use those scripts to learn a great amount of information from about your AWS usage, you will eventually outgrow PowerShell.  In this post I will show you how to load the bill into SQL Server for more detailed analysis.   In the prior posts we used the monthly reports.

It took 6 months but I finally got time to continue the series on Decoding Your AWS bill.  In the last post, we used PowerShell to download and query the monthly bill.  In this post we use tags to create a cost allocation report.  In the next, and final post in this series, I will show you how to load the hourly detail report into SQL Server. Let's assume that we have multiple project teams at our company and they all have servers running in the same AWS account.

I have been testing a a preview of a new PowerShell command, Import-EC2Instance, that will be added to the AWS PowerShell API next week. The new command allows you to import a VM from VMware or Hyper-V. I covered this in my book, but at the time the functionality was not available in PowerShell and I had to use the Java API. While the new command will upload and convert your VM, you can also do the upload and convert independently.

I have been building a bunch of Windows AMIs for EC2 recently. If the instance fails to build it can be a real bear to diagnose issues. You don't have access to the console to watch what's happening. It would be great if I could log to the EC2 Console (also called the System Log on the web site) so I knew what was happening. So I hacked the EC2Config Service to see how it was writing to the console.

When you create an Windows AMI (Amazon Machine Image) it is configured to generate a random server name. Often this name does not meet your needs. Maybe your company has a specific naming convention (e.g US-NYC-1234) or you just want to use a descriptive name (e.g. WEB01). Whatever the reason, let's look at how to set the name when you launch the machine. In this post we will use PowerShell to read the name from a Tag on the instance.

As you begin to adopt AWS you will likely be asked to report on both usage and cost. One way to do this is using the Monthly Billing report. In this post I will show you how to download your bill and analyze it using PowerShell. AWS offers a feature called Programmatic Billing Access. When programmatic billing access is enabled, AWS periodically saves a copy of your bill to an S3 bucket.

CloudTrail is new service that logs all AWS API calls to an S3 bucket. While the obvious use case is creating an audit trail for security compliance, there are many other purposes. For example, we might use the CloudTrail logs to keep a Change Management Database (CMDB) up date by looking for all API calls that create, modify or delete an instance. In this exercise I’ll use CloudTrail, Simple Storage Service (S3), Simple Notifications Services (SNS), Simple Queue Service (SQS) and PowerShell to parse CloudTrail logs looking for new events.