Category: Cloud

A couple of gotchas setting up AWS Lambdas with TypeScript

The latest project I am working on requires us to set up a number of cloud services on AWS. While I have some experience doing that on Google Cloud Platform, the AWS system and tooling are a learning experience.

This week we are setting up a number of AWS Lambdas. We are trying to do this right from day one by enabling unit/integration tests and automated build systems from the start. We also made the choice to start immediately with TypeScript rather than migrate to it over time as we have done in other projects.

I tried using the AWS SAM CLI for creating the Lambdas today. There were a couple of hiccups that I wanted to share here.

1. Typescript templates are only available in ZIP-based deployments

I was glad to see that the SAM CLI allowed for the creation of TypeScript based projects out of the box. What took me a long time to realize was that the TypeScript templates only appear as options if you use the ZIP-based package type. I tried various combinations of node versions, etc before trying the zip based option which finally worked. There are also way too many tutorials on the web that start you out with a JS based project and then you add a number of config changes to migrate to TS which is a pain.

2. Colima-based hiccups

Most of the tutorials I found using SAM required you to use Docker Desktop, which unfortunately is no longer an option at work given their new licensing terms.

Thankfully this gist helped set up a local environment using Colima that seemed to work at first.

The only problem was that trying to test the code locally did not work as the local connections to the container would timeout without being able to connect. This turned out to be caused by Colima not enabling reachable IP addresses by default for performance reasons. Restarting Colima with colima start --network-address fixed this but I only discovered this after a couple of hours of banging my head against the computer.

Ah well, things are working right now and I need to get home 👋

Beyond Firebase Magic: Backups, Testing and Extensions

Firebase is pretty magic and for the most part delivers on the idea of an instant-on backend for your mobile or web-app. The problem with magic though is its hard to plan for if or when something does break. Take database backups for example. While traditional database backups are a known science, backing up Firebase’s storage (FireStore) setup is still a poorly documented / infrequently attempted effort.

I tried implementing that today and the official documentation only stressed me out. Thankfully, I found this article which got me most of the way there, though of course a few things had changed (api endpoints are now versioned v1 and the GoogleAuth library has a minor change on how it is initialized). Anyway, I am glad its finally done, but I wanted to share it here in case anyone else is looking for it.

While in Firebase, I also got a chance to play around with Firebase’s testing setup (my one cloud function in my project didn’t have a test so I implemented one). The local testing setup seems interesting but unfortunately I couldn’t get too far since FirebaseAuth isn’t supported yet, so for now my tests actually connect to a real Firebase account and run the test. Hopefully FirebaseAuth Triggers will be supported soon.

Finally I did try using Firebase Extensions for the first time and tied my Firebase Auth system to MailChimp. That worked flawlessly. I am really excited about the future of Firebase Extensions. Hopefully we’ll start seeing more functionality available as just a one-click addition to your project.

Trying out Google’s Cloud Endpoints vs Amazon API Gateway

I was recently looking to scaling up an API currently hosted on Heroku.com. While adding dynos to Heroku was an option, I also thought it was a good excuse to get more familiar with Google Cloud Platform (GCP), which I have been curious about for a while and have had some really good conversations on at previous events at GDG Philly.

GCP’s Cloud Endpoints seemed to be the right answer, especially since they do claim to support hybrid cloud deployments.

After setting up the Google Cloud SDK   and enabling billing on my active project, I clicked on the Cloud Endpoints left nav item to get this screen…

Turns out, the Cloud Endpoints service has no user interface by default. To get started, you have to deploy your api spec (as a Swagger2 / OpenAPI document) to see anything at all. Maybe that works for folks that live and breathe the command line but I was kinda put off, specially since I was aware of how AWS API Gateway worked (we’ll look at that later)

Anyway, so clicking on “Learn more” and then a further link to the documentation takes you to this page with the following table:

So based on their documentation, it seems that the only JSON apis that Cloud Endpoints can front are those hosted on Google App Engine or Compute Engine.

Argh! (Though gRPC does look kinda interesting, just not for now)

At this point I actually debating deploying the Rails app on the App Engine. Ok, let me try out how hard that is. After all they have a blog post on how to do it.

After an hour of configuring, I was able to deploy my Rails app to Google Cloud, but it took me too much effort to connect it to the CloudSQL backend and even more to try to get migrations running (never did, and I finally gave up). Google does have a video on how to get rails migrations working using their Cloud SQL proxy   but I was kinda done by then. This was taking me down a path I wasn’t really planning to go down.

Getting it work on Amazon’s API Gateway was so much easier.

The AWS console for API Gateway actually has a fairly intuitive user interface for fronting any third party api with API Gateway. At the end, you end up with a nice UI that describes the flow from the client through the gateway to your api base URL (Screenshot below)

I generally was surprised how easy AWS made their service and hopefully Google Cloud Endpoints reach the same level of simplicity. I am sure given how much Google is investing in their cloud services, this post will be out of date in a couple of months but for now AWS API Gateway does seem to have the edge