Python

A convenience function to assume a IAM Role via STS before running a command. Add the following to your ~/.zshrc (or equivalent) file: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 function with-role { readonly role_arn=${1:?"The role_arn must be specified."} env -S $( aws sts assume-role \ --role-arn ${role_arn} \ --role-session-name ${USER} \ | \ jq -r '.Credentials | " AWS_ACCESS_KEY_ID=\(.AccessKeyId) AWS_SECRET_ACCESS_KEY=\(.SecretAccessKey) AWS_SESSION_TOKEN=\(.SessionToken) "' ) ${@:2} } This assumes that you have both the AWS CLI and jq installed....

When building systems to process messages, it’s not unlikely to find yourself in a situation where you need to process a number of inputted heterogeneous messages (i.e. messages of varying shapes/types). For example, consider a situation where you are processing messages from an SQS queue via a Lambda function. This post attempts to highlight how this can be achieved in a clean and elegant manner by utilizing Pydantic, Python’s typing system, and some helpers from the Python standard library....

This post is a quick capture of how to easily secure your FastAPI with any auth provider that provides JWKS. Background: RS256 RS256 is a signing algorithm used to generate and validate JSON Web Tokens (JWTs). Unlike the common HS256 algorithm that uses the same secret string to both generate and validate JWTs, RS256 uses a private key to generate JWTs and a separate public key for validating JWTs: RS256 generates an asymmetric signature, which means a private key must be used to sign the JWT and a different public key must be used to verify the signature....

At times, a developer may need to access infrastructure not available on the public internet. A common example of this is accessing a database located in a private subnet, as described in the VPC Scenario docs: Instances in the private subnet are back-end servers that don’t need to accept incoming traffic from the internet and therefore do not have public IP addresses; however, they can send requests to the internet using the NAT gateway....

Below is a very simple example of a script that I write and re-write more often than I would like to admit. It reads input data from a CSV and processes each row concurrently. A progress bar provides updates. Honestly, it’s pretty much just the concurrent.futures ThreadPoolExecutor example plus a progress bar.

Alternate title: How to be master of your domain. The basic idea of this post is to demonstrate how CloudFront can be utilized as a serverless reverse-proxy, allowing you to host all of your application’s content and services from a single domain. This minimizes a project’s TLD footprint while providing project organization and performance along the way. Why Within large organizations, bureaucracy can make it a challenge to obtain a subdomain for a project....

A quick note about how to generate a database URI (or any other derived string) from an AWS SecretsManager SecretTargetAttachment (such as what’s provided via a RDS DatabaseInstance’s secret property). 1 2 3 4 5 6 7 8 9 10 11 db = rds.DatabaseInstance( # ... ) db_val = lambda field: db.secret.secret_value_from_json(field).to_string() task_definition.add_container( environment=dict( # ... PGRST_DB_URI=f"postgres://{db_val('username')}:{db_val('password')}@{db_val('host')}:{db_val('port')}/", ), # ... )

I would argue that S3 is basically AWS’ best service. It’s super cheap, it’s basically infinitely scalable, and it never goes down (except for when it does). Part of its beauty is its simplicity. You give it a file and a key to identify that file, you can have faith that it will store it without issue. You give it a key, you can have faith that it will return the file represented by that key, assuming there is one....

S3 Batch Operation provide a simple way to process a number of files stored in an S3 bucket with a Lambda function. However, the Lambda function must return particular Response Codes. Below is an example of a Lambda function written in Python that works with AWS S3 Batch Operations.

S3 Inventory is a great way to access a large number of keys in an S3 Bucket. Its output is easily parsed by AWS Athena, enabling queries across the key names (e.g. find all keys ending with .png) However, sometimes you just need to list all of the keys mentioned in the S3 Inventory output (e.g. populating an SQS queue with every keyname mentioned in an inventory output). The following code is an example of doing such task in Python:

Here’s a quick example of creating an file-like object in Python that represents an object on S3 and plays nicely with PIL. This ended up being overkill for my needs but I figured somebody might get some use out of it.

I’ve been experimenting with Python’s Natural Language Toolkit, following along with Steven Bird, Ewan Klein, and Edward Loper’s book “Natural Language Processing with Python — Analyzing Text with the Natural Language Toolkit” (pdf version). So far, the book’s been great. As I’m going through the book, I’ve been writing down notes relating to the book’s examples. I’ve made a Github repo to store these notes and experiments that I may be doing using the NLTK here....