Project information

  • Development Language: Golang, Ruby on Rails
  • Technology: ECS, SNS, SQS, EventBridge, S3, Firehose, Snowflake
  • Coolness Factor: 🌟🌟🌟🌟🌟

Notification Pipeline

I loved this project because it was my first deep-dive into golang. The goal was to process, transform, and normalize a large amount of unstructured data in the most efficient way possible. Using golang, I was able to significantly increase the speed of processing by leveraging concurrency. Because the contents of a single incoming data record could potentially end up in 3-4 different tables, I could have a separate go routine work individually on each slice of data. This opened my eyes to the world of golang data processing which I have since used in various projects.