AWS SQS: CASE STUDY

Amazon Web Services are leading the Cloud market with nearly 40% of the market with them. AWS provides over 200 services, thus AWS is capable of providing unified solutions for most companies. One of the many services provided by AWS for Message Queue is AWS SQS

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.

What is a Messaging Queue Service?

A message queue is a form of asynchronous service-to-service communication used in serverless and microservices architectures. Messages are stored on the queue until they are processed and deleted. Each message is processed only once, by a single consumer. Message queues can be used to decouple heavyweight processing, to buffer or batch work, and to smooth spiky workloads.

Note: The size of the message data is a minimum 1Kb and a maximum of 256Kb.

Why Do we need a Messaging Queue?

Consider the above scenario of Order confirmation and mailing the customer an acknowledgment of the confirmed order. If a consumer buys an item, the records are to be stored in the Database and to be sent to a mail server to get the confirmation of the order via email.

In the tightly coupled architecture (The red path), if the server goes down there would be a loss of data transmitted at the moment. A tightly coupled system is a concept of system design and computing where every hardware and software component are linked together in such a manner that each component is dependent upon the other. So if the customer confirms the order and if the database server is down (even for a fraction of a second) the data of the order confirmation would not be recorded in the system but the email for the order confirmation would be received by the customer. This is inadmissible.

Whereas, a loosely coupled system is one in which each of its components has, or makes use of, little or no knowledge of the definitions of other separate components. In the above architecture, the green path follows a loosely coupled system architecture. Amazon SQS is a component of the loosely coupled architecture. When the customer confirms the order the data is sent to the Message Queue. The Message Queue keeps the data until the data is been received by the servers in a specific order if the servers go down due to some reason. The eliminates the potential data loss as compared to a tightly coupled architecture. System Components and Services like Amazon SQS are termed as middleware.

Amazon SQS

Amazon SQS is a distributed queuing webservice for exchanging messages between software systems asynchronously. SQS is used to develop large-scale distributed systems with decoupled components without worrying about creating and managing the queue.

SQS comes in two modes: Standard and FIFO (First-In-First-Out).

FIFO queue maintains the order of the messages that are produced and consumed providing a once-only delivery mechanism. Use Case: guaranteed order, once only delivery.

Standard queue attempts to preserve the order but is flexible, and it provides at least one delivery. The message in the Standard queue can be delivered more than once and the standard Queue offers high throughput. Use Case: Faster messaging, can be duplicates, messages sent out of order.

General Use Case: One-to-one messaging, sending emails, form, and file processing.

Message Availability Limit: By default, the messages in SQS will be deleted after 4 days, but this can be extended up to 14 days

Amazon SQS provides common middleware constructs such as dead-letter queues and poison-pill management.

A dead letter queue is a queue that other (source) queues can target for messages that can’t be processed successfully. You can set aside and isolate these messages in the dead letter queue to determine why their processing did not succeed. This feature makes troubleshooting simple.

Poison pills are special messages that can be received, but not processed. They are a mechanism used in order to signal a consumer to end its work so it is no longer waiting for new inputs.

Features

Unlimited Throughput:

A practically infinite number of transactions per second (TPS) are supported by standard queues for each API action.

At-Least-Once Delivery:

A message is delivered at least once, but occasionally more than one copy of a message is delivered.

Exactly-Once Processing:

A message is sent once and remains available until it is processed and deleted by the recipient. There are no duplicates added to the queue.

First-In-First-Out Delivery:

The order of sending and receiving messages is rigorously adhered to (i.e. First-In-First-Out).

Benefits

• Security — You have complete control over who can send and receive messages from an Amazon SQS queue. Server-side encryption (SSE) protects the contents of messages in queues using keys controlled by the AWS Key Management Service, allowing you to send sensitive data (AWS KMS).
• Durability — Amazon SQS saves your messages on numerous servers to ensure their security. At-least-once message delivery is supported by standard queues, and exactly once message processing is supported by FIFO queues.
• Availability — Amazon SQS makes use of redundant infrastructure to provide highly concurrent message access as well as high availability for message production and consumption.
• Scalability — Without any provisioning instructions, Amazon SQS can process each delayed request individually, scaling transparently to manage any demand surges or spikes.
• Reliability — Messages in Amazon SQS are locked during processing so that several producers and consumers can send and receive messages at the same time.
• Customization — Your queues don’t have to be identical; for example, you can set a queue’s default delay. You can use Amazon Simple Storage Service (Amazon S3) or Amazon DynamoDB to store the contents of messages greater than 256 KB, with Amazon SQS holding a pointer to the Amazon S3 object, or you can divide a large message into smaller messages.

CASE STUDY:

• redBus

redBus is expanding its AWS solution to include Amazon SQS and SNS for monitoring, alerts, and intercommunication. “Amazon SQS is an especially good solution for enabling messaging between external applications and our applications.”

• BMW

The BMW Group is using AWS for its connected car application that collects sensor data from BMW 7 Series cars to give drivers dynamically updated map information. BMW built its new car-as-a-sensor (CARASSO) service in only six months leveraging Amazon SQS, Amazon S3, Amazon DynamoDB, Amazon RDS, and AWS Elastic Beanstalk.

NASA

More than 1,40,000 still images, audio, recordings, and videos are available through the NASA Image and Video Library, which documents NASA’s more than a half-century of achievements in exploring the great unknown. Amazon SQS is used to separate incoming jobs from pipeline operations, and Amazon Simple Notification Service is used to start the processing pipeline when new content is added.

Connect me on my LinkedIn as well.