Introducing AWS IoT Core’s Direct Messaging: Decrease-cost server-to-device communication with higher observability

AWS IoT Core now helps direct messaging for point-to-point communication. Beforehand, sending a message to a single IoT system required publishing to an MQTT matter the system subscribed to, with no built-in supply affirmation from the system. With the brand new direct messaging functionality, you’ll be able to ship a message to any system linked to AWS IoT Core, lowering messaging price in comparison with publish-subscribe (Pub-Sub) messaging for one-to-one communication patterns, and offering a supply acknowledgment from the IoT system. AWS IoT Core additionally makes use of the supply acknowledgment to supply detailed API response codes and emit Amazon CloudWatch Logs, so you will have visibility into message supply standing and failure causes.

Level-to-point communication between backend servers and IoT gadgets is a typical sample in linked system architectures. It reveals up in use circumstances like sending firmware updates to a wise dwelling equipment, pushing transaction updates to a fee system, or controlling a wise automobile. For these one-to-one interactions, routing by way of a Pub-Sub message dealer isn’t environment friendly, since you’re not utilizing the fan-out profit that Pub-Sub was designed to supply. As well as, you obtain the message supply acknowledgment from the Pub-Sub dealer quite than from the IoT system, requiring you to construct your individual customized answer for supply affirmation.

What’s Direct Messaging?

AWS IoT Core introduces the SendDirectMessage HTTP API. With this API, you’ll be able to ship a message to a particular IoT system or shopper recognized by its MQTT shopper ID. Somewhat than publishing to an MQTT matter and counting on subscription matching, AWS IoT Core routes the message point-to-point to the goal system. You skip the subscription-matching layer, there’s no fan-out, and you’ve got the choice to obtain affirmation from the shopper.

The message is delivered to the system on its current MQTT connection (over TCP) to AWS IoT Core, with no device-side adjustments required. The API helps two modes:

1. With out supply affirmation – The message is delivered at MQTT High quality of Service (QoS) 0. The HTTP API returns 200 OK after AWS IoT Core accepts and dispatches the message.
2. With supply affirmation – The message is delivered at MQTT QoS 1. The HTTP API returns 200 OK solely after the system acknowledges receipt with an MQTT PUBACK (publish acknowledgment), offering true end-to-end supply affirmation. If the system doesn’t acknowledge throughout the timeout, the API returns 504 Gateway Timeout.

Key advantages

Constructed-in supply affirmation

With supply affirmation enabled (affirmation=true), you obtain acknowledgment immediately from the system, not solely from the dealer. AWS IoT Core delivers the message at QoS 1 and waits for the system’s PUBACK earlier than returning 200 OK. This removes the necessity to construct customized acknowledgment logic.

Fast offline system suggestions

With conventional Pub-Sub, a server publishing to a subject receives a profitable response no matter whether or not the system obtained the message. With Direct Messaging, if the goal system isn’t linked, the API returns 404 Not Discovered instantly. The HTTP response message and Amazon CloudWatch Logs describe the precise motive. For instance, when the response message states that the goal shopper ID isn’t linked however has an energetic persistent session, the system has an unexpired persistent session however is at the moment offline.

Value optimization

Should you use direct messaging with out supply affirmation, you pay for a single Direct Message as an alternative of separate publish-in and publish-out operations. Should you use direct messaging with supply affirmation, you pay for a single Direct Message with Affirmation as an alternative of separate publish-in, publish-out, and publish-ack operations. For particulars, see the AWS IoT Core pricing web page.

No device-side adjustments

Direct messages are delivered on current MQTT connections already established with AWS IoT Core. No firmware updates, SDK adjustments, or new matter subscriptions are required. In case your system already subscribes to the goal matter, Direct Messaging works instantly. In case your system doesn’t subscribe to the subject, confirm that your MQTT shopper library doesn’t filter messages on unsubscribed subjects. Most manufacturing shoppers (together with the AWS IoT Machine SDKs) deal with this accurately.

Use circumstances

Direct Messaging is the appropriate selection when:

• That you must ship a message to a particular system (not a multicast).
• That you must know whether or not the system truly obtained the message.
• You wish to simplify retry and error-handling logic.

Examples: server-to-device instructions (locking a automobile, toggling a wise equipment, pushing a configuration replace), system acknowledgments, cost-sensitive high-volume messaging, and real-time notifications.

Conventional Pub-Sub stays the appropriate selection whenever you want fan-out (one message to many subscribers) or message queuing for offline gadgets.

Getting began

On this publish, you’ll configure the AWS Id and Entry Administration (IAM) insurance policies, ship a Direct Message in three other ways, and obtain it on a linked system.

Stipulations

It’s essential to have the next conditions to observe together with this publish.

• An AWS account with AWS IoT Core configured.
• IoT gadgets registered and linked by way of MQTT 3.1.1 or MQTT 5.0. See the AWS IoT Core Developer Information to discover ways to join gadgets.
• Backend server code that calls the Direct Messaging API (server-side adjustments solely).

Step 1: Configure authorization

Each the sender and the receiver require particular coverage actions. The sender wants iot:SendDirectMessage permission with the goal shopper’s Amazon Useful resource Identify (ARN) because the useful resource. You’ll be able to optionally limit which subjects can be utilized with the iot:Subject situation key.

The next sender coverage permits direct messages to shopper myDevice on the subjects instructions/reboot and instructions/replace. For SigV4-authenticated backend servers, add this to an IAM coverage. For X.509-authenticated gadgets, add it to an AWS IoT Core coverage.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "iot:SendDirectMessage",
            "Resource": "arn:aws:iot:us-west-2:123456789012:client/myDevice",
            "Condition": {
                "StringEquals": {
                    "iot:Topic": ["commands/reboot", "commands/update"]
                }
            }
        }
    ]
}

The receiver will need to have iot:Obtain permission on the goal matter. The receiver doesn’t want iot:Subscribe. Direct Messaging delivers to the linked shopper with out requiring a subject subscription.

The next receiver coverage permits iot:Obtain on two particular subjects:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "iot:Receive",
            "Resource": "arn:aws:iot:us-west-2:123456789012:topic/commands/reboot"
        },
        {
            "Effect": "Allow",
            "Action": "iot:Receive",
            "Resource": "arn:aws:iot:us-west-2:123456789012:topic/commands/update"
        }
    ]
}

You too can use a wildcard to obtain direct messages on any matter beneath a prefix:

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Effect": "Allow",
            "Action": "iot:Receive",
            "Resource": "arn:aws:iot:us-west-2:123456789012:topic/commands/*"
        }
    ]
}

For extra coverage examples, together with sending to any shopper on particular subjects or on any matter, see Direct messaging coverage examples.

Step 2: Ship a Direct Message

Senders make an HTTP POST request to a client-specific URL:

POST https://{IoT_data_endpoint}/connections/{url_encoded_client_id}/messages?matter={url_encoded_topic}&affirmation=true&timeout=10

{IoT_data_endpoint} is your account’s AWS IoT system knowledge endpoint. The next three examples ship the identical instructions/reboot message to shopper myDevice with supply affirmation.

Utilizing curl (X.509 shopper certificates authentication, port 8443):

curl --tlsv1.2 
  --cacert AmazonRootCA1.pem 
  --cert system.pem.crt 
  --key personal.pem.key 
  --request POST 
  --data '{"motion": "reboot"}' 
  "https://{IoT_data_endpoint}:8443/connections/myDevice/messages?matter=commandspercent2Freboot&affirmation=true&timeout=10"

Utilizing the AWS Command Line Interface (AWS CLI):

aws iot-data send-direct-message 
  --client-id myDevice 
  --topic instructions/reboot 
  --confirmation 
  --timeout 10 
  --payload '{"motion": "reboot"}' 
  --cli-binary-format raw-in-base64-out 
  --region us-west-2 
  --endpoint-url https://{IoT_data_endpoint}

The --cli-binary-format raw-in-base64-out possibility is required with AWS CLI v2 so the --payload worth is shipped as-is. To make it the default, run aws configure set cli-binary-format raw-in-base64-out. This command requires AWS CLI v2 model 2.34.57 or newer.

Utilizing the AWS SDK for Python (Boto3):

import boto3
import json

shopper = boto3.shopper("iot-data", region_name="us-west-2")

response = shopper.send_direct_message(
    clientId="myDevice",
    matter="instructions/reboot",
    affirmation=True,
    timeout=10,
    contentType="utility/json",
    payload=json.dumps({"motion": "reboot"}).encode("utf-8"),
)

# The response accommodates a standing message and a traceId for troubleshooting.
print(response["message"], response["traceId"])

With affirmation=true, the API waits for the system to acknowledge. A 200 OK confirms end-to-end supply. A 504 Gateway Timeout signifies the receiver didn’t acknowledge throughout the timeout interval. The supply state is ambiguous, so implement idempotent dealing with in the event you retry.

Request parameters:

Step 3: Obtain Direct Messages on the system

Your current MQTT shopper receives Direct Messages on the desired matter. In contrast to normal MQTT Pub-Sub, the system doesn’t want an energetic subscription. Direct Messages are delivered primarily based on the shopper ID. The system can nonetheless optionally subscribe to the subject, but it surely’s not required for supply.

The QoS degree of the delivered message is ready by the sender’s affirmation parameter, not by the receiver’s subscription. When affirmation=true, the message arrives at QoS 1 and the shopper should ship a PUBACK to acknowledge supply. Most MQTT shopper libraries do that routinely. When affirmation=false, the message arrives at QoS 0 with no acknowledgment required. Ensure your shopper handles each QoS 0 and QoS 1 incoming messages accurately.

Monitoring with CloudWatch

AWS IoT Core returns detailed HTTP response codes for each SendDirectMessage name. While you allow AWS IoT Core CloudWatch logging, the service additionally emits SendDirectMessage occasion logs that embody a machine-readable motive area, so you’ll be able to construct automated retry logic, monitor system connectivity, and troubleshoot supply points programmatically. Assessment the HTTP response message or CloudWatch logs to establish the precise motive for any failure.

HTTP response standing codes:

Limitations and concerns

• No message queuing – Direct Messages aren’t queued for offline gadgets. Use Pub-Sub with QoS 1 and chronic periods for message persistence.
• No message retention – The MQTT retained flag isn’t supported. Use Amazon DynamoDB or AWS IoT Machine Shadow for state synchronization.
• No Guidelines Engine processing – The AWS IoT guidelines engine doesn’t course of Direct Messages.
• Reserved subjects – Direct Messaging helps customized subjects. Subjects should not begin with $ and should not be AWS IoT reserved subjects.
• Consumer ID restrictions – Consumer IDs should not exceed 128 characters and should not begin with $. Consumer IDs containing characters which are invalid in HTTP requests (corresponding to /) should be percent-encoded within the URL path.
• Limits and quotas – See the AWS IoT Core developer information and repair quotas for payload measurement, matter depth, and API limits.
• Protocol – Works with current MQTT 3.1.1 and MQTT 5.0 shoppers. (MQTT 5.0 properties corresponding to content material sort and response matter are usually not delivered to MQTT 3.1.1 shoppers.)

Availability

AWS IoT Core Direct Messaging is out there immediately in all AWS Areas the place AWS IoT Core is out there.

Conclusion

On this publish, you noticed how AWS IoT Core Direct Messaging routes server-to-device messages point-to-point by MQTT shopper ID, returns a real end-to-end supply acknowledgment whenever you decide in with affirmation=true, and stories actionable HTTP response codes for offline or unauthorized targets. The aptitude works on current MQTT connections, so you’ll be able to undertake it with out modifying system firmware.

To get began, see the Direct Messaging matter within the AWS IoT Core Developer Information. For pricing particulars, see the AWS IoT Core pricing web page. Check in to the AWS IoT console to start out sending direct messages to your linked gadgets.

Concerning the authors