MQTT Test Suite ¶

Contents

MQTT Test Suite

MQTT Protocol ¶

A very brief summary of MQTT from the FAQ: MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks.

Note

We provide an annotated version of the official MQTT specification which can be directly referenced (e.g. [MQTT-1.5.3-1] and [MQTT-1.5.3-2])

From a general and abstract perspective MQTT has two basic architectures for testing. This architecture directly reflects the choice of your SUT. We will call the first architecture Broker Testing. A MQTT Broker is the SUT as shown in the figure blow:

The second major architecture we will call Client Testing as now, the Client is in focus as the SUT.

Now we can start to extract different configurations from the test architectures. The image below depicts the step of retrieving test configurations from the architecture:

The output of this exemplary step is a test configuration (CF01) where the Broker is the SUT and the TS takes the role of a MQTT Client.

The MQTT test suite uses four test configurations in order to cover the different test scenarios. In these configurations, the tester simulates one or several MQTT clients or brokers implementing the MQTT protocol.

ID:	MQTT_Conf_01
Description:	The MQTT Broker is the IUT and the TS takes the role of a MQTT Client

ID:	MQTT_Conf_02
Description:	The MQTT Broker is the IUT and the TS takes the role of multiple MQTT Clients.

ID:	MQTT_Conf_03
Description:	The MQTT Client is the IUT and the TS takes the role of a MQTT Broker. For this configuration an optional UT might be required.

ID:	MQTT_Conf_04
Description:	As well the MQTT Broker as the MQTT Client, each is a IUT in this configuration. The part of the UT from the previous configuration is here replaced by a concrete application.

Test Purposes ¶

TODO: link to .tplan2 from GitHub and .pdf from ETSI

  Test Purpose {
    TP Id TP_MQTT_Broker_CONNECT_001

    Test objective
    "The IUT MUST close the network connection if fixed header flags in CONNECT Control Packet are invalid"
    Reference
      "[MQTT-2.2.2-1], [MQTT-2.2.2-2], [MQTT-3.1.4-1], [MQTT-3.2.2-6]"
    PICS Selection PICS_BROKER_BASIC

    Expected behaviour
    ensure that {
      when {
          the IUT entity receives a CONNECT message containing
          header_flags indicating value '1111'B;
      }
      then {
          the IUT entity closes the TCP_CONNECTION
      }
    }
  }

Test System ¶

TODO: describe Test System -> TTCN-3 code

Test Cases ¶

You can find all MQTT Tests on GitHub.

We will examine the procedure of a single TC in order to get the understanding of the code structure.

The code block below shows the TTCN-3 implementation of the TC TC_MQTT_BROKER_CONNECT_01 for the cohesive TP TP_MQTT_BROKER_CONNECT_01

/*
* @purpose The IUT MUST close the network connection if fixed header flags in CONNECT Control Packet are invalid
*
* @reference [MQTT-2.2.2-2], [MQTT-3.1.4-1], [MQTT-3.2.2-6]
*/
testcase TC_MQTT_BROKER_CONNECT_001() runs on MQTT_Client
{
  if(f_init("mqtt_client", "mqtt_server"))
  {
    f_TC_MQTT_BROKER_CONNECT_001();
  }
  f_cleanUp();
}

Let’s have a deeper look into the details of a TC. The first block comment contains only two TTCN-3 documentation tags but these give us a direct connection between a TC, a TP, and the MQTT specification.

/*
* @purpose The IUT MUST close the network connection if fixed header flags in CONNECT Control Packet are invalid
*
* @reference [MQTT-2.2.2-2], [MQTT-3.1.4-1], [MQTT-3.2.2-6]
*/

The signature of a TTCN-3 TC contains many information which help us to reflect the Test Architecture.

testcase TC_MQTT_BROKER_CONNECT_001() runs on MQTT_Client

We have a distinct name for the TC which can be easy mapped to it’s cohesive TP. (see in the MQTT |tp| catalogue for TP_MQTT_BROKER_CONNECT_001) The signature tells us also, that this TC will be executed on a Client (runs on MQTT_Client).

The body of the TC is used to initialize the test configuration and start the TC behaviour which is wrapped into a single function.

  if(f_init("mqtt_client", "mqtt_server"))
  {
    f_TC_MQTT_BROKER_CONNECT_001();
  }
  f_cleanUp();

Test Case Functions ¶

How does a TC function look like?

function f_TC_MQTT_BROKER_CONNECT_001() runs on MQTT_Client
{
  var UTF8EncodedString v_clientId := f_getClientId();

  var template MQTT_v3_1_1_Message v_conMsg := t_connect_flags(p_client_id := v_clientId, p_flags := '1111'B);
  f_send(valueof(v_conMsg));

  if(f_receiveNetworkClosedEvent())
  {
    setverdict(pass, "IUT closed the Network Connection correctly");
  }
  else
  {
    setverdict(fail, "IUT MUST close the Network Connection");
    f_disconnectMqtt();
  }
}

While syntactically this function might appear confusing, though the semantic behind is quite powerful and simple. Let’s go through the implemented TC and the according TP.

MQTT Test Suite ¶

MQTT Protocol ¶

Test Configurations ¶

Test Purposes ¶

Test System ¶

Test Cases ¶

Test Case Functions ¶

Translation Port ¶

Generic Functions ¶

Test Templates ¶

Default Behaviours ¶