## V4 - View Voltha Async Events with Kafkacat

### Test Objective

* Demonstrate and verify the async event stream (output) from Voltha

Voltha is sending asynchronous events to a Kafka "bus" system.
There are four types of events emitted by Kafka, using four separate Kafka
"topics":

* *heartbeat*: Voltha emits a periodic heart-beat through this channel. By default
the Kafka topic name for this channel is ```voltha.heartbeat```
* *kpi/performance metrics*: Voltha uses this channel to send out consolidated
 performance metric data about its internal components as well as about the
 devices managed by Voltha. While the channel is live, the metrics are limited
 to a few sample metrics at the moment (work in progress).  By default the
 Kafka topic name for this channel is ```voltha.kpis```.
* *alarms*: Voltha uses this channel to send out consolidated alarm signals,
covering the devices it manages as well as alarms about its internal operation.
 This is not used yet, albeit the internal plumbing in Voltha exists. By default
 the Kafka topic name for this channel is ```voltha/alarms```.
* *events*: Voltha uses this channel to send out consolidated async events 
 (other than alarms), covering the devices it manages as well as events about
  its internal operation. This is not used yet, albeit the internal plumbing 
  in Voltha exists. 
  By default the Kafka topic name for this channel is ```voltha/events```.

In a production deployment it is expected that other subsystems will consume
these Kafka topics. 
Our goal here is to demonstrate and verify that Voltha is working as expected.


### Test Configuration

* Voltha ensemble is launched (V1)

### Test Procedure

In order to show the Kafka channels, we use a command line tool ```kafkacat```
to "tune into" the channels.
To run kafkacat we need to know the port number of the kafka broker. Use the
 following command to retrieve the assigned port to Kafka:

```
docker inspect compose_kafka_1 | \
    jq -r '.[0].NetworkSettings.Ports["9092/tcp"][0]["HostPort"]'
```

For example, it may say:

```
32769
```

This is the port number Kafka is reacheable at the integration server.

To show the topics that have data, run the command (make sure you use the
port number that you retreived in the above step):

```
kafkacat -b localhost:32769 -L
```

Here is what you should see:

```
(venv-linux) ubuntu@voltha:/voltha$ kafkacat -b localhost:32769 -L
Metadata for all topics (from broker -1: localhost:32769/bootstrap):
 1 brokers:
  broker 1001 at 10.0.2.15:32769
 2 topics:
  topic "voltha.kpis" with 1 partitions:
    partition 0, leader 1001, replicas: 1001, isrs: 1001
  topic "voltha.heartbeat" with 1 partitions:
    partition 0, leader 1001, replicas: 1001, isrs: 1001
```

There are only two topics shown from the four we described above. The reason
is that the other topics have no published data yet.

In order to show data published to the heartbeat channel, use the following
command (*Make sure you use the correct port number!*):

```
kafkacat -b localhost:32769 -C -t voltha.heartbeat \
    -f 'Topic %t [%p] at offset %o: key %k: %s\n'
```

The above will show all existing data in the queue and will stay connected
and print messages as they arrive ("tail" the channel). You can use Ctrl-C
to interrupt the tailing and return to the Linux prompt.

Example output, showing only the first few lines:

```
(venv-linux) ubuntu@voltha:/voltha$ kafkacat -b localhost:32769 -C -t voltha.heartbeat -f 'Topic %t [%p] at offset %o: key %k: %s\n'
Topic voltha.heartbeat [0] at offset 0: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291374}
Topic voltha.heartbeat [0] at offset 1: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291384}
Topic voltha.heartbeat [0] at offset 2: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291394}
Topic voltha.heartbeat [0] at offset 3: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291404}
Topic voltha.heartbeat [0] at offset 4: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291414}
Topic voltha.heartbeat [0] at offset 5: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291424}
Topic voltha.heartbeat [0] at offset 6: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291434}
Topic voltha.heartbeat [0] at offset 7: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291444}
Topic voltha.heartbeat [0] at offset 8: key : {"ip": "172.18.0.9", "type": "heartbeat", "voltha_instance": "compose_voltha_1", "ts": 1484291454}
```

Similarly, you can watch KPI metrics as follows:

```
kafkacat -b localhost:32769 -C -t voltha.kpis \
    -f 'Topic %t [%p] at offset %o: key %k: %s\n'
```

This may show only two Voltha metrics, yet it still verifies the proper operation
 of the KPI metric collection mechanism. A sample output is as follows, truncated
 to only the first 15 lines:
 
```
(venv-linux) ubuntu@voltha:/voltha$ kafkacat -b localhost:32769 -C -t voltha.kpis -f 'Topic %t [%p] at offset %o: key %k: %s\n'
Topic voltha.kpis [0] at offset 0: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 20, "rss-mb": 79}}, "type": "slice", "ts": 1484291374}
Topic voltha.kpis [0] at offset 1: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291389}
Topic voltha.kpis [0] at offset 2: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291404}
Topic voltha.kpis [0] at offset 3: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291419}
Topic voltha.kpis [0] at offset 4: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291434}
Topic voltha.kpis [0] at offset 5: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291449}
Topic voltha.kpis [0] at offset 6: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291464}
Topic voltha.kpis [0] at offset 7: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291479}
Topic voltha.kpis [0] at offset 8: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291494}
Topic voltha.kpis [0] at offset 9: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291509}
Topic voltha.kpis [0] at offset 10: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291524}
Topic voltha.kpis [0] at offset 11: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291539}
Topic voltha.kpis [0] at offset 12: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291554}
Topic voltha.kpis [0] at offset 13: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291569}
Topic voltha.kpis [0] at offset 14: key : {"data": {"voltha.internal.compose_voltha_1": {"deferreds": 173, "rss-mb": 79}}, "type": "slice", "ts": 1484291584}
```

### Pass/Fail Criteria

* The above feeds (topics) should exist in Kafka and have data in both
topics.
