Read from a Service

To allow for communication, services can read from each others outputs. The service.yaml file and a service's source code are tightly coupled to allow for this.

Elias Groot

Software Lead, Project Administrator

Prerequisites

You have roverctl and Docker installed
You have VS Code and the Devcontainer plugin installed
You cloned a service template using roverctl and opened it in its Devcontainer
You are familiar with using roverctl-web to configure and execute a pipeline
You know how to upload your service using roverctl

How to Read

Suppose that we have a simple pipeline of two services: a producer service that generates data and outputs it, and a consumer service that reads this data and uses it for its functionality.

For the consumer to know where and how to read, it must define its dependencies as input in its service.yaml definition. Turn to the service.yal of your template service:

service.yaml
...
inputs:
  - service: imaging
    streams:
      - path
...

The above declaration tells us that our example-service depends on data from an "imaging" service. This imaging service should write this data to its "path" stream.

Then in your service's source code, you can open a read stream and read from it using the APIs provided by roverlib for your language.

Go
Python
C
C++
Other languages

Open the main program in src/main.go.

roverlib-go exposes the GetReadStream() method, that can be used to open a read stream that corresponds to the one defined in the service.yaml file, like so:

src/main.go
func run(service roverlib.Service, configuration *roverlib.ServiceConfiguration) error {
	readStream := service.GetReadStream("imaging", "path")
	if readStream == nil {
		return fmt.Errorf("Failed to get read stream")
	}

    data, err := readStream.Read()
    if data == nil || err != nil {
        return fmt.Errorf("Failed to read")
    }
}

Open the main program in src/main.py.

roverlib-python exposes the GetReadStream() method, that can be used to open a read stream that corresponds to the one defined in the service.yaml file, like so:

src/main.py
def run(service : roverlib.Service, configuration : roverlib.ServiceConfiguration):
    read_stream : roverlib.ReadStream = service.GetReadStream("imaging", "path")
    if read_stream is None:
        raise ValueError("Failed to get read stream")

    data = read_stream.Read()
    if data is None:
        raise ValueError("Failed to read")

Open the main program in src/main.c.

roverlib-c exposes the get_read_stream() function, that can be used to open a read stream that corresponds to the one defined in the service.yaml file, like so:

src/main.c
int user_program(Service service, Service_configuration *configuration) {
    read_stream *read_stream = get_read_stream(&service, "imaging", "path");
    if (read_stream == NULL) {
        printf("Failed to get read stream\n");
    }

    ProtobufMsgs__SensorOutput *data = read_pb(read_stream);
    if (data == NULL) {
        printf("Failed to read\n");
    }
}

Open the main program in src/main.cpp.

The C++ template uses roverlib-c, which exposes the get_read_stream() function, that can be used to open a read stream that corresponds to the one defined in the service.yaml file, like so:

src/main.cpp
int user_program(Service service, Service_configuration *configuration) {
    read_stream *read_stream = get_read_stream(&service, "imaging", "path");
    if (read_stream == NULL) {
        printf("Failed to get read stream\n");
    }

    ProtobufMsgs__SensorOutput *data = read_pb(read_stream);
    if (data == NULL) {
        printf("Failed to read\n");
    }
}

You will need to parse the service information that is injected through the ASE_SERVICE environment variable manually. This environment variable contains the bootspec which you need to represent in the native format of your language. Then, you need to open a ZMQ socket based on the correct stream properties.

What to Read

When you know where to read from, you must also know what format the data you are reading is in. Because services can be written in many different languages, they must agree to all "speak" the same wire format, such as JSON.

While JSON could work, it is rather slow and lacks type safety. Hence, we have opted for the Protobuf wire format with a fixed collection of messages that you can send and receive, as defined in our rovercom package.

Protobuf definitions can be transpiled to serialization and deserialization code for virtually any programming language. Each roverlib already takes care of this for you. You can switch or match on the message type when reading.

For example, you can read an CameraSensorOutput message like so:

Go
Python
C
C++
Other languages

src/main.go
func run(service roverlib.Service, configuration *roverlib.ServiceConfiguration) error {
    readStream := service.GetReadStream("imaging", "path")
    if readStream == nil {
        return fmt.Errorf("Failed to get read stream")
    }

    data, err := readStream.Read()
    if data == nil || err != nil {
        return fmt.Errorf("Failed to read")
    }

    imagingData := data.GetCameraOutput()
    if imagingData == nil {
        return fmt.Errorf("Message does not contain camera output. What did imaging do??")
    }
}

src/main.py
def run(service : roverlib.Service, configuration : roverlib.ServiceConfiguration):
    read_stream : roverlib.ReadStream = service.GetReadStream("imaging", "path")
    if read_stream is None:
        raise ValueError("Failed to get read stream")

    data = read_stream.Read()
    if data is None:
        raise ValueError("Failed to read")

    imaging_data = data.camera_output
    if imaging_data is None:
        return ValueError("Message does not contain camera output. What did imaging do??")

src/main.c
int user_program(Service service, Service_configuration *configuration) {
    read_stream *read_stream = get_read_stream(&service, "imaging", "path");
    if (read_stream == NULL) {
        printf("Failed to get read stream\n");
    }

    ProtobufMsgs__SensorOutput *data = read_pb(read_stream);
    if (data == NULL) {
        printf("Failed to read\n");
    }

    ProtobufMsgs__CameraSensorOutput *imaging_data = data->cameraoutput;
    if (imaging_data == NULL) {
      printf("Message does not contain camera output. What did imaging do??");
      return 1;
    }
}

src/main.cpp
int user_program(Service service, Service_configuration *configuration) {
    read_stream *read_stream = get_read_stream(&service, "imaging", "path");
    if (read_stream == NULL) {
        printf("Failed to get read stream\n");
    }

    ProtobufMsgs__SensorOutput *data = read_pb(read_stream);
    if (data == NULL) {
        printf("Failed to read\n");
    }

    ProtobufMsgs__CameraSensorOutput *imaging_data = data->cameraoutput;
    if (imaging_data == NULL) {
        printf("Message does not contain camera output. What did imaging do??");
        return 1;
    }
}

You will need to transpile the rovercom message definitions to your language of choice manually.

Then, you can use the properties of the CameraSensorOutput message to find the track edges - if that is what the imaging service provides.

Bring Your Own Messages

We strongly recommend to use our messaging definitions, as you will enjoy first-class type safety and debugging support. However, if it does not fit your needs, you can always resort to reading and writing raw bytes using the roverlib, so that you can use your own serialization and deserialization logic.

Go
Python
C
C++
Other languages

src/main.go
func run(service roverlib.Service, configuration *roverlib.ServiceConfiguration) error {
    readStream := service.GetReadStream("imaging", "path")
    if readStream == nil {
        return fmt.Errorf("Failed to get read stream")
    }

    data, err := readStream.ReadBytes()
    if err != nil {
        return fmt.Errorf("Failed to read bytes")
    }
}

src/main.py
def run(service : roverlib.Service, configuration : roverlib.ServiceConfiguration):
    read_stream : roverlib.ReadStream = service.GetReadStream("imaging", "path")
    if read_stream is None:
        raise ValueError("Failed to get read stream")

    data = read_stream.ReadBytes()
    if data is None:
        raise ValueError("Failed to read bytes")

src/main.c
int user_program(Service service, Service_configuration *configuration) {
    read_stream *read_stream = get_read_stream(&service, "imaging", "path");
    if (read_stream == NULL) {
        printf("Failed to get read stream\n");
    }

    char buf[4096]; // your duty to allocate enough memory!
    int res = read_bytes(read_stream);
    if (res < 0) {
        printf("Failed to read bytes\n");
    }
}

src/main.cpp
int user_program(Service service, Service_configuration *configuration) {
    read_stream *read_stream = get_read_stream(&service, "imaging", "path");
    if (read_stream == NULL) {
        printf("Failed to get read stream\n");
    }

    char buf[4096]; // your duty to allocate enough memory!
    int res = read_bytes(read_stream);
    if (res < 0) {
        printf("Failed to read bytes\n");
    }
}

Completing the Pipeline

Open roverctl-web again and upload your service using roverctl. Enable only your service and the ASE "imaging" service. (This service can be installed from https://github.com/VU-ASE/imaging/releases/latest).

Press "start execution". The pipeline should start running successfully, but the Rover does not seem to drive. Oh no.

The Rover does not drive because there is no service enabled yet that takes in the outputs from your service and turns it into hardware signals to steer the servo and spin the motors. ASE provides the actuator service to do this for you, it can be installed from https://github.com/VU-ASE/actuator/releases/latest.

Once installed, enable your service, together with the ASE "imaging" and "actuator" services. Put the Rover on the ground and press "start execution".

The Rover will start driving in a circle while logging input it receives from the imaging service in the process. You can view the logs through roverctl-web, or by using the roverctl logs command.

# View the logs of my-example-service on Rover 12
roverctl logs elias my-example-service 1.0.0 --rover 12

Press "stop execution", so that we can alter the service's behavior.

Prerequisites​

How to Read​

What to Read​

Bring Your Own Messages​

Completing the Pipeline​

Prerequisites

How to Read

What to Read

Bring Your Own Messages

Completing the Pipeline