Navigation
Home
Up
Jens
Subpages

The ususal disclaimer

The purpose of this page is to give a little more than an introduction to binary serial protocols. Ther might be errors typos etc. Now you are warned

So the given example code is not intended to be used unmodified in real life.

A Simple Binary Serial Protocol

Exchanging information in a system do often use serial connections.

That might be I2C, SPI, CANBUS, ethernet and standard serial lines.

On this page we will look into use of a standard serial connection for data packet transfer.

We will only give a sketch and will not cover topics like checksum validation of packet integrity,
buffering issues, timing and other stuff, loss of bytes or data etc.
Things and facilities that can be added to a simple solution - if needed.

The asynchronous isue

Communication by use of serial communication may introduce some problems.

A serial link is asynchronous, meaning t hat sender do not know when receiving part start reading.

A sender transmit the message as shown below

Sender transmit:  helloYou

The receiver is a little late in starting receiving so a number of bytes are lost.

So the receiver might receive only the tail of the message packet.

Reveiver receives:  lloYou

Data - binary data

We are heading for a byte oriented protocol. A byte can be from 0 -> 255 ( 0 -> 2^8-1)
In the following examples we will use visible bytes (when printing) ABCD etc,
but we can ofcourse use all values.

Basic idea

Information is organized in packages. You can compare it with a written letter in an envelope.

So we just have to pass an envelope to a destination.

We do define the letter as a number of bytes.

Taking this to serial communication we will show a way to transmit a packet: a number of bytes.

Block delimiters or frame or packet idea

The lowest entity on the serial link is a byte or char (8 bit entity aka int8_t in C)

To ensure that the receiver can identify a block we will use two uniq byte values one for start and one for end of block.

So a package transmission will be

  1. send uniq start-of-package symbol

  2. send packet data

  3. send uniq stop-of-package symbol

SThose two symbols is selected to be each a byte with a uniq value.

In this way we will envelope the data in a packet with a uniq start byte and a uniq stop byte It can be any value (0..255)

A quick example

In the following example we will use 

start delimiter:  $  (dollar character,  byte value /ascii value 0x24)

stop delimiter:   #  (hash tag character, byte value /ascii value 0x23)

Data packet to transmit:    HelloWorld

We will transmit:      $HelloWorld#

Quick and dirty C code for TX(transmit)

data is defined as an array of a given size.

const byte startDelim='$',
           stopDelim='#';

 // data pointer shall point on your data to b TX'ed
// in Arduino language :-)
void txPkg( byte *data, int lgt)
{
  Serial.write(startDelim);

  for (int i = 0 ; i < lgt; i++) {
    Serial.write(*(data+i));
  }

  Serial.write(stopDelim);
}

The simple idea for receive a package is to read and skip received bytes until we read the start delimiter (in this example $)

Next read data and save them until we read the stopdelimiter (in this example

It is left to the reader to code it

// simple skip until start of package

void findStartOfPackage()
{
int i;  // bw read returns an int so they can give us a -1 is there is no data avaibale
byte b;
  while (1) {
    i = Serial.read();
    if (i == -1)
		continue; // read buffer empty
    b = (byte)i;
    if (b == startDelim)
		return:  // found start :-)
  }
}

void readPackage()
{
int i;  // bw read returns an int so they can give us a -1 is there is no data avaibale
byte b;

  findStartOfPackage();

  while (1) {
    i = Serial.read();
    if (i == -1)
		continue; // read buffer empty so nothing read
    b = (byte)i;  // we take byte part of integer == data byte
    if (b == stopDelim)
		return:  // found start :-)
	//save the received byte here in a buffer - left to you
  }
}

Receiving a block

As stated the receiver is not synchronized with the sender.

(It can be wise to have a mximum lenght of telegram so receiver
abort if to many bytes have been read before coming to the stop delimiter.
There can be bit errors in data - you never know - but rare on a wired connection.

  
graphviz code for the figure using http://viz-js.com/
digraph G {
initRXBuf -> readByte;
readByte ->readByte[label="  $ was not read"];
readByte -> readDataByte[label="  $ was read"];
readDataByte -> saveDataByte[label=" # was not read  "];
saveDataByte -> readDataByte[label=" read one more"];
readDataByte -> deliverData[label=" # was read"];
deliverData -> initRXBuf;
}

This should be fairly easy to implement in statemachine.

A built in problem - delimiters occurs on the data

In this example we do use $ and

Because we to use $ and 

ok Telegram:  $hello you#

Bad telegram: $hello$you#

Bad telegram: $hello#you#

In the two bad cases the receiver can read you and hello instead of hello you.

Delimiter values occurs in data

In a binary protocol all value (0->255) can occur

So start and stop delimiter values might occur in data.

We need to modify these values before sending.

For this we need a way to substitute these values with someting else.

We do introduce an escape character or byte so we do a substition during transmit

We do select % here in this example but you can use any value

Substitution rules

all byte values are transmitted as there are

except:  $ is transmitted as %1
         # is transmitted as %2
         % is transmitted as %%

could also be
         $ is transmitted as %a
         # is transmitted as %b
         % is transmitted as %e

just not use $ o # as second byte :-)

So the bytes with start, stop and escape value($#%) will replaced with two bytes.

So your telegram can grow in length and there in transmission time.

A very bad packet could be $$$$$ which will be send as %1%1%1%1%1


An example we want to transmit herR$og#og%iData (remember space is also a character but not used here)

data that sender want to transmit

herR$og#og%iData

We do substitute acc to rule above:
  $ ->  %1
  # ->  %2
  % ->  %%

The data for transmission becomes
herR%1og%2og%%iData

Putting the data in the frame we have to send
$herR%1og%2og%%iData#

The state machine above will receive

herR%1og%2og%%iData

And will use the substitution rule inverse

herR%1og%2og%%iData

%1 -> $
%2 -> #
%% -> %

so we do get

herR$og#og%iData

A schematic Finite State Machine for decoding

  
//graphviz code for the figure using http://viz-js.com/
digraph G {
decodeRXBuf -> readByte;
readByte ->saveByte[label="  % was not read"];
saveByte -> readByte;
readByte-> readOneMore[label="  % was read"];

readOneMore -> setByteToDollar[label="1 was read"];
readOneMore -> setByteToHashTag[label="2 was read"];
readOneMore -> setByteToPercent[label="% was read"];
setByteToDollar -> saveByte;
setByteToHashTag ->saveByte;
setByteToPercent->saveByte;
readByte->endOfDecoding[label="nothing read"];
}

Final Comment

There is missing a lot in the state machines above
like …

  • handling messages are too long

  • missing start and stop delimiters due to errors in commmunication

  • maybe my state machine ideas is not 100 % correct

  • and much more I think

happy hacking :-)

Jens

no guarantee for no errors in this page