8 years, 4 months ago.

Serial looses characters or just blocks

Hi,

i am new to mbed (also to the whole microcontroller programming). I come from the java world and just started to learn mbed. I have a sample application which just accepts commands (of length 40 characters) from pc and answers depending on the command. For now i am just printing the received commands back to pc to check if i my board gets the whole command.

i have a nucleof103rb attached to pc (mac) through the debbuger usb port and have coolterm as the terminal to send the commands and also show them when they are sent back from the mbed to pc (baudrate on both is 9600).

The problem is that P1 - when i send less characters than 40 it blocks. P2 - when i send as fast as possible for a human many commands(40 characters per time through fast clicking of send button on the "send string..." window of coolterm) mbed does not get all the characters. If i send the commands slow (click send and wait and click send again) than the board gets all characters and prints them right.

Here is the code

#include <string>

#include "mbed.h"

//attributes - peripherals
Serial pc(SERIAL_TX, SERIAL_RX);
DigitalOut myled(LED1);

//attributes - program variables
char stringOverSerialBuffer[41];  	// buffer to store received string over pc
int newCommandFlag = 0;  			// flag for ISR

void serialDataCallback() {
	if (newCommandFlag) {
		//do not do anything as long as another command is not handled
		return;
	}

	if (pc.readable()) {
		//----------------------------------------
		pc.gets(stringOverSerialBuffer, 41);	//last character '\0' is added by c++

		// set flag
		newCommandFlag = 1;
	}

}

int main() {
	pc.attach(&serialDataCallback);  // attach pc ISR

	while (1) {
		while (!newCommandFlag) {
			pc.printf("waiting for command.\n");
			wait(1);
		}
		if (newCommandFlag) {  // if command has been read
			if (strlen(stringOverSerialBuffer) != 40) {
				pc.printf("unknown command");
			} else {
				pc.printf("received: %s \n\r", stringOverSerialBuffer);

				if (startsWith(stringOverSerialBuffer, "L_ON")) {
					pc.printf("turn light on.\n");
					myled = 1;

				} else if (startsWith(stringOverSerialBuffer, "L_OFF")) {
					pc.printf("turn light off.\n");
					myled = 0;
				}
			}

			// clear flag. do not reset flag until the command is handled. that way other interrupts will not be handled   
                       newCommandFlag = 0;  
		}
	}
}

int startsWith(const char *originalString, const char *prefixToCheck) {
	if(strncmp(originalString, prefixToCheck, strlen(prefixToCheck)) == 0)
		return 1;

	return 0;
}

i Have read a lot through the forum and the more i read the more i become confused on how to really(right) use the serial communication.

In forums is said that print f is wrong but

Q1 - how do you do debugging?

If i remove printf it works *more* reliable (still blocks when i send less than 40 characters - i am seeing it through setting debug points since i removed printf) but i am not sure if

Q2 - the code i have is really as it should be?

Q3 - Should i work with interrupt disabling and enabling?

Q4 - Should i instead of

pc.attach(&serialDataCallback); 

do just

pc.attach(&serialDataCallback, Serial::RxIrq);

Q5 - Should i better work with timouts to go out of blocked code or is there a better way?

I know that the question may sound silly but i hope i will become better ;) Any tips are really welcome.

2 Answers

8 years, 4 months ago.

I think you have a timing and buffer size issue. This is the sort of problem that it's very common to get tripped up by if you aren't used to embedded stuff that has to deal with the hardware.

Currently as soon as some data is received you copy it into your stringOverSerialBuffer buffer. The rx interrupt doesn't wait for the entire message, it triggers as soon as the first byte is received.

That removes that data from the serial port buffer rx and you end up with a string of length 1 and print out an error.

Printing out the error is slow and so takes time (printf is slow but more importantly sending data over the serial is very slow if the message is longer than the hardware Tx buffer.) During that time several more bytes are received. You read them (or at least the most recent few, the hardware receive buffer will be 16 bytes or less depending on the CPU, anything over that buffer size is lost), realize you have less than 40 bytes and print an error. This repeats until the end of the input string.

What you need to do is read the data from the serial port into your buffer as it arrives but only process the message once you have the whole thing.

e.g. (note - this is untested but I think it will work)

//attributes - program variables
char stringOverSerialBuffer[41];    // buffer to store received string over pc
int bytesRecieved = 0;

volatile bool newCommandFlag = false;             // flag for ISR - volatile so that the main loop picks up changes from the ISR.
 
void serialDataCallback() { 

    while (pc.readable()) { // read all available data
        if ((bytesRecieved  == 40) || newCommandFlag)  // avoid buffer overflow
          pc.getc();
        else {
          stringOverSerialBuffer[bytesRecieved] = pc.getc();   // get waiting data
          bytesRecieved++;
          if ((bytesRecieved == 40) || (stringOverSerialBuffer[bytesRecieved-1] == '\n')) {   // buffer full or a new line
            stringOverSerialBuffer[bytesRecieved] = 0;                                                              // append a null
            newCommandFlag  = true;
          }
    }
}
 
int main() {
    pc.attach(&serialDataCallback);  // attach pc ISR
 
    while (1) {
        while (!newCommandFlag) {
            pc.printf("waiting for command.\n");
            wait(1);
        }
        if (newCommandFlag) {  // if command has been read
            if (strlen(stringOverSerialBuffer) != 40) {
                pc.puts("unknown command");   // puts has less overhead than printf.
            } else {
                pc.printf("received: %s \n\r", stringOverSerialBuffer);
 
                if (startsWith(stringOverSerialBuffer, "L_ON")) {
                    pc.puts("turn light on.\n");
                    myled = 1;
 
                } else if (startsWith(stringOverSerialBuffer, "L_OFF")) {
                    pc.puts("turn light off.\n");
                    myled = 0;
                }
            }
 
            bytesRecieved = 0;  // clear the message size pointer so the next message is at the start of the buffer
            newCommandFlag = false;   // clear the flag.
        }
    }
}

To answer your questions: Q1 - toggling/setting LEDs is a good way to track the progress of the code with minimal impact on timings. If you want serial port tracing in time critical sections of code use putc to output a single character rather than printf and a long string. Assuming the serial port transmit buffer is empty putc has minimal overhead.

Q2 - Not really, see above.

Q3 - You shouldn't need to but you could disable them while processing the message. Your current code could lock up if a byte arrived at the wrong time. You ignore interrupts without reading the incoming data while newCommandFlag is set, the interrupt for that byte won't get generated again. If that was the last byte you will never see it. Some systems only generate an rx interrupt on the first byte, if you ignore it you'll never get another rx interrupt until you empty the buffer.

Q4 - They are effectively identical. RxIrq is the default interrupt type if not stated.

Q5 - No. the best way out of blocked code is to not block. There are situations where a timeout is useful (e.g. in my code above you could have a timeout if you received some data but didn't get 40 bytes or a line end within a reasonable time).

One other gotcha to keep in mind - If you have code along the lines of

int myflag;

void someInterruptCode () {
  myflag = false;
}

main {
  myflag = true;
  setupInterrupt();
  while (myflag) {
    ...
  }

  FlagChanged()
}

The system could get stuck forever in the while loop even if the interrupt was triggered. The compiler tries to be clever, it knows that myflag is set to true, if nothing in the while loop changes that then that condition is always true and so could potentially optimize out that check.

This can really throw people because one compile it may work and the next compile it won't with no clear reason why things changed.

The keyword volatile (as used in my code above) indicated that the variable could change without notice and so the compiler should always check the value. Any variables that are changed within an interrupt and then checked in the main code should be marked as volatile in order to ensure the compiler doesn't try to be too clever. stringOverSerialBuffer is OK since that's a pointer (in c pointers and arrays are basically the same thing). The compiler may assume that the address it points to is unchanged (which is true) but it won't assume that the data in that address is unchanged.

Accepted Answer

Hi Andy

i can not think of a more cleared explained answer than yours. Thanks a lot for answering all questions and for the code example (tested and works). Many thanks also for the extra info with the compiler optimization. Was something i did not know about.

posted by anteo c 05 Jan 2016
8 years, 4 months ago.

In addition to Andy's answer:

Q1: In principle you can get really far with printf style debugging, I have used it for alot of things. However using serial port to debug your serial port code is a bit a pain from my own experience ;).

In general if you need to receive significant strings over Serial while other stuff is happening I would advice you to use BufferedSerial lib, it reduces your potential problem with small hardware buffers alot by adding a large software buffer (https://developer.mbed.org/users/sam_grove/code/BufferedSerial/)

Hi Erik,

i see you everywhere in this forum ;). Thanks for the info. I will give it a try but do you mean i can use BufferedSerial only for debugging? i mean use both Serial and BufferedSerial in one program like:

Serial pc(SERIAL_TX, SERIAL_RX);   //for normal use
BufferedSerial buffered_pc(SERIAL_TX, SERIAL_RX);   //only for debugging "buffered_pc.printf(...)"

or replace Serial with BufferedSerial? Are there pros and cons to the option of replacing or is BufferedSerial always better?

posted by anteo c 05 Jan 2016

I am everywhere :D.

I meant completely replacing it with BufferedSerial. Downsides of BufferedSerial is the memory consumption (it uses software buffers in addition to the hardware buffers, these consume some memory), and that the complete part of writing the data out to the hardware is hidden from you. Of course that is what you wanted, you want it to be sent/received, and you don't care how. But if you do care how, and you really need to have a specific solution for your use case, then a generic library might not be the optimal solution. Of course later on you can always modify the library.

posted by Erik - 05 Jan 2016