stream I/O

[supercamel]

Hello,

The stream class should have functions that simplify and assist with receiving ASCII text and binary data from a serial input. I propose these functions should add the ability to

• check number of bytes available
• read all bytes into a list/array/container object
• read bytes until a specific value is reached
• extract bytes between two specified values. often binary packets have a start byte and an end byte
• get binary int (16, 32 or 64 bit, little or big endian). for an int16, this would read two bytes and convert it into an int16.
• getline
• get string until a byte is received (like getline, only it doesn't stop at '\n', it stops at whatever char is specified).
• parse next integer. ignore all bytes until an integer is received, then return that value
• parse next float

At the moment, the stream class requires a 'put' function to write a byte. To receive data, it would require an 'available' function that returns the number of bytes available to process, and a 'get' function that reads a single byte.

Feedback & discussion around how to best implement this is very welcome. To actually contribute code, please do a pull request.

Cheers!

[dmamalis]

If you want to keep this minimal I think the stream class should only have functions to

• check number of bytes available
• read all bytes until a variable value into some data structure

The rest is post processing functionality which is always added to generic libraries such as Arduino. To me and my C background functions such as

• getInt, getFloat, getYouNameIt
• parse
• extract etc
  are all utilities. Each time I have to work in C++ -and they are still rare- I tend to keep utilities out of the class. Call it a personal preference but in an IO class I want IO stuff. Parsing is something you do after you have the input available and thus it is not IO.

What I like in this toolikit is that it is quite clean. Take for example the link class. I have seen implementations where the link class provides all the functionality for any type of class you can imagine. In your implementation there is list.h, linked_list.h, forward_list.h, dynami_list.h. This makes it simple and clean. 👍

What do you think of that?

[supercamel]

Thanks for the feedback. It's been said before that etk is pretty muddled - lots of totally different things thrown together in one project. So I certainly appreciate the need to keep it clean although I'm not sure how best to achieve that. It might be sub-directories. So vector.h moves to maths/vector.h. I don't know. But anyway . .

The stream class is all about dealing with I/O streams. Typically this would be UARTs but may be screens, keyboards, even IP/TCP/UDP. It doesn't do any I/O itself. It just provides a reusable implementation of common serialising and parsing functions that an I/O class could inherit.

So, if you are writing a hardware abstraction layer for a UART, all you would need to do is create a class with the put, get and available functions. Inheriting the stream class will then give you some basic serialising & parsing functions.

From that point of view I think stream is the right place for some basic parsing functions. But, it also shouldn't re-implement what rope, static string or the tokeniser do.

[dmamalis]

A toolkit is a kit of lots of different tools. What I like about it is that it is a place where I can find a quaternion class, a state machine class and a pool class. I think that is exactly its purpose. What I don't like in toolkits is dependencies. I don't want to pull quaternion functionality in a state machine. To wrap it up I think that the "lots of different things thrown together" is perfect.

This issue however is exactly where things get messy and I would love to help clean this! :-) There is a missing functionality (input) which is essential for communicating with humans and devices.

Your point is clear and I had exactly the same problem when trying to add my getLine(): should it be placed in String, Rope, or Tokeniser? (I just added it to stream to get my job done but this needs indeed a bit of thought).

Given your above comment I think Tokeniser should be moved to the stream class since it is actually a parsing function. What is not clear to me is the need for both Rope and StaticString. I guess there was a need to have two different and dependent classes but I just don't see it. If it is not out of the scope of this issue could you elaborate on that?

[supercamel]

Hmm, the idea about moving the Tokeniser into the Stream class is very interesting. What about a stringstream implementation similar to the STL?

The difference between Rope and StaticString is that Rope is about performing operations on an existing c string / buffer where as StaticString is a string object that contains its own memory. Rope is about doing stuff to a c string, whereas StaticString is about being an alternative/replacement for c strings - if that makes sense.

If you are given a buffer or C string, and you needed to perform some operation (append text, parse a number, etc), it's faster to use a Rope because it doesn't needlessly copy the data and it's keeps a track of the 'cursor position' so subsequent calls to 'append' don't need to recalculate the length of the string.

There are considerations around memory ownership. For instance, if you create a copy of a Rope, the new Rope will still have a pointer to the original buffer. And if that buffer has gone out of scope you're in trouble. If you copy a StaticString, it will create new memory and deep copy the contents.