Serial protocol pre-design discussion - give us your input!

[ababkin]

CRC is more than enough for non fixed data. i.e. all functions going to ecu should be a couple/3/4 bytes ie fixed data packet predefined. all daat going from ecu (tables,realtime,etc) should include CRC. no sense to make anything else. the main idea do not to make it slow with all check for integrity etc.

Serj

what do you mean by "make it slow with all check for integrity" ? I though checking CRC _is_ checking for integrity.

[Serj]

i mean any other additional checking. this is not necessary in that type of application.

[ababkin]

i mean any other additional checking. this is not necessary in that type of application.

you are probably right.

Something has to be said about how exactly one would perform CRC check, as there are more ways than one.
I would try to hide the CRC check latency "behind" the serial comm latency - example:
Consider two ways of sending and CRC verifying a 1k block of data (i'll simplify the example on purpose).

First way: (trivial)
1. send the whole 1k block (wait till it's all transfered),
2. do CRC (wait till done),
3. send ack back

Second way: (with slicing to hide latencies)
1. break down the 1k block into several blocks (how many, will have to be determined experimentally. I am guessing it's a balance thing),
2. send the first slice (wait till it transfers)
3. send ack that it received but has not verified yet (so the sender can send next slice)
4. start CRC check on the first slice
5. start transfering the second slice concurrently with the first slice's CRC check
....
x. once CRC on slice is performed, send ack. If CRC is faulty, send invalid ack, which will force the sender to schedule the slice for resend.

As one can see, data transfer and CRC check can be pipelined this way. One could also be creative and, for example, start performing CRC check on a data slice even before it is completely received (just start adding with no carry as bytes get received).

[ababkin]

This (linux specific) link is a good intro into serial comms:

http://tldp.org/HOWTO/Serial-HOWTO.html#toc3

[Fred]

I agree with what mtx_man has to say :

i.e. tuning software assembles a block and performs a CRC on that block and holds that CRC. Sends the data. the ecu receives, CRC checks the stream and compares it with the crc at the end and report either a a "good" (CRC passed) or "bad" (CRC mismatch) response.

The same would work for whe nthe ecu is told to send back a block or data (rtvars, general memory, etc) ecu would compute a checksum and tack it onto the end of the block, tuning SW would do the same verification of the block.

It would be really easy and really effective to do the rolling checksum while sending the data out like this.

I agree with what Serj says about keeping it fast. That is at the back of my mind too. Datalogging must be fast, but tunning stuff can be slowish, provided that it is slow in a defined way. Because tuning data is small, it should remain fast even with decent checking for burns etc.

Thanks for the link Alex, I'll have a read up through that as soon as I have finished attending my forum duties.

[FlappySocks]

I have been in the vehicle tracking business for 15 years or so, and in that time I have come across many protocols that I have had to implement, and design. Some are designed to use the minimum of data, some are designed to be fast, some are designed to be robust, some are very expandable, some are simple.

Speed is is going to be important here, and as bandwidth is minimal, to get speed, the data needs to be compact.
As already discussed, data integrity is also important, as is extendability.

I fear KISS is not an option!

If KISS is really needed .... use two protocols. The default protocol being a simple console based connection, where you use something like minicom or hyperterminal to issue commands to get results. One of those commands can then switch the comms into a new protocol mode. It's a little trick I use on one of my apps, and it works well.

Stuart.

[Fred]

Hmmmm, in order to get speed inside this cpu without impacting it's primary function of getting the fueling and ignition done too much, it seems to me that KISS is required. If by compact you are saying we should compress the data? I'm not sure how that would fit in with speed when the cpu will already be loaded significantly. It's good that you are switched on to serial protocols though, as I really haven't done anything with them at all.

You know, the more people sign up, the more I think my dream of a site where I am the LEAST capable individual starts to become reality! If that were the case, we would have quite a team indeed! :-)

I now see what you meant by domain name :-) Was it yours that was registered at the same time as mine? I did find yours afterwards somehow and had a look around it :-)

Admin.

[FlappySocks]

No, not compress the data - keep it compact. For example, if you are sending lots of integer values, some of them may only need a single byte, yet the specification may call for up to 4 bytes. What you could do, is implement a scheme where you have variable integer sizes. Its very simple.

Mind you, I think we are talking about different things - processor time vs time to get it down the serial port.
I guess what I am getting at, is keeping it too simple, adds it's own problems.

Yeah, we registered our domains at around the same time. Only .com was taken at the time I ordered mine, although I registered it 3 or 4 days after yours. Never even heard about your project until a few days ago. lol

[Fred]

Right, my idea for keeping it compact is the following :

The data in memory has some natural order which both the firmware and software know about.
Some chunks might be 64 bit, others 8 bit etc, but the order is known, so the offset of each is also known.
The tuning data includes a mask of variables to send for logging in binary, ie 100100101 means send the 1st, 4th, 7th, and 9th variables only regardless of size.
Both the firmware and software know about this mask so when data is sent or recieved regardless of total length or which ones are included/excluded it can be reconstituted into something useful.

On the front of it would come some unique byte or header that means "logging data" and on the end the checksum.

To me this seems a simple effective fast way of getting the data out of memory and down the serial line. Just grab all the data sequentially, drop those not required, send and checksum those that are.

For tuning tasks speed is not at all critical, so it should be much more robust.

Thoughts?

Fred.

[FlappySocks]

Sounds like a good idea. How about using 7-bits for the mask, and the 8th bit to say that the following byte is also a mask (another 7 bits + 1). In that way, you can have an infinite number of fields.

Stuart.