plc4x-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Łukasz Dywicki <l...@code-house.org>
Subject Re: Big and littleendian fields in one mspec
Date Tue, 05 May 2020 23:33:37 GMT
Last Saturday I promised to Chris I will confirm the case and I just did.

As suggested - I rearranged field order in State type and made a test to
proof myself, that it works:
https://github.com/apache/plc4x/commit/3010fdd15524410dfd55746562ed44bcee0fe80f

I've closed related issue (PLC4X-193) and created pull requests. Changes
in mspec are not needed for that case.

It took quite long, but we managed to find solution.

Cheers,
Łukasz


On 01.05.2020 10:53, Christofer Dutz wrote:
> Hi folks,
> 
> so Lukasz just sent me a little pcap file with some AMS packets in there and my assumption
was correct:
> 
> for example in the byte-stream I can see the "flags" being 0x0400 ... However wireshark
read this as unsigned short LE and therefore re-aranged the bytes:
>     StateFlags: 0x0004
>         .... .... .... ...0 = RESPONSE: Not set
>         .... .... .... ..0. = NO RETURN: Not set
>         .... .... .... .1.. = ADS COMMAND: Set
>         .... .... .... 0... = SYSTEM COMMAND: Not set
>         .... .... ...0 .... = HIGH PRIORITY COMMAND: Not set
>         .... .... ..0. .... = TIMESTAMP ADDED: Not set
>         .... .... .0.. .... = UDP COMMAND: Not set
>         .... .... 0... .... = INIT COMMAND: Not set
>         0... .... .... .... = BROADCAST: Not set
> 
> So in above example only one field is true: "ADS COMMAND" ... so if I decode 0x0400 to
binary that’s: 00000100 00000000
> 
> So this exactly fits my proposed mspec definition of: 
> 
>     [type 'State'
>             [simple     bit 'initCommand'           ]
>             [simple     bit 'updCommand'            ]
>             [simple     bit 'timestampAdded'        ]
>             [simple     bit 'highPriorityCommand'   ]
>             [simple     bit 'systemCommand'         ]
>             [simple     bit 'adsCommand'            ]
>             [simple     bit 'noReturn'              ]
>             [simple     bit 'response'              ]
>             [simple     bit 'broadcast'             ]
>             [reserved   int 7 '0x0'                 ]
>         ]
> 
> So I strongly object introducing any special endianness "feature" for this (in this case)
... 
> we might encounter something where we need it, but for this we don't.
> 
> Chris
> 
> 
> 
> Am 30.04.20, 23:02 schrieb "Christofer Dutz" <christofer.dutz@c-ware.de>:
> 
>     Hi Lukasz,
> 
>     I wasn't suggesting to look how WireShark decodes things. Cause I would assume that
they read a short in little endian and then (on the rearranged bytes) apply some bit checks.

> 
>     I was more suggesting that you look into the bytes Wireshark shows you and to check
their exact position.
> 
>     When implementing drivers, I used the Mac's Calculator tool quite often to see the
bits for a given byte value.
> 
>     In your example I can't see any need for endianness at all ... it's all just bits
and one empty block which forms a full byte together with the first bit.
>     As this block doesn't cross any byte boundaries I can't see any problems with this.
> 
>     Assuming I understand what you are implying with 
> 
>     [type little endian 'State'
>             [simple     bit 'broadcast'             ]
>             [reserved   int 7 '0x0'                 ]
>             [simple     bit 'initCommand'           ]
>             [simple     bit 'updCommand'            ]
>             [simple     bit 'timestampAdded'        ]
>             [simple     bit 'highPriorityCommand'   ]
>             [simple     bit 'systemCommand'         ]
>             [simple     bit 'adsCommand'            ]
>             [simple     bit 'noReturn'              ]
>             [simple     bit 'response'              ]
>         ]
> 
>     Then it should be equal to:
> 
>     [type 'State'
>             [simple     bit 'initCommand'           ]
>             [simple     bit 'updCommand'            ]
>             [simple     bit 'timestampAdded'        ]
>             [simple     bit 'highPriorityCommand'   ]
>             [simple     bit 'systemCommand'         ]
>             [simple     bit 'adsCommand'            ]
>             [simple     bit 'noReturn'              ]
>             [simple     bit 'response'              ]
>             [simple     bit 'broadcast'             ]
>             [reserved   int 7 '0x0'                 ]
>         ]
> 
>     Which also seems to be the exact order how the bits go over the wire ... I mean ...
it doesn't make any sense to put the 7 empty bits in the middle.
> 
>     Chris
> 
> 
>     Am 30.04.20, 22:42 schrieb "Łukasz Dywicki" <luke@code-house.org>:
> 
>         Hey Chris and others,
>         I let this topic to dust and freeze a bit, however I would like to lift
>         it up again. Since the 0.7 release is now in discussion and ADS is not
>         ported yet I believe it is reasonable to validate this problem.
> 
>         As you suggested I've checked wireshark sources to see how they parse
>         state fragment of AMS header. While I can not read C properly and was
>         unable to locate actual reading logic, I do see they care about endianness:
>         https://github.com/wireshark/wireshark/blob/5cf3fd03f1538b37704d83b6c38b8223f9036108/plugins/epan/ethercat/packet-ams.c#L429
> 
>         Their frame traversal goes in following order:
>         - response
>         - no return
>         - ads cmd
>         - system cmd
>         - high priority
>         - timestamp add
>         - udp
>         - init cmd
>         - broadcast
> 
>         When I've tried to model mspec with respect to little endian order, I
>         end up with following field order:
>         - init cmd
>         - udp
>         - timestamp add
>         - high priority
>         - system cmd
>         - ads cmd
>         - no return
>         - response
>         - [reserved]
>         - broadcast
> 
>         I haven't seen ADS protocol book, my main source of information is
>         infosys which is rather short in this area. I doubt if they would
>         describe fields in above order.
> 
>         My initial attempts were similar to what you said - to read the value
>         and then shift bytes. With your feedback I started to look for better
>         ways to solve it. With my last attempt I solved the problem by
>         re-adjusting fields during code generation, so they appear in a valid
>         byte order. Beside additional flag at the mspec there is a need for this
>         helper:
>         build-utils/protocol-base-mspec/src/main/java/org/apache/plc4x/plugins/codegenerator/language/mspec/model/definitions/DefaultComplexTypeDefinition.java
> 
>         In above commit the State declaration in the mspec looks like this:
>         [type little endian 'State'
>             [simple     bit 'broadcast'             ]
>             [reserved   int 7 '0x0'                 ]
>             [simple     bit 'initCommand'           ]
>             [simple     bit 'updCommand'            ]
>             [simple     bit 'timestampAdded'        ]
>             [simple     bit 'highPriorityCommand'   ]
>             [simple     bit 'systemCommand'         ]
>             [simple     bit 'adsCommand'            ]
>             [simple     bit 'noReturn'              ]
>             [simple     bit 'response'              ]
>         ]
> 
>         Maybe it is not in line with wireshark, but it is consistent and:
>         1) requires minor change in code generator
>         2) does not involve additional processing of data
>         3) mspec is still easy to write
>         4) type declaration is in line with wireshark output when reading from
>         the left and not from the top. ;-)
>         5) it can be adjusted to be 100% consistent with wireshark by changing
>         helper logic. ;-)
> 
>         Best regards,
>         Łukasz
> 
> 
>         On 14.04.2020 21:25, Christofer Dutz wrote:
>         > Hi Lukasz,
>         > 
>         > I see it differently.
>         > 
>         > If you say something is LE/BE then it should be consistent. 
>         > 
>         > I for my part like it if the mspec is consistent with what I see in WireShark.
Otherwise you would always have to do a sort of 2 step parsing:
>         > 
>         > "Read 4 bytes in BE and then on the rearranged byte sequence take the first
3 bits and call them X and the next bit after that Y and so on."
>         > 
>         > In this case you would also have to know how large the chunk is to read
in order to know how to decode it's content.
>         > Having a historically partitioned set of two 16 bit bit-fields would look
completely different to a 32 bit bit-field. 
>         > 
>         > I don't like adding this extra level, even if it's probably the way protocols
were implemented when lacking the ability to write individual bits.
>         > I guess the usual process was to read a BE short (2 bytes) and to the resulting
unsigned short value apply different logical or operations to get the information.
>         > In PLC4X we can simply read any number of bits and don't have to twist our
minds in which chunks I have to read things first. 
>         > 
>         > So I think the way you are proposing is probably the wider used version
due to legacy restrictions, but also the more complicated way.
>         > 
>         > So I would like to keep it simple, even if the developer will have to do
a little more brain-work. 
>         > 
>         > I mean all we are doing in this project is going the extra mile so others
don't have to ... at least the ones contributing are doing that.
>         > 
>         > What do the others say to this? (Ideally ones that have already used mspec
or ones that understand the problem)
>         > 
>         > Chris
>         > 
>         > 
>         > Am 14.04.20, 15:46 schrieb "Łukasz Dywicki" <luke@code-house.org>:
>         > 
>         >     Hey Chris,
>         >     Sorry for wrongly addressed mail, it was intended to go to mailing list.
>         >     
>         >     To the point - I think that what you propose with reordering fields
>         >     manually shift the load from the tool to the developer. It will lead
to
>         >     further implementation errors caused by a need to reorder fields while
>         >     creating spec. It will be inconsistent with rest of places where things
>         >     are in "natural order" (big endian) except one or two types which
>         >     collects bit flags which will have to be declared in different order.
>         >     What I did in my naive implementation with "little endian" flag on the
>         >     State type is re-arrangement of fields during generation time.
>         >     It does not require any byte shifting at the runtime and keeps natural
>         >     order of fields without the need for manual reordering of flags in mspec
>         >     declaration.
>         >     
>         >     I understand that keeping mspec simple is an important priority, however
>         >     doing it at the cost of developers who will use it to implement codecs
>         >     will not help us in long term.
>         >     I'm quite sure that sooner or later we will get a  protocol with mix
of
>         >     BE/LE fields.
>         >     
>         >     Cheers,
>         >     Łukasz
>         >     
>         >     
>         >     On 14.04.2020 14:57, Christofer Dutz wrote:
>         >     > Hi Lukasz (adding dev@ to the recipients again)
>         >     > 
>         >     >  
>         >     > 
>         >     > you are not quite correct. Enum types do declare a base type where
they
>         >     > are declared … not where they are used.
>         >     > 
>         >     >  
>         >     > 
>         >     > If you have bit-fields then why don’t you define them as sequence
of
>         >     > bits? I really can’t understand why you need to flip anything.
>         >     > 
>         >     > I think you’re trying to do something similar to Sebastian when
he
>         >     > wanted to introduce some bit-mask types.
>         >     > 
>         >     >  
>         >     > 
>         >     > I could imagine that in the spec it might be written … these
two bytes
>         >     > are a bit-mask and the bit 1 means X bit 10 means Y, …
>         >     > 
>         >     > So why not define the bit fields in the sequence they are sent
over the
>         >     > wire?
>         >     > 
>         >     >  
>         >     > 
>         >     > Assuming you have a bit field of 16 bits BE uint. If the spec now
says:
>         >     > 
>         >     >  
>         >     > 
>         >     >  
>         >     > 
>         >     > Bit:                        15|14|13|12|11|10|9 |8 |7 |6 |5  |4
|3  |2
>         >     >  |1  |0
>         >     > 
>         >     > Meaning:            A  |B  |C  |D  |E  | F  |G|H |I  | J | K |L
| M|N |O |P
>         >     > 
>         >     >  
>         >     > 
>         >     > Then the order they are sent is:
>         >     > 
>         >     > 7 |6 |5  |4 |3  |2  |1  |0 | 15|14|13|12|11|10|9 |8
>         >     > 
>         >     > So you have to declare the bit fields in the order:
>         >     > 
>         >     >                                I  | J | K |L | M|N |O |P  |A  |B
 |C  |D
>         >     >  |E  | F  |G|H
>         >     > 
>         >     >  
>         >     > 
>         >     > So I don’t quite understand what you’re trying to flip here.
>         >     > 
>         >     >  
>         >     > 
>         >     > Chris
>         >     > 
>         >     >  
>         >     > 
>         >     > *Von: *Łukasz Dywicki <luke@code-house.org>
>         >     > *Datum: *Dienstag, 14. April 2020 um 13:26
>         >     > *An: *Christofer Dutz <christofer.dutz@c-ware.de>
>         >     > *Betreff: *Re: Big and littleendian fields in one mspec
>         >     > 
>         >     >  
>         >     > 
>         >     > I've added the endianness switch to the types in my experiments.
These
>         >     > are not needed for enums as these are read as int and then mapped
to
>         >     > constants. I referred to these as an example - enum does have a
length
>         >     > indication and type behind it.
>         >     > 
>         >     >  
>         >     > 
>         >     > Again, I will bring the issue - StateIO currently fails to do anything
>         >     > useful due to BE/LE switch. Because this type uses single bits
which
>         >     > need to be flipped before reading or after writing otherwise they
end up
>         >     > in improper order. Out of 16 bits 9 are in use so we have a lot
of
>         >     > possible combinations (2^9), which seems too much to create an
enum.
>         >     > 
>         >     > Since you didn't like my initial proposal and modifications (as
they
>         >     > might be redundant to what is available in the buffer API), how
would
>         >     > you handle State serialization without affecting mspec and without
>         >     > further complication to code generation?
>         >     > 
>         >     >  
>         >     > 
>         >     > Best,
>         >     > 
>         >     > Łukasz
>         >     > 
>         >     >  
>         >     > 
>         >     > wt., 14 kwi 2020 o 12:47 Christofer Dutz <christofer.dutz@c-ware.de
>         >     > <mailto:christofer.dutz@c-ware.de>> napisał(a):
>         >     > 
>         >     >     Hi Lukasz,
>         >     > 
>         >     >     but we don't have a:
>         >     >     [enum uint 16 little endian 'CommandId']
>         >     >     But only a:
>         >     >     [enum uint 16 'CommandId']
>         >     > 
>         >     >     And in your case I think perhaps the constants are not correct.
So
>         >     >     having a 16 bit uint will result in a 4 digit hex string:
>         >     >     So if you are having problems in mapping them to your constants,
>         >     >     perhaps the constants are in the wrong endianness.
>         >     > 
>         >     >     I have encountered (but can't recall where) that the Enum constants
>         >     >     in a BE protocol were written down in LE notation.
>         >     >     Of course the thing can't work then.
>         >     > 
>         >     >     So if for example you have the constant "1" in a BE protocol
with a
>         >     >     uint 16 enum type, your constant is not 0x0001, but 0x0100
instead.
>         >     > 
>         >     >     Chris
>         >     > 
>         >     > 
>         >     > 
>         >     > 
>         >     >     Am 14.04.20, 12:37 schrieb "Łukasz Dywicki" <luke@code-house.org
>         >     >     <mailto:luke@code-house.org>>:
>         >     > 
>         >     >         The legendary "two bytes" (shall we create a band with
this
>         >     >     name?!) are
>         >     >         coming from unfortunate State type. I don't mind these
to be an
>         >     >         int/sint/uint or whathever - this is matter of interpretation.
>         >     > 
>         >     >         If you could please check again my earlier messages you
will find
>         >     >         struggle I have which is - how much given type takes.
>         >     > 
>         >     >         We have that for enums
>         >     >         [enum uint 16 little endian 'CommandId']
>         >     > 
>         >     >         but we don't have that for complex types
>         >     >         [type 'State']
>         >     > 
>         >     >         This leads to situation that we don't know how to read
and interpret
>         >     >         incoming byte sequence or how to properly write it back
to stream.
>         >     > 
>         >     >         It would be great if we could limit the issue just to
>         >     >         // read
>         >     >         state = StateIO.parseStatic(io.readUnsignedInt(16, true));
//
>         >     >     true->LE
>         >     >         // write
>         >     >         wio = new WriteBuffer(2);
>         >     >         StateIO.staticSerialize(wio, new State(...));
>         >     >         io.writeUnsignedInt(8, wio.read, true); // true->LE
>         >     > 
>         >     >         However to do that first we need to know that State is
a uint 16
>         >     >     LE and
>         >     >         then do a lot of mambo-jambo in code generators to cope
with
>         >     >     that. :-)
>         >     > 
>         >     >         Best,
>         >     >         Łukasz
>         >     > 
>         >     > 
>         >     >         On 14.04.2020 12:20, Christofer Dutz wrote:
>         >     >         > Oh gee ... I totally remember having exactly the same
>         >     >     discussion with Sebastian about "bytes" ...
>         >     >         >
>         >     >         > The problem is there is generally no "byte" "int"
and "uint"
>         >     >     because a byte = 8 bits is either signed or unsigned. That
should
>         >     >     the 3rd option be? Perhaps-signed?
>         >     >         > So generally I use "uint 8" for what you refer to
as a "byte 8".
>         >     >         >
>         >     >         > Wo what would be the difference between your "2 byte
little
>         >     >     endian" and an "uint 16" with a LE ReadBuffer?
>         >     >         >
>         >     >         > I think what you have to rid yourself of thinking
of int as
>         >     >     number and a byte not being a number.
>         >     >         >
>         >     >         > Chris
>         >     >         >
>         >     >         >
>         >     >         >
>         >     >         > Am 14.04.20, 10:52 schrieb "Łukasz Dywicki"
>         >     >     <luke@code-house.org <mailto:luke@code-house.org>>:
>         >     >         >
>         >     >         >     Hey Christian,
>         >     >         >     The problem I face is quite simple. State type
in mspec i
>         >     >     declared as a
>         >     >         >     bunch of bits. Type length is fixed, however not
available
>         >     >     anywhere in
>         >     >         >     mspec or code generator to re-arrange bytes upfront.
All
>         >     >     we have exposed
>         >     >         >     at reader/write level is read/write bit.
>         >     >         >     To be fair LE/BE support in read/write buffers
is limited
>         >     >     just to
>         >     >         >     numbers. There is no such support for raw bytes
or bits,
>         >     >     cause for that
>         >     >         >     you need to declare a length of LE/BE sequence.
>         >     >         >     
>         >     >         >     I would love if I could just declare State as
'2 byte
>         >     >     little endian' so
>         >     >         >     it would be read properly upfront and parsed with
no
>         >     >     changes in
>         >     >         >     generated code, however I'm not sure how to do
it and
>         >     >     where. That's why
>         >     >         >     I'm playing with different things described in
earlier mail.
>         >     >         >     Since all type handling is general I am just afraid
of
>         >     >     more complicated
>         >     >         >     scenarios where we have variable length structures
such as
>         >     >     arrays.
>         >     >         >     
>         >     >         >     Best regards,
>         >     >         >     Łukasz
>         >     >         >     
>         >     >         >     
>         >     >         >     On 14.04.2020 09:41, Christofer Dutz wrote:
>         >     >         >     > Hi Lukasz,
>         >     >         >     >
>         >     >         >     > I am not sure I am understanding the problems
you are
>         >     >     facing. We already have LE and BE protocols.
>         >     >         >     > For example EIP is LE and the rest is generally
BE. It
>         >     >     seems that ADS/AMS is also LE.
>         >     >         >     > mspec doesn't even know about endianness.
>         >     >         >     >
>         >     >         >     > Up till now the endianness doesn't have an
effect on
>         >     >     bit-fields or single-bit ints.
>         >     >         >     > It only starts to affect if a field goes
from one byte
>         >     >     to the next, which is usually for (u)int and floating point
values.
>         >     >         >     >
>         >     >         >     > That's why we have created the Read/WriteBuffers
to set
>         >     >     their endianness in the constructor.
>         >     >         >     >
>         >     >         >     > So if you're creating a driver for ADS/AMS
which is LE,
>         >     >     then you write the mspec according to the sequence the information
>         >     >     is located in the transferred bytes and have the Read/WriteBuffer
>         >     >     handle the endianness issue.
>         >     >         >     >
>         >     >         >     > I do see a problem when there are drivers
that use mixed
>         >     >     endianness, but we have still to encounter such a protocol.
>         >     >         >     >
>         >     >         >     > So I have to admit that I don't like any
of the mspec
>         >     >     changes you proposed, as I think you are just not using the
tools we
>         >     >     have the right way.
>         >     >         >     >
>         >     >         >     > Chris
>         >     >         >     >
>         >     >         >     >
>         >     >         >     >
>         >     >         >     > Am 14.04.20, 00:32 schrieb "Łukasz Dywicki"
>         >     >     <luke@code-house.org <mailto:luke@code-house.org>>:
>         >     >         >     >
>         >     >         >     >     Hey Niclas,
>         >     >         >     >     I realized how old the old things are
when I started
>         >     >     preparing
>         >     >         >     >     automation training for mere mortals
and got into
>         >     >     history of frames and
>         >     >         >     >     even cabling. Mr. Modbus and EIA-485
is definitely
>         >     >     older than I. ;-)
>         >     >         >     >     
>         >     >         >     >     Getting back to the point - yes. I been
thinking how
>         >     >     to address the byte
>         >     >         >     >     order in effective way. Here are two
approaches I
>         >     >     have for now:
>         >     >         >     >     
>         >     >         >     >     A) My initial attempt is just a temporary
buffer
>         >     >     which is then written
>         >     >         >     >     in reverse order to caller. For reading
it is
>         >     >     similar - just getting N
>         >     >         >     >     bytes in reversed order. The hard part
is.. knowing
>         >     >     N. I had to add a
>         >     >         >     >     static calculation in order to allocate
valid buffer
>         >     >     sizes. I tend to
>         >     >         >     >     work but I'm not happy with this approach
cause it
>         >     >     involves additional work.
>         >     >         >     >     B) Second idea I've got is really simple
and relies
>         >     >     on code generation.
>         >     >         >     >     We know in which order fields are coming.
Here I'm
>         >     >     referring to a State
>         >     >         >     >     field which is just bunch of bits. If
we would group
>         >     >     fields in bytes and
>         >     >         >     >     generate code in reverse order then it
has chance to
>         >     >     work. Requirement
>         >     >         >     >     for that - ability to know basic field
sizes upfront.
>         >     >         >     >     C) Try to combine above with bit-io or
>         >     >     Read/WriteBuffers as these are
>         >     >         >     >     places which know actual position and
state of
>         >     >     buffers which are being
>         >     >         >     >     read/written.
>         >     >         >     >     
>         >     >         >     >     Now, getting to two cases which are a
problem.
>         >     >     CommandId and State. So
>         >     >         >     >     with command id situation is simple as
it is
>         >     >     declared as enum and it is
>         >     >         >     >     read as uint. We know size upfront and
can generate
>         >     >     valid method call
>         >     >         >     >     (readIntLE).
>         >     >         >     >     [enum uint 16 little endian 'CommandId'
>         >     >         >     >         ['0x00' INVALID]
>         >     >         >     >         ['0x01' ADS_READ_DEVICE_INFO]
>         >     >         >     >         ['0x02' ADS_READ]
>         >     >         >     >         ['0x03' ADS_WRITE]
>         >     >         >     >         ['0x04' ADS_READ_STATE]
>         >     >         >     >         ['0x05' ADS_WRITE_CONTROL]
>         >     >         >     >         ['0x06' ADS_ADD_DEVICE_NOTIFICATION]
>         >     >         >     >         ['0x07' ADS_DELETE_DEVICE_NOTIFICATION]
>         >     >         >     >         ['0x08' ADS_DEVICE_NOTIFICATION]
>         >     >         >     >         ['0x09' ADS_READ_WRITE]
>         >     >         >     >     ]
>         >     >         >     >     
>         >     >         >     >     Second candidate is what I'm stuck right
now sniping
>         >     >     next cycles of
>         >     >         >     >     problems. So in case of State we have
complex type
>         >     >     composed from 2
>         >     >         >     >     bytes. A note here - instead of two bytes
we might
>         >     >     have a variable
>         >     >         >     >     length type which includes array or other
variable
>         >     >     section.
>         >     >         >     >     [type little endian 'State'
>         >     >         >     >         [simple     bit 'broadcast'     
       ]
>         >     >         >     >         [reserved   int 7 '0x0'         
       ]
>         >     >         >     >         [simple     bit 'initCommand'   
       ]
>         >     >         >     >         [simple     bit 'updCommand'    
       ]
>         >     >         >     >         [simple     bit 'timestampAdded'
       ]
>         >     >         >     >         [simple     bit 'highPriorityCommand'
  ]
>         >     >         >     >         [simple     bit 'systemCommand' 
       ]
>         >     >         >     >         [simple     bit 'adsCommand'    
       ]
>         >     >         >     >         [simple     bit 'noReturn'      
       ]
>         >     >         >     >         [simple     bit 'response'      
       ]
>         >     >         >     >     ]
>         >     >         >     >     
>         >     >         >     >     The order of reading big endian encoded
data to
>         >     >     impose little endian
>         >     >         >     >     shift would be (please correct me if
I'm wrong):
>         >     >         >     >     1) init
>         >     >         >     >     2) udp
>         >     >         >     >     3) add timestamp
>         >     >         >     >     4) priority
>         >     >         >     >     5) system
>         >     >         >     >     6) ads
>         >     >         >     >     7) noreturn
>         >     >         >     >     8) response (end of byte 1)
>         >     >         >     >     9) broadcast
>         >     >         >     >     10) reserved (end of byte )
>         >     >         >     >     We can do same trick for writing, by
re-arranging
>         >     >     fields. By this way we
>         >     >         >     >     avoid any additional byte level operations.
>         >     >         >     >     
>         >     >         >     >     Overall trouble with generated driver
is to declare
>         >     >     "how much" bytes
>         >     >         >     >     should be read and interpreted. We have
precise size
>         >     >     information at the
>         >     >         >     >     runtime - due to length fields, we can
leverage it
>         >     >     at generation time,
>         >     >         >     >     but then we won't be able to cover all
cases.
>         >     >         >     >     
>         >     >         >     >     I would love to keep it simple and do
not break
>         >     >     things thus I need your
>         >     >         >     >     advice on how to approach this problem
in a valid way.
>         >     >         >     >     
>         >     >         >     >     Cheers,
>         >     >         >     >     Łukasz
>         >     >         >     >     
>         >     >         >     >     
>         >     >         >     >     On 13.04.2020 03:26, Niclas Hedhman wrote:
>         >     >         >     >     > <anecdotal-rant>
>         >     >         >     >     > For us who were around and shaping
the protocols
>         >     >     in the 1980s, and people
>         >     >         >     >     > before us (and before standards
like RS-232), a
>         >     >     lot of the "specifications"
>         >     >         >     >     > came out of "observation of implementation
we
>         >     >     managed to get to work",
>         >     >         >     >     > rather than "implement this spec".
A lot was due
>         >     >     to extreme memory
>         >     >         >     >     > constraints (in my case, multi-tasking
operating
>         >     >     system, serial protocol
>         >     >         >     >     > 187kbps, interpreted programming
language with
>         >     >     floating point ops and user
>         >     >         >     >     > applications in 2kB RAM and 8kB
EPROM) and a
>         >     >     general lack of information,
>         >     >         >     >     > like what other people were doing,
sharing
>         >     >     experiences and so on.
>         >     >         >     >     >
>         >     >         >     >     > And there were many "innovative"
ways to squeeze
>         >     >     just a little bit extra
>         >     >         >     >     > out of the hardware, resulting in
"hard to
>         >     >     understand" consequences. Bit
>         >     >         >     >     > packing was a typical one, multiple
functions
>         >     >     packed into a single byte.
>         >     >         >     >     > Look at page 14 in
>         >     >     https://www.nxp.com/docs/en/data-sheet/80C31_80C32.pdf
>         >     >         >     >     > and read up on "UART Enahanced Mode",
and we used
>         >     >     this, i.e. 9 bits, no
>         >     >         >     >     > parity and clever use of address
and mask to
>         >     >     create a slave-to-slave direct
>         >     >         >     >     > protocol, where the master's role
was to signal
>         >     >     which slave "owned" the
>         >     >         >     >     > cable. Yeah, in that 8kB ROM limitation
(I think
>         >     >     protocol was about 1kB
>         >     >         >     >     > ROM) and something like 150 bytes
RAM for comm
>         >     >     protocol.
>         >     >         >     >     >
>         >     >         >     >     > Could you implement a compatible
device to this
>         >     >     with PLC4X and modern
>         >     >         >     >     > hardware (i.e. no 8031/32 co-processor)?
Possibly
>         >     >     but bit-banging is needed
>         >     >         >     >     > to support the 9bit data (+start
and stop bits)
>         >     >     and an awful lot of CPU
>         >     >         >     >     > cycles on something that was automatic
on one of
>         >     >     the slowest long-lived
>         >     >         >     >     > microcontroller ever.
>         >     >         >     >     > </anecdotal-rant>
>         >     >         >     >     >
>         >     >         >     >     > My point was only to highlight that
some of the
>         >     >     strange things you see in
>         >     >         >     >     > protocols today, have its roots
in
>         >     >     pre-standardization days. Today no one
>         >     >         >     >     > would go down that route, because
the hardware
>         >     >     cost nothing now (8031  +
>         >     >         >     >     > 8kB EPROM + 2kB static RAM + battery
backup =>
>         >     >     ~$50 in 1983's currency) and
>         >     >         >     >     > longevity of software is more important.
>         >     >         >     >     >
>         >     >         >     >     > Cheers
>         >     >         >     >     > Niclas
>         >     >         >     >     >
>         >     >         >     >     >
>         >     >         >     >     > On Sun, Apr 12, 2020 at 10:10 PM
Christofer Dutz
>         >     >     <christofer.dutz@c-ware.de <mailto:christofer.dutz@c-ware.de>>
>         >     >         >     >     > wrote:
>         >     >         >     >     >
>         >     >         >     >     >> Hi Lukasz,
>         >     >         >     >     >>
>         >     >         >     >     >> I think it really gets tricky
when using BE and
>         >     >     having some byte-odd-sizes
>         >     >         >     >     >> ... I remember in the Firmata
protocol there were
>         >     >     some bitmasks and then 10
>         >     >         >     >     >> bit uint as BE ... not it really
got tricky as
>         >     >     the specs were written from
>         >     >         >     >     >> a point of view: You read 16
bits BE and then the
>         >     >     first6 bits mean XYZ
>         >     >         >     >     >> instead of describing how the
bits actually
>         >     >     travel over the wire.
>         >     >         >     >     >>
>         >     >         >     >     >> Chris
>         >     >         >     >     >>
>         >     >         >     >     >>
>         >     >         >     >     >>
>         >     >         >     >     >> Am 11.04.20, 01:21 schrieb "Łukasz
Dywicki"
>         >     >     <luke@code-house.org <mailto:luke@code-house.org>>:
>         >     >         >     >     >>
>         >     >         >     >     >>     I've made some progress
with topic by
>         >     >     modyfing mspec and allowing
>         >     >         >     >     >>     'little endian' flag on
fields. This moved me
>         >     >     further to next issue -
>         >     >         >     >     >>     which is whole type encoded
little endian.
>         >     >         >     >     >>
>         >     >         >     >     >>     In ADS driver such type
is State, which has 2
>         >     >     bytes and uses 8 bits for
>         >     >         >     >     >>     various flags.
>         >     >         >     >     >>     There are two cases which
require different
>         >     >     approach - reading and
>         >     >         >     >     >>     writing. So for reading
we need to swap N
>         >     >     bytes based on type length.
>         >     >         >     >     >>     For writing we need to alocate
buffer for N
>         >     >     bytes and swap them before
>         >     >         >     >     >>     writing.
>         >     >         >     >     >>
>         >     >         >     >     >>     I am stuck now with freemaker
templates and
>         >     >     bit-io.
>         >     >         >     >     >>
>         >     >         >     >     >>     Cheers,
>         >     >         >     >     >>     Łukasz
>         >     >         >     >     >>
>         >     >         >     >     >>
>         >     >         >     >     >>
>         >     >         >     >     >>     On 10.04.2020 17:57, Łukasz
Dywicki wrote:
>         >     >         >     >     >>     > I am doing some tests
of ADS serialization.
>         >     >         >     >     >>     >
>         >     >         >     >     >>     > I've run into some
troubles with payload
>         >     >     which is generated with new
>         >     >         >     >     >>     > driver. I'm not sure
if that's my fault or
>         >     >     generated code.
>         >     >         >     >     >>     >
>         >     >         >     >     >>     > I did a verification
of what Wireshark
>         >     >     shows and how ads structures
>         >     >         >     >     >> are
>         >     >         >     >     >>     > parsed. There is a
gap I think. For example
>         >     >     ams port number 1000
>         >     >         >     >     >>     > (0x1027) is read as
4135.
>         >     >         >     >     >>     >
>         >     >         >     >     >>     > Obviously I used wrong
structures while
>         >     >     implementing protocol logic
>         >     >         >     >     >> in
>         >     >         >     >     >>     > first place, but now
I am uncertain of how
>         >     >     fields are encoded. How we
>         >     >         >     >     >>     > mark field as little
endian when rest of
>         >     >     payload is big endian? Do we
>         >     >         >     >     >>     > have `uint_le`?
>         >     >         >     >     >>     >
>         >     >         >     >     >>     > As far I remember route
creation logic I
>         >     >     was tracking last week used
>         >     >         >     >     >>     > combination of LE and
BE.
>         >     >         >     >     >>     >
>         >     >         >     >     >>     > Best regards,
>         >     >         >     >     >>     > Łukasz
>         >     >         >     >     >>     >
>         >     >         >     >     >>
>         >     >         >     >     >>
>         >     >         >     >     >>
>         >     >         >     >     >
>         >     >         >     >     
>         >     >         >     >
>         >     >         >     
>         >     >         >
>         >     > 
>         >     
>         > 
> 
> 

Mime
View raw message