plc4x-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Misztur <cmisz...@mriiot.com>
Subject Re: Source Time of PLC Value?
Date Wed, 10 Mar 2021 15:56:58 GMT
I would like to comment on the simplicity of MQTT.  It does indeed simplify
creating the pipe between endpoints, but the problem of data opacity
remains.  I believe that premature implementation of MQTT everywhere and to
everything will do more harm than good, because we will naturally "invent"
our own topic structures and payloads.  Maybe the correct approach is to
continue pushing proprietary protocols over MQTT for distribution
purposes?  I also think that departure from XML to JSON has been premature;
the richness of expression is lost with JSON.

Just my thoughts for the day :)

*Chris*


On Wed, Mar 10, 2021 at 9:22 AM Christofer Dutz <christofer.dutz@c-ware.de>
wrote:

> Hi Andreas,
>
> Glad I didn't upset you too much ;-)
>
> And I guess I'm also a bit pissed about OPC-UA in general. At first, I had
> such a hard time selling PLC4X as OPC-UA was promising so much, but without
> proof, that almost nobody gave open-source and PLC4X a chance. So much
> money was burnt in 2018 and 2019 with unsuccessful OPC-UA projects by the
> big automation companies.
>
> Then end of 2019 some of the ones, that decided to go the OPC-UA path,
> started contacting me again. But when we wanted to discuss terms, they
> sometimes had the nerve to say something like: We wasted so much money on
> OPC-UA and our budget is almost depleted ... we can only hire your company
> if you greatly reduce the hourly rate. Guess what my answer was? ;-)
>
> Chris
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: Andreas Vogler <andreas.vogler@me.com.INVALID>
> Gesendet: Mittwoch, 10. März 2021 16:06
> An: dev@plc4x.apache.org
> Betreff: Re: Source Time of PLC Value?
>
> No problem, I think I understand what you mean… and I agree on the device
> level OPC UA is maybe too much / to fat (or not well implemented :-))
>
>
>
> > On 10.03.2021, at 15:50, Christofer Dutz <christofer.dutz@c-ware.de>
> wrote:
> >
> > Hi Andreas,
> >
> > I didn't want to upset you. I should have been a bit more clear on this:
> >
> > In 2018, when I was travelling through the country proclaiming PLC4X,
> all I heard was: But OPC-UA can do all of this (Without having tested it)
> ... 2019 most of the people who tried it were already looking for
> alternatives. End of 2019 and beginning of 2020 was definitely the age of
> MQTT (Of course OPC UA now provided a MQTT-like communication form).
> >
> > I think OPC-UA is great for machine-to-machine communication and stuff
> like dashboards. But as soon as you want to do high volume data
> acquisition. Especially for use cases like machine-learning etc. the
> processing power of even modern PLCs can't handle the load.
> >
> > For example, at one of the biggest German car manufacturers, they were
> not able to get more than 200 datapoints every 2 seconds from a new S7 400
> PLC via OPC-UA. These are probably the most powerful ones Siemens has to
> offer. If they requested 300, the PLC shut down the IO process due to
> overload. With PLC4X using the S7 protocol, we were able to get 2600
> datapoints in 200ms from the same machines.
> >
> > On the client side on one of the standard VMs they were using, running
> the OPC-UA stack was able to collect data on 10 PLCs with 200 datapoints.
> Then the CPU power was 100% ... in our case we run PLC4X on the same VMs
> using S7 and there we could collect 200 PLCs with 2600 datapoints. So also
> on the client side OPC-UA is a very fat project, that requires a lot of CPU
> time.
> >
> > I hope this explains things a bit more. If you are just reading a few
> datapoints or using OPC-UA for Server-to-Server communication the servers
> will be able to cope with more. And if it doesn't you just add more nodes
> to the compute cluster.
> >
> > Chris
> >
> >
> >
> >
> >
> > -----Ursprüngliche Nachricht-----
> > Von: Andreas Vogler <andreas.vogler@me.com.INVALID>
> > Gesendet: Mittwoch, 10. März 2021 15:38
> > An: dev@plc4x.apache.org
> > Betreff: Re: Source Time of PLC Value?
> > Priorität: Niedrig
> >
> > Just a side note: it hurts to hear that OPC UA sucks badly
> > performance-wise. I have done tests with Milo OPC UA Client and SCADA
> > OPC UA servers, and they perform pretty fine. 200 MQTT clients writing
> > values via a Gateway with OPC UA Milo (Client) to two OPC UA servers,
> > each OPC UA server handles 50000 values changes per second:
> > https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-v
> > ertx-activity-6769029730797719552-ZkjG
> > <https://www.linkedin.com/posts/andreas-vogler-361115122_graphql-mqtt-
> > vertx-activity-6769029730797719552-ZkjG>
> >
> > But I fully agree with MQTT - it’s simple - and I think that’s the
> > reason why it is so popular…
> >
> >
> >> On 10.03.2021, at 14:34, Christofer Dutz <christofer.dutz@c-ware.de>
> wrote:
> >>
> >> Yes,
> >>
> >> OPC-UA has all of this ... and it's probably one of the reasons so many
> implementations differ, and you have to again interpret results. It's also
> probably a reason why OPC-UA sucks that badly performance-wise.
> >>
> >> Just saying that we should be careful what we choose as reference. In
> contrast typical MQTT doesn't have all of this and it seems to be gaining
> more and more traction, just because it doesn't have all of this overhead.
> >>
> >> Just my thoughts on this.
> >>
> >> Chris
> >>
> >>
> >>
> >>
> >> -----Ursprüngliche Nachricht-----
> >> Von: Łukasz Dywicki <luke@code-house.org>
> >> Gesendet: Mittwoch, 10. März 2021 12:58
> >> An: dev@plc4x.apache.org
> >> Betreff: Re: AW: Source Time of PLC Value?
> >>
> >> Matthias,
> >> I do believe there are several points which are relevant and still not
> possible to be implemented using available PLC4X API. Metadata might not be
> most fortunate way, but it can enable or ease analytical and
> troubleshooting scenarios.
> >>
> >> Two major points I see:
> >> - Verification of device condition - ie. how does caller know response
> latency? This is especially relevant with frequent polling. Currently
> caller has no measures to know that and can not measure that in reliable
> way due to async nature of API.
> >> There is a way to implement backpressure technics with callback chains
> (we can automatically delay execute call), but first we need to bring
> measures for that.
> >> - Caching of values - when we do cyclic subscriptions or emulation of
> these through passive connections. There is no way to make caller code
> aware of nature of data received. Unless he code everything on his end.
> >> - Per operation timeout settings/automatic retry are another
> functionalities which are relevant in case of querying and writing. We do
> not want every driver to implement it, so making a metadata and/or
> decorator would unify this layer between drivers.
> >>
> >> I speak of above from my own perspective cause I did implement an
> experimental decorator API for PLC4X [1] to cope with CANopen arbitration
> issues I found in devices we interface with. It allows to catch read/write
> failures [2] and re-attempt entire operation. It is proven to work [3]
> independently of driver. Sadly it has no measures to signal its execution
> back to caller. In the end, traceability is pretty tough.
> >>
> >> Obviously OPC UA has everything sorted out, but device specific
> protocols often do not. Even a limited set of metadata can give us a way to
> start bridging the gap and making Ben's Milo/OPC UA server much closer to a
> real thing.
> >>
> >> Best,
> >> Łukasz
> >>
> >> [1]
> >> https://github.com/ConnectorIO/connectorio-addons/tree/master/bundles
> >> /
> >> org.connectorio.plc4x.decorator/src/main/java/org/connectorio/plc4x/d
> >> e
> >> corator
> >> [2]
> >> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles
> >> /
> >> org.connectorio.plc4x.decorator.retry/src/main/java/org/connectorio/p
> >> l
> >> c4x/decorator/retry/ReadRetry.java
> >> [3]
> >> https://github.com/ConnectorIO/connectorio-addons/blob/master/bundles
> >> /
> >> org.connectorio.plc4x.decorator.retry/src/test/java/org/connectorio/p
> >> l c4x/decorator/retry/RetryDecoratorReadTest.java
> >>
> >>
> >> On 08.03.2021 23:15, Matthias Milan Strljic wrote:
> >>> Hi all,
> >>>
> >>> i see there a point of how useful this meta information could be in
> >>> some scenarios. But i would rather go the route of chris. Because I
> >>> see there is more of a risk of an overcomplicated inhomogeneous api
> >>> structure. I would not say that all API should have the same java
> >>> structure on each platform but if you have for all the different
> >>> platforms other featuresets you could also just use partially the
> >>> code generation attempt to allow a separate meta info layer.
> >>>
> >>> And there would be the question if that is worth the additional
> feature?
> >>> I mean I know we could add especially on the OPC UA side some meta
> >>> information but is there more than the  actual source timestamp? And
> >>> if there is none, would it be not better to use it as the actual
> >>> timestamp of the value?
> >>>
> >>>
> >>> Einen schönen Abend ;)
> >>> Greetings Matthias
> >>>
> >>>
> >>>
> >>> Am Mo., 8. März 2021 um 14:29 Uhr schrieb Christofer Dutz <
> >>> christofer.dutz@c-ware.de>:
> >>>
> >>>> Hi,
> >>>>
> >>>> I'm generally more concerned about users expecting us to deliver
> >>>> feature that driver X has for driver Y too.
> >>>>
> >>>> I won't object, if you think it's worth doing.
> >>>>
> >>>> Perhaps if you could whip up an example in a feature branch? I
> >>>> think perhaps I was still not understanding what you propose.
> >>>>
> >>>> Would that be ok?
> >>>>
> >>>>
> >>>> Chris
> >>>>
> >>>>
> >>>> -----Ursprüngliche Nachricht-----
> >>>> Von: Lukas Ott <ott.lukas.14@gmail.com>
> >>>> Gesendet: Montag, 8. März 2021 14:20
> >>>> An: dev@plc4x.apache.org
> >>>> Betreff: Re: AW: Source Time of PLC Value?
> >>>>
> >>>> +1 to Lukasz
> >>>>
> >>>> Am Mo., 8. März 2021 um 14:09 Uhr schrieb Łukasz Dywicki <
> >>>> luke@code-house.org>:
> >>>>
> >>>>> Hold on for a second.
> >>>>>
> >>>>> Making options pushed over connection string is a long term recipe
> >>>>> for disaster. Apache Camel is a prime example of what could happen
> >>>>> if you starting with configuration with URIs and dynamic parameters.
> >>>>> Most of components distinguish producer (writer) and consumer
> >>>>> (receiver) options, most complicated components ended up with 40+
> >>>>> options settable via URI. Having a mix of PlcValues which are
> >>>>> timestamped and
> >>>> 'qualified'
> >>>>> and not will complicate encoding logic (how we push above info to
> >>>>> IEC encoder/handler or decorate it?) and how we keep custom types
> >>>>> compatible with above?
> >>>>>
> >>>>> That is why I rather opt for extending API in a way which does not
> >>>>> force existing drivers to do anything. When any of community
> >>>>> members, regardless if its developer or user, will find it
> >>>>> necessary to be ported in another driver, she or he have right to
> >>>>> do it. It is not only yours duty to keep all drivers and languages
> the same.
> >>>>>
> >>>>> Speaking of which I am also not concerned about making all
> >>>>> languages having the same functionality as languages tend to offer
> >>>>> different options which might be hard to be ported. Lets stick to
> >>>>> basics which are same (connection strings, field definitions,
> >>>>> read/write/subscribe
> >>>>> syntax) and let various runtime lead their own ways toward
> >>>> implementation.
> >>>>>
> >>>>> Key point is simple - we won't be able to maintain all languages
> >>>>> equally. We should not hold moving ie. go or java forward because
> >>>>> we can't port new API functionality to C/Python/C#/whenever. If
> >>>>> there is someone who *does* use above in production and *wish* to
> >>>>> have new API parts then that person/organization duty is to do it.
> >>>>>
> >>>>> Best,
> >>>>> Łukasz
> >>>>>
> >>>>>
> >>>>> On 08.03.2021 13:17, Christofer Dutz wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> I must admit that I would be in favor of keeping it as simple
as
> >>>>> possible (max age in the connection string) and to implement more
> >>>>> of the missing parts in plc4x (like the subscription simulation
> >>>>> layer) and hereby getting the drivers we have a bit more aligned,
> >>>>> than to implement more and more special API features, that only
> >>>>> one or two drivers support. Cause this way we're running away form
> >>>>> the project-promise of running a given program at any type of PLC
> >>>>> with any
> >>>> protocol.
> >>>>>>
> >>>>>> I mean ... at the current velocity the project is underway,
I
> >>>>>> wouldn't
> >>>>> really want to add a new MetaData layer that probably in the end
> >>>>> only
> >>>>> 1-2 people will be implementing and which we must not only find
> >>>>> out how to fill with life in the first place, but also port to all
> >>>>> of our supported languages.
> >>>>>>
> >>>>>> If someone's willing to bring in some serious man/womenpower,
I'm
> >>>>>> fine
> >>>>> with that ... or we add it to some wish-list for future extensions.
> >>>>> But I wouldn't want to start something like this right now.
> >>>>>>
> >>>>>> Chris
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> -----Ursprüngliche Nachricht-----
> >>>>>> Von: Łukasz Dywicki <luke@code-house.org>
> >>>>>> Gesendet: Montag, 8. März 2021 13:07
> >>>>>> An: dev@plc4x.apache.org
> >>>>>> Betreff: Re: Source Time of PLC Value?
> >>>>>>
> >>>>>> I have not mentioned metadata term in my answer, so credits
for
> >>>>>> mining
> >>>>> it API go to you. :-)
> >>>>>>
> >>>>>> Now since you brought it I remember that JDBC has a "result
set
> >>>>> metadata". That might be thing we still miss. The result code is
> >>>>> most important and available instantly, other information is
> >>>>> supplemental and can be read additionally.
> >>>>>>
> >>>>>>
> >>>>>> Best,
> >>>>>> Łukasz
> >>>>>>
> >>>>>> On 08.03.2021 13:01, Ben Hutcheson wrote:
> >>>>>>> +1 for Łukasz‘s metadata approach, it would avoid having
to
> >>>>>>> +create
> >>>>>>> duplicate PLcValue classes and give us a lot of flexibility
in
> >>>>>>> the
> >>>>> future.
> >>>>>>>
> >>>>>>> On Mon, Mar 8, 2021 at 5:14 AM Christofer Dutz
> >>>>>>> <christofer.dutz@c-ware.de>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> sorry for being late to the party ... KNX is currently
> >>>>>>>> consuming all my cycles.
> >>>>>>>> Just wanted to add my thoughts to the discussion.
> >>>>>>>>
> >>>>>>>> Initially I thought about adding this sort of information
to
> >>>>>>>> the API, but then I thought that we have so little protocols
> >>>>>>>> supporting this sort of concept that blowing up every
PLCValue
> >>>>>>>> with this time information just would waste CPU time
and Memory.
> >>>>>>>>
> >>>>>>>> When it comes to simulating subscriptions by polling
in the
> >>>>>>>> background, I thought it would be a valid solution to
provide a
> >>>>>>>> default age a value can have in the cache and to make
it
> >>>>>>>> possible to override this in the connection string options.
If
> >>>>>>>> a value is too old,
> >>>>> a return code of:
> >>>>>>>> STALE_DATA could be returned.
> >>>>>>>>
> >>>>>>>> Also, we do have an option of adding an additional set
of
> >>>>>>>> PlcValue
> >>>>> types.
> >>>>>>>> If a driver supports this quality-of-service type of
data, we
> >>>>>>>> could return special versions of PlcValues, that have
these
> >>>> properties?
> >>>>>>>>
> >>>>>>>> This way we don't have to waste the CPU time and Memory
for
> >>>>>>>> protocols that don't support this concept (We could
even
> >>>>>>>> implement the methods on the default PlvValues to simply
return
> >>>>>>>> some constants to the PlcValues all have the same API.
> >>>>>>>>
> >>>>>>>> Chris
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> -----Ursprüngliche Nachricht-----
> >>>>>>>> Von: Łukasz Dywicki <luke@code-house.org>
> >>>>>>>> Gesendet: Montag, 8. März 2021 10:43
> >>>>>>>> An: dev@plc4x.apache.org
> >>>>>>>> Betreff: Re: Source Time of PLC Value?
> >>>>>>>>
> >>>>>>>> Looking at issue we could utilize marker interfaces.
> >>>>>>>> At least in Java we could define Qualified and Timestamped
types.
> >>>>>>>>
> >>>>>>>> Yet looking at present state of API we are unlikely
to make
> >>>>>>>> each and every PlcValue in each and every variant (qualified
> >>>>>>>> and timestamped, timestamped, qualified). It would force
quite
> >>>>>>>> a lot of
> >>>> wrapping.
> >>>>>>>> Safest option I see is to extend subscription/response
API with
> >>>>>>>> additional method to retrieve all markers. For most
of drivers
> >>>>>>>> it would return just an empty set. It leaves us also
an open
> >>>>>>>> door for future drivers (or devices) to ship additional
piece
> >>>>>>>> of metadata which
> >>>>> does not fit into present API.
> >>>>>>>>
> >>>>>>>> Best,
> >>>>>>>> Łukasz
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 08.03.2021 04:30, Otto Fowler wrote:
> >>>>>>>>> So, maybe the extension is timestamp *and* timestamp
source.
> >>>>>>>>>
> >>>>>>>>> uint64_t timestamp;
> >>>>>>>>> typedef enum {
> >>>>>>>>>     /* timestamp was generated by the source device
*/
> >>>>>>>>>     SOURCE,
> >>>>>>>>>     /* timestamp was generated by plc4x on receiving
*/
> >>>>>>>>>     GENERATED,
> >>>>>>>>> } TimestampQuality;
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On Mar 5, 2021, at 18:04, Ben Hutcheson
> >>>>>>>>>> <ben.hutche@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> We do have some error codes, but it would just
need to be
> >>>>>>>>>> extended a bit. I don't think we have one for
BAD (we still
> >>>>>>>>>> receive data but the source has marked it bad).
> >>>>>>>>>>
> >>>>>>>>>> Timestamps I think are definitely a good idea,
especially if
> >>>>>>>>>> we eventually DNP3.
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Mar 5, 2021 at 5:39 PM Łukasz Dywicki
> >>>>>>>>>> <luke@code-house.org>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> You have response code for each individual
field which
> >>>>>>>>>>> allows you to determine state (OK, REMOTE_ERROR
etc.). Same
> >>>>>>>>>>> information should be available also for
subscriptions.
> >>>>>>>>>>>
> >>>>>>>>>>> Best,
> >>>>>>>>>>> Łukasz
> >>>>>>>>>>>
> >>>>>>>>>>> On 05.03.2021 21:04, Andreas Vogler wrote:
> >>>>>>>>>>>> Ah, forgot: and is there a status available?
> >>>>>>>>>>>> Valid/invalid/good/bad
> >>>>>>>>>>> indicator of a plc value?
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On 05.03.2021, at 21:02, Andreas
Vogler
> >>>>>>>>>>>>> <andreas.vogler@me.com.INVALID>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> How can I get the source time of
a PlcValue?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> a) from a subscription PlcSubscriptionEvent
> >>>>>>>>>>>>> b) from a read request  PlcReadResponse
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Best regards,
> >>>>>>>>>>>>> Andreas
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message