Hi,

I guess no way to avoid my typical long answers, sorry.

First a note that things with PubSub are noway near on the level I would like them to be, i.e. there is still a lot to be done, also OPC UA 1.05 will probably bring some improvements as well once it properly arrives (since people are pretty much only now starting to use PubSub, the spec typically evolves once someone actually uses something specified as typically it usually is not that great to start as use-cases are not that well known). Also, this answer is mostly related to UDP-UADP PubSub, as in MQTT-UADP + MQTT-JSON the publisher would publish the metadata to the Broker.

IF the Publisher supports metadata discovery requests (https://reference.opcfoundation.org/v104/Core/docs/Part14/7.2.2/#7.2.2.4.1), that will happen automatically. SDK makes a metadata request if the Reader didn’t have metadata. This is done on the PubSubUdpUadpConnectionConf.getDiscoveryNetworkAddress(), if specified (or tried on the normal getNetworkAddress() if not). Our Publishers do support this and this is basically the configuration we test with for the time being. Yes, this also requires a Publisher to listen to incoming messages which might sound odd.

There are at least 2 alternatives to the discovery request. One is to “just magically know it” (e.g. some external information or a “well known” DataSet). Another would be to check the server via Client-Server connection and Browse/Read the “PubSub configuration address space”.

We do not yet support the “PubSub configuration address space”, so for our Publishers the metadata discovery requests (or “just magically know it”) are the current options. However, note that the configuration types are part of the standard namespace i.e. you can Read/Browse them IF the Publisher you are connecting to does support them (per our IOP workshop partisipation it would seem some do and some do not yet and some have had some custom solutions as the address space model is … complicated). So you could observe that and then have the info that way (you just cannot yet directly push that to PubSubSystem without doing conversions yourself).

As for the SampleSubscriberClient. Long story, but in short our DataSetMetadata is not exactly correct yet. It has a flaw that the NamespaceArray and custom structures definitions are not sent within it. Only the DataSet’s fields and their types are. So we use the client-server connection to obtain NamespaceTable+definitions for the custom structures. Yes, this does mean that it must be _that server_ that is doing the Publishing. Or well, standard namespace data would work from any Publisher, as that is inherently known.

Also, basically the samples are split in 4 (server/client + publisher/subscriber), but nothing prevents having both publisher and subscriber in the same PubSubSystem.

Also, while not very useful yet, the SampleSubscriberClient is sort of the convert PubSub -> Client-Server while SamplePublisherClient is Client-Server -> PubSub. The SampleSubscriberClient doesn’t define TargetVariables (at least not yet), but that concept should work. If it would, it would Write the data received from Publishers to the server it is connected to. Probably at some point in the future we should make a “standalone PubSubSystem”.

Note that we too sort of have these questions, as we for the most parts are just implementing the spec, not specifying it.

Not all of those variables would be in the “filter” of the DataSetReader, the DataSetWriterId would be most important (with the PublisherId).

You are looking for a “get all data from a publisher” solution here. As far as I’m aware, there is no good way to do that (if at all; we too would like to do this as well for our Browser application soon). It would pretty much seem the intended way is to look the address space configuration (or some external thing) and construct DataSetReaders “one by one”.

Yes, the DataSetReader “filter” parameters can take in 0 as a wildcard. But per the address space configuration model, each DataSetReader could only hold a single metadata (https://reference.opcfoundation.org/v104/Core/docs/Part14/6.2.8/#6.2.8.11.1 + https://reference.opcfoundation.org/Core/docs/Part14/9.1.8/#9.1.8.2). So.. I’m not 100% sure yet of their usefullness. Maybe in the future we can offer some “none Readers matched this message” event or something that could be used to drive some automatic generation of the configuration.

While we sort of intentionally went with our own configuration format (different story for a different post), we do also support (only) a single metadata per reader.

Also I should note that for the time being, we pretty much have tested just those samples that come with the SDK. We have done some interop tests in the IOP Workshops the OPC Foundation hosts (though they have been quite basic at this point).

For your questions:

1.
No idea. But like sooner or later we must do that, as I would assume the need for custom structures will be there (i.e. “maybe this year”. I believe you are the first to sort of ask for this. Typically we try to do things that our customers have demand for (or if we need things in our apps).

What would be the “target” of the subscriber? Just visualizing it or actually doing something with it or pushing it to somewhere? Because, this is sort of unexplored territory, but IF the “push target” is an OPC UA Application, it sort of would also need to “know” the custom types. That is to say, the namespaceindex of namespaceuris of custom types can be different, but you could not have a TargetVariable of a Server that would not be of correct type (or assuming it was BaseDataType so anything goes, the OPC UA Client reading it from that server would not have the metadata then in turn, unless the server has it).

2.
I’m not aware of such. Those samples are basically the best we have to offer right now. The OPC Foundation’s .NET (https://github.com/OPCFoundation/UA-.NETStandard) might have some samples that have it.

An extremely .. expected and valid question, but also extremely more complicated topic than you maybe might think.

Since we mostly just implement the spec here (it is a bit .. more complicated, but basically easiest if you think it like that), this question would be better asked at the OPC Foundation’s own forums: https://opcfoundation.org/forum/. Also, our hands are still quite full with doing 1.04 stuff so we have not had that much time yet to look about 1.05 (it also was supposed to “just be” 1.04 with all amendments and errata texts added to it, but apparently it is not so easy..).

So the rest of the post are just some comments and opinions of mine (sorry, ended up too much a wall of text, but I might refer to this post in some other answers later):
1. “Versioning” in OPC UA is … complicated.
2. Backward compatibility is … subjective
3. Forward compatibility is … subjective
4. You must make your product in such way that you or your customer can update it, you almost certainly sooner or later need to update it.

Larger explanations:
I too use the terms “1.04” and “1.05”, but in reality that doesn’t tell much; the spec is too big. I’m not aware of a single SDK or application that would support every possible feature of OPC UA, there are also features that … no-one uses (obiviously eventually we would like to support everything, but is more like new features are added faster than we can implement old ones, at least currently; depends also what our customers want). The “OPC UA way” to better indicate what is supported is with Profiles and Facets (https://reference.opcfoundation.org/Core/docs/Part7/). But… as the spec … evolves, so have the profiles as well.. now there is e.g. “Standard 2017 UA Server Profile” (https://reference.opcfoundation.org/Core/docs/Part7/6.6.71/) and the “old one” is “Standard UA Server Profile” (https://reference.opcfoundation.org/Core/docs/Part7/6.6.70/).

The “version” DOES, but also DOES NOT affect the “binary protocol”, which eventually is all that matters. Note that I’m here talking about opc.tcp (i.e. Client-Server stuff). Basically, you could say it is “sort of unchanged” from the very first specification version, thus it is “backwards-compatible”. This is because the Built in types (https://reference.opcfoundation.org/Core/docs/Part6/5.2.2/) are still the same as they were. BUT! all Structures are encoded as ExtensionObjects. We have gotten new rules and new Structure types. The very latest in the very latest nodeset 1.04.10 we got 2 new types (in OPC UA 1.03 also 2 new kinds of structures). For these specific 2 new types, the client must be 1.04.10 or it would have no knowledge how the ExtensionObject should be decoded to the real Structure instance. So, … is the binary protocol “truly backwards-forwards-compatible”? Like e.g. 1.01 Client could most likely “Read any 1.04 Structure”, but does it help if it cannot ever decode the ExtensionObject binary blob?

Also, the metadata of Structures before 1.04 was in so called “DataTypeDictionaries”. 1.04 introduced a new DataTypeDefinition Attribute (and well, new Attributes for the first time in history). And what I looked at the 1.05 the old DataTypeDictionary way is marked as deprecated. Also, 1.04 structures gained the ability to have multidimensional fields (or well, their encoding was defined). That is not representable with a DataTypeDictionary, only by DataTypeDefinition. So, assuming a server has the old DataTypeDictionaries and only models data with structures that do not use 1.04 new rules, an older client could get it and decode the Structures. IF anything new is used, typically the client could not understand it. Basically a 1.03 Client would not even know there are new Attributes.

Also, I believe from the client-server’s securitypolicies none of the 1.01 policies are anymore defined as “safe to use” in 1.04. So, the security as well is sort of “backwards-forwards-compatibility problem”.

So, I would expect (note that this is an assumption!) the basic encodings of the PubSub messages to remain as-is and be (truly) backwards-compatible. In general the OPC Foundation does have motivation to “of course” keep backwards-compatibility, but also a lot new stuff has been specified. That is why I say it is “subjective”. For example, I believe there was something to be done with how complicated the address space configuration model of PubSub is and there might be in 1.05 a way to “update whole configuration”. Thus, IF that happens, some servers might … not support the 1.04 way of updating the configuration. Given that our SDK doesn’t have that yet (technically with a lot of work it is “user-doable” but most likely not really), depending on how the things go it might make sense for us to only support that 1.05 way (IF 1.05 Part 14 is out at a suitable time, I have no knowledge when it would be released).

In OPC UA 1.03 the SOAP/HTTP transport option was deprecated/removed as basically “no one” had implemented that (except maybe one implementation etc.). I have not seen anyone implement the “AMQP stuff” of PubSub yet, maybe same thing happens again.

However, note that the SDK is “inbetween” your application and the spec. For example we made the StructureSpecification thingy to abstract the structure metadata. Client tries to read DataTypeDefinition, but failing to do that it can also try to look up the older DataTypeDictionaries (if the server has them, IF it doesn’t use 1.04 new stuff for Structures). If something new in the spec appears, it may be possible that we can represent it with our existing API. Obiviously you still need to update the SDK for this to happen “within the SDK”. Thus, due to this (and in general for security updates) it is a good idea to design you product in such way that it is somehow updateable.

However, given that PubSub is basically just now being really used, both spec and SDK-wise, larger changes can happen (i.e. be expected) than if the thing would be .. 15 years old (which OPC UA is, PubSub only few years now and that was mostly “not used by anyone” period). So it might make sense to fix some things now than be stuck them forever. This is also why SDK’s API regarding PubSub .. is less set in stone than client-server parts.