9:35, EET
February 21, 2014
Greetings!
for memory/storage efficiency (application specific) reasons I’d like to cache and persist the value attribute of data variables in “raw” byte format, i.e. not yet decoded, and decode them at a later time on demand. The values may be received either through a readAttribute() call or by a subscription callback.
What would be the most “elegant” or efficient way to accomplish this?
Thanks,
Hans-Uwe
10:55, EET
April 17, 2013
Hi Hans-Uwe,
Thank you for an interesting question. Can you tell what is the datatype in question?
This will be impossible at least for standardized datatypes. However, it seems you’re using some application specific datatype in this case. For example, if you have a customized structure value, it may be possible to skip the encoding step and treat the values as ExtensionObjects and then decode them later.
19:01, EET
February 21, 2014
Hi Heikki,
well, the approach would need to work for any and all data types – not just a particular one. Without going too much into any details about the application, I’m “scanning” an unknown server (emphasizing *unknown*), put its entire address space into a database and rebuild the server then from the database for simulation purposes. This includes the initial scan of the static address space components as well as subscriptions to selected variable values for a replay of scenarios. Everything works nicely for “basic data types” as I can extract the decoded values. But even for basic data types I’d like to avoid the decoding/encoding as the replay server really just serves the data without interpreting them. I’d like to store the raw bytes into the database as they come in from the wire and then replay them in the original format.
17:43, EET
April 17, 2013
Hi Hans-Uwe,
Thank you for the information. As mentioned before, you cannot avoid the automatic decoding and encoding of values. Furthermore, I’m not even convinced if this would make a noticeable difference in the performance. The application needs to decode the service call message anyway so whether or not it decodes the value should have only negligible effect on the speed of the communication.
That being said, it’s technically possible to store only the binary encoded values in database. Looking back to your original post, this is actually the point that you were interested in: ‘to cache and persist the value attribute of data variables in “raw” byte format’. You can achieve this by using EncoderCalc and BinaryEncoder to encode values and BinaryDecoder to decode values.
In other words: it’s not possible to stop the SDK (and underlying Java Stack) from decoding and encoding values. However, it’s possible to ‘manually’ encode the values for storage purposes if this is needed. If you manually encode the values, you’ll also need to manually decode them later on.
If you’re still interested in encoding the values for storage purposes, I’m happy to send more instructions about using the EncoderCalc, BinaryEncoder and BinaryDecoder. This is not a standard functionality that would be meant to be used by normal SDK users, so the documentation regarding these is not on a high level.
14:35, EET
February 21, 2014
11:43, EET
April 3, 2012
Hi,
In a general scenario it is not possible to avoid decoding the values (to a point) in OPC UA.
It would be technically possible to implement the decoding as such that you would get both the encoded and decoded values (after decoding), however the current architecture of the Java Stack (on top of which our SDK is built) does not make that possible.
Technical explanation below:
All Attribute Reads are done using the Read Service call. Client send ReadRequest, which contains an array of NodeId/Attribute combinations and then the server responds with ReadResponse.
I skip some of the details, however eventually the client has the ReadResponse as a byte array
ReadResponse contains:
* ResponseHeader
* Array of results
* Array of diagnostics
The encoding rules for Service requests/responses are the same as for Structures, each field is encoded in order to the stream, therefore the first bytes belong to the ResponseHeader. After that is the 2 arrays. Arrays are encoded as Int32 and after that each element in sequence.
The ResponseHeader (see spec 1.03 Part 4 section 7.29) size is not constant (if diagnostics are off it might be, however that does not help much here), therefore it must be decoded in order to know where the array of results begin. After that we can read how many DataValues are encoded. However as with the ResponseHeader, the encoded size of the DataValue is not constant, as it depends of the encoded value. Therefore the general problem here is that it is impossible to say which bytes belong to which DataValue without decoding them in order to know where the next one begins.
The DataValue.Value field (Variant) size depends on the value (type, array dimensions), unless it is a Structure (i.e. ExtensionObject), the size of the value is not part of the encoding. Most built-in-types have also masks that define which fields are present and such the entire value pretty much has to be decoded to determine the size.
After a DataValue has been decoded it should technically be possible to persist the raw value as the begin and end index for the original byte array for the entire request is now known. However the current stack implementation takes the entire request byte array as a queue and pops individual parts away. It would require some redesigns.
Additionally there are a few catches here that should be mentioned: Many of the encoded values do contain server-specific parts, namely the NamespaceIndexies, these are part of NodeIds and QualifiedNames, which makes the encoded values only valid if they were used for a server that has the same exact NamespaceTable as the one from which they are read. Also the encoded value is not the complete information. For example Enumerations are encoded as Int32. Therefore without the actual DataType Attribute value together with the EnumStrings Property of the DataType node, it is not possible to say which enum value it was. The same is with every subtype of some non-structure Built-in-Type, e.g. the Image subtypes are ByteStrings when encoded, therefore without the information model of the node it is not possible to know which image subtype it was.
– Bjarne
P.S. If only one attribute would be read at a time, then the decoding could be technically skipped (if diagnostics are off) as would be possible to know the exact position of that in the byte array. However at least in your case this would make the reading order of magnitude slower.
Most Users Ever Online: 1919
Currently Online:
53 Guest(s)
Currently Browsing this Page:
1 Guest(s)
Top Posters:
Heikki Tahvanainen: 402
hbrackel: 144
rocket science: 88
pramanj: 86
Francesco Zambon: 83
Ibrahim: 78
Sabari: 62
kapsl: 57
gjevremovic: 49
Xavier: 43
Member Stats:
Guest Posters: 0
Members: 746
Moderators: 7
Admins: 1
Forum Stats:
Groups: 3
Forums: 15
Topics: 1529
Posts: 6471
Newest Members:
qsireinaldo, scvchad954, misty3446453365, KelsonzFu, Kelsonz, lienbelisario, erick34s63346, Kaitlyntvsl, lonaerskine7, KTP21ideftModerators: Jouni Aro: 1026, Pyry: 1, Petri: 0, Bjarne Boström: 1032, Jimmy Ni: 26, Matti Siponen: 349, Lusetti: 0
Administrators: admin: 1