Hi,

In theory with the source edition you could change everything, though probably not very practical, so hopefully we could find some solution that doesn’t require that..

All values we use are Objects (ok, internals of them might not be, e.g. ByteString has byte[] internally, but meaning API-wise or top-lvl-object-wise), thus if they are not values that JVM would cache (e.g. Integer has a cache of values from -128 to 127), the object header will be part of each element. Though, still a 50kb binary probably takes less than a 1Mb as Objects in the worst case (or well, I would need to calc that for multidim, it could sky-rocket maybe under certain scenarios), but obiviously that is still a lot (multiplied by amount you read at once). Personally I would prefer to keep everything as Objects and not use primitives, since they require special handling in each step whereas Objects generify well. Plus eventually I think JVM devs will either make primitives useable in generics or implement value types (https://openjdk.java.net/jeps/169), at which point it is not a memory problem (assuming of course that people can upgrade to use those java versions).

How many dimensions we are talking about?
Java does not have (true) multidimensional arrays. It can only have “arrays of arrays of arrays of….”. The difference is that each array itself is an Object, thus it has the Object header overhead. Thus in theory this could multiply by a large margin (but the number of dimensions probably would have to be a lot, or something). Thus a single dimensional array of the same values could use considerable amount of less space. In UA Binary encodings all arrays are single dimensional and if the actual value was multidim, it stores the dimensions of the “box” to be then unraveled/constructed to the actual value. Currently the Variant does store the value as multidim, in theory it could also instead store the 1-dim value with the ArrayDimensions (would take less space if not used for anything, though then at least in the current API it would have to be re-constructed each time it would be accessed).

How much custom logic can you build? That is to say, is it acceptable that you build a custom solution for this particular case, or should it be something that must be part of a “generic client”?

If you would HistoryRead just a single value, would that work?

If the answer to above questions is that you can build custom logic to just this case, you could try to read the values by limiting the number of values per node to 1 and possibly limiting the nodes to request from also to 1. Though, the methods of UaClient (that usually are used here) do not (yet) provide a “stream processing” way of handling the call, thus you would need to call the method that has signature:

public HistoryReadResult[] historyRead(HistoryReadDetails details, TimestampsToReturn timestampsToReturn,
Boolean releaseContinuationPoints, HistoryReadValueId… nodesToRead)

Eventually all the methods do call that one, but here you must manually keep track of the so called ContinuationPoints (which allows you to continue the operation if the numValuesPerNode within e.g. ReadRawModifiedDetails is not enough to return all values from the time period you requested). You can check the call chain from the method you are currently using to eventually that being called (most likely the one you call now will call that in a loop until all values have been received; which is sort of fine but it returns them all at once and probably this is too much memory usage). We probably should build some sort of Iterator style (or Stream, if in Java 8 world at some point).