Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Rate limiting for server
June 21, 2022
12:45, EEST
Avatar
t.tulka
Member
Members
Forum Posts: 15
Member Since:
March 31, 2022
sp_UserOfflineSmall Offline

Hello,
Is there any possibility of rate-limiting in the server provided by the SDK for Java?

– Rate limit for opening a secure channel by IP addresses (eg. max 10 connections per minute per IP)
– Rate limit for browsing by the user identity
– Rate limit for reading attributes (both non-value and value) by the user identity

If the SDK provides no solution out of the box, what would be the right approach to implement that by the custom application?

I guess for the latter two an implementation of NodeManagerListener could be used. I am not sure about the first one. Is there any event listener for connection establishment with detailed client information like IP etc.?

Thanks!

June 21, 2022
15:50, EEST
Avatar
Bjarne Boström
Moderator
Moderators
Forum Posts: 1026
Member Since:
April 3, 2012
sp_UserOfflineSmall Offline

Hi,

Well, in short “Rate limiting” is not a thing in OPC UA (as far as I’m aware, could be wrong, but basically this is how it has been the last 10-15 years).
Connections could be limited (specification talks about this), though our SDK doesn’t offer a way to limit them (we might someday). It can be done outside the application/SDK in the firewall level, which is also probably a better idea to prevent DOS attacks etc. Also an IP address might not be a good metric, companies and some ISPs do use NATs thus the same IP might be for very many connections and all could in theory be valid.

SDK does limit the number of Sessions (that is by default 100, in SessionManager of the UaServer).

Once there is a valid Session, in general it would not be expected for e.g. a Read to “randomly fail” due to any “rate limiting” etc. a response would just be returned once possible. This is to say, clients would not know how to deal with this kind of a failure and I do not think there is exactly any suitable status code. There is Bad_TcpNotEnoughResources, but that would be only if the tcp connection could not be formed. Probably clients would just resend the request immediately.

Is there a specific use-case you have in mind that would require this (for Read/Browse)?

Anyway, in technical terms, there is a shared 64 thread “blocking work pool” in StackUtils, which is used in very many things (more in the “stack layer”) due to historical reasons, but it also basically means at max 64 service calls are processed concurrently. It doesn’t exactly prevent clients for sending more (though at some point the Selector that multiplexes all sockets would get slown down as well and then the tcp buffers would fill and that would maybe eventually on the tcp level prevent clients sending more), but I guess can somewhat be thought as a “rate limiter”, but not in application level sense.

June 21, 2022
16:40, EEST
Avatar
t.tulka
Member
Members
Forum Posts: 15
Member Since:
March 31, 2022
sp_UserOfflineSmall Offline

Many thanks, your answer is sufficient so far.

Bjarne Boström said Is there a specific use-case you have in mind that would require this (for Read/Browse)?

Yes, I am evaluating a solution where clients have “packages” characterized by the amount of data they can request.

June 21, 2022
17:27, EEST
Avatar
Bjarne Boström
Moderator
Moderators
Forum Posts: 1026
Member Since:
April 3, 2012
sp_UserOfflineSmall Offline

In that case I do want to note a bit more.

I do not think it would be typical OPC UA (though you can ask the OPC Foundation, they would probably be better to answer that).

IF you meant like this, there is a status code, https://reference.opcfoundation.org/v104/Core/docs/Part4/7.34.2/, Bad_LicenseLimitsExceeded, though I do not think it is at least intended to be used exactly like this. Plus it might be a bit newer definition, thus older clients would not know what it means.

The biggest “problem” (and half of the point of OPC UA) probably is that OPC UA is self-descriptive, i.e. there is a lot of “metadata” in the information modelling in addition to the “real data”. It might be hard to differentiate that (maybe on the namespace level it is easy, after that harder). Basically the clients must be able to read this metadata, always, repeatedly maybe, or they will fail. This is to say, at least half of the point of OPC UA is that clients can read this metadata about the data.

Also, we do read the ServerStatus once per second. This must work, a failure would trigger a reconnect. I would expect other clients to do something similar.

Also nowadays we do read the whole Types part of the server. While that is a bit crazy, it works out quite well in practice in most cases, and is somewhat needed anyway for custom Structures (and basically getting all/rest of the types at the same time is somewhat efficient). That is typically around ~50000 operations (which on our defaults server+client will get packed to around 5-10 service calls) on connect (this can also be disabled, though any interaction with UaNodes will then result in ~100 service calls instead; depending a bit what is done as the meta info are also nodes and it kinda works recursively one-by-one if we have not read them on connect).

June 22, 2022
9:45, EEST
Avatar
t.tulka
Member
Members
Forum Posts: 15
Member Since:
March 31, 2022
sp_UserOfflineSmall Offline

I see, thanks for the reply.

I guess Bad_LicenseLimitsExceeded is the right choice as it pretty much describes the use-case:

The UA Server has limits on number of allowed operations / objects, based on installed licenses, and these limits where exceeded.

Of course, it must be communicated to the clients as a limitation – but it is other than a technical concern.

I understand the problem with reading the definitions (Types). But this shall not be a concern as the limits would be applied only to reading “real data” based on the user identity. In particular, as I suggested, this could be potentially achieved by implementing NodeManagerListener’s onBrowseNode method that checks the user identity and its “license” and the node being read. Moreover, the listener could be added to only a particular namespace to leave the Server namespace and others completely unlimited.

Does it make sense?

June 22, 2022
12:06, EEST
Avatar
Bjarne Boström
Moderator
Moderators
Forum Posts: 1026
Member Since:
April 3, 2012
sp_UserOfflineSmall Offline

In tech terms “yes”, but as a general concept, “no”.
I will repeat that as far as I know or remember (of the ~10 years I have been working with OPC UA) it would be the first Server to do this (obiviously I will see just a slice of the world, but still) . So Clients _really_ would NOT be expecting it. Also clients might not be able to directly control when they Read, for example our client side SDK using UaNodes will automatically re-read or Browse things on cache misses and expirations.

Also note that OPC UA has Subscriptions, thus any Read limits would be meaningless for clients using those. And limiting those is kinda hard in tech terms, as it is not, well, part of OPC UA. I’ll note that in the MyBigNodeManager the monitored items are controlled more as that example is a bit lower-API (doesn’t use UaNodes), but anything UaNodes might be hard to control, as SDK internally processes the value to the items (sort of what MyBigNodeManager does manually).

June 22, 2022
12:31, EEST
Avatar
t.tulka
Member
Members
Forum Posts: 15
Member Since:
March 31, 2022
sp_UserOfflineSmall Offline

I see, thanks for the input. I guess we have to clarify the requirements conceptually once more.

But staying purely on the technical aspect of the thing…
Regarding the monitored items, would NodeManagerListener’s onModifyMonitoredDataItem provide a solution here as well?

June 22, 2022
14:28, EEST
Avatar
Bjarne Boström
Moderator
Moderators
Forum Posts: 1026
Member Since:
April 3, 2012
sp_UserOfflineSmall Offline

I would recommend asking the OPC Foundation, they might have ideas (or verify my “this is not a good idea” note).

In technincal terms with the SDK, there are lot of listeners, and things could be subtyped and protected methods overridden. Not all, but most things eventually become possible, though we might not like support every possible thing made this way, i.e. https://www.prosysopc.com/services/ might be needed as it might require a lot of time to investigate is X possible.

However, that exact method doesn’t help, it gives a note that client requested changes in e.g. sampling interval. It could be used in MyBigNodeManager-style managers who are managing the monitored items on their own, as they must also do the data-changes manually to the items. IF you do all your own namespace managers like MyBigNodeManager, you could limit the data (though you cannot then use UaNodes). It still would not be like per specification I guess to do that or it would be a gray area of what exactly should happen. I guess you could push a DataValue with the statuscode mentioned above posts, but I would expect you to get lots of questions from people using your server.

So, in short in normal conditions and using UaNodes, you could prevent creation of an item or subscription or revise e.g. publishing interval, but once the item is done and active it will operate and (simplified) any value set on the UaNode will be seen by all the clients who have made an item to that node.

So you could e.g. have a limit of items/subscriptions in total per a session. Servers typically have some limits on these. But it would not affect how much data client will see form the item. I guess in theory you could have different revise logic for e.g. publishinng intervals, though that too would be a bit weird I think.

Forum Timezone: Europe/Helsinki

Most Users Ever Online: 1919

Currently Online:
23 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

Heikki Tahvanainen: 402

hbrackel: 144

rocket science: 88

pramanj: 86

Francesco Zambon: 83

Ibrahim: 78

Sabari: 62

kapsl: 57

gjevremovic: 49

Xavier: 43

Member Stats:

Guest Posters: 0

Members: 735

Moderators: 7

Admins: 1

Forum Stats:

Groups: 3

Forums: 15

Topics: 1523

Posts: 6449

Newest Members:

rust, christamcdowall, redaahern07571, nigelbdhmp, travistimmons, AnnelCib, dalenegettinger, howardkennerley, Thomassnism, biancacraft16

Moderators: Jouni Aro: 1026, Pyry: 1, Petri: 0, Bjarne Boström: 1026, Jimmy Ni: 26, Matti Siponen: 346, Lusetti: 0

Administrators: admin: 1