Best Practices for Big Data Support on OPC UA Clients

This article is relevant for all .Net products

When dealing with big data (lots of variables, that shall be read, written or subscribed), there are three general aspects that have to be considered:

- Message Size
- Timing
- Resources

All OPC UA services are designed to handle the requests and responses for multiple nodes (like read, write and subscribe) and many of them can contain optional data.

In general, the more nodes are handled within service calls (the bigger the messages), compared to several smaller service calls, the less is the message overhead and the better the performance.

On the other hand, be aware that the OPC UA stacks have has a maximum message size, e.g. the C Stack of the OPC UA C++ SDK has a fixed size of 16 MB.
Trying to send or receive bigger messages will produce errors. Try to limit the maximum message size by splitting too big service calls to several smaller ones or by specific configuration options (like Subscription::setMaxItemsPerPublish() or Application::setMaxMonitoredItemsPerService()).

Splitting operations into several smaller ones might increase the total required time, but might help to prevent timeout problems, as every single service can be processed faster.

Regarding the optional data, try to strip down not required optional information to reduce the message size and increase the speed.
For example when reading values, you can reduce the size of each transported value by 16 bytes by not requesting the Server- and SourceTimestamps of the values.

The more data is transferred, the longer it will take to process it. Have a look on the configured service timeouts, they might need to be increased to prevent timeouts.

The client itself shouldn't have any resource problems, as a client usually does not have big memory storages or value monitors, but it influences the server's resources, like memory and CPU usage.

Theoretically the server should define proper limits to prevent resource problems, but even the client might think about a proper usage.
For example the subscription service requires some memory for the buffering and some CPU for the observation at the Server, where the read service only needs to copy current values once but cannot collect only the changed values.
Try to use subscriptions only for regularly changing nodes or for data that is important to be received as soon as possible (like alarms and events).
Prefer the read service to receive values that have to be received seldom.