Avatar

Please consider registering
guest

sp_LogInOut Log In sp_Registration Register

Register | Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
Custom Ua Server Performance issues with large variable count
May 18, 2023
0:06, EEST
Avatar
ChilsonR
Watertown, SD, USA
Member
Members
Forum Posts: 3
Member Since:
July 26, 2017
sp_UserOfflineSmall Offline

I am migrating an existing ProSys DA custom OPC Provider service to the UA platform. The service provides a large point database for a network analysis platform which subscribes to the points provided by this service. We publish about half of our 160,000 available SCADA points via OPC DA currently, but now have issues since the network analysis platform has moved to the UA platform. Thus I have taken our existing OPC Provider custom service and retro fitted it to the UA server much like I did the original DA OPC Provider. I used the sample DA server before to model our custom service and also did so for the UA server. What I am noticing is that the process of adding the TUaVariable Node to the NodeManager via the AddComponent() call seems to be a large bottleneck in this process. I am seeing the progress bar for the build process move along well for the first one-third of the points, but then slow to a crawl afterwards. I’m assuming what’s possibly going on is further memory allocations for the larger address space for the additional points/nodes. The build finally get finished but only after 30 minutes or more, which is unacceptable. The DA OPC Provider also shows this performance issue, but typically completes in 5 minutes or less. Thus I’ve attached some of the sample code below and screenshots of the app with the timings I’m reporting.

// Then create the Tag’s RTU & Area prefex name
RtuTag = Area + “_” + Strutils::RightStr((“0000″+RtuNum),4);

// Now check that the SDT is addressable
// Otherwise it will skip this RTU
// The SDT is used to determine the that the
// point exists via the RTUnum is in Pt Src RTUnum
SDT* Sdt = SdtGet( StrToInt(RtuNum) );
if( Sdt != NULL )
{
// Now select the point Class Analog (4)
ptcnt = RtuRow->Strings[7].ToInt();
if (ptcnt > 0)
{
// Create the Point Class tagname prefex and loop thru these
ClsTag = RtuTag + “04”;
for (int j = 0; j Algi( j );
if ( ( Algi != NULL ) && (Algi->SrcRtu() == StrToInt(RtuNum)) )
{
// build the name of the point tag name
TagName = ClsTag + RightStr((“0000″+IntToStr(j)),4);
// For analogs use a Float type variable – ODMS only uses
// two types of variables – so create and assign the tagname
TPsFloat *Analog = new TPsFloat(this);
Analog->Name = TagName.c_str();
UaAnalog = MyNodeManager->CreateVariable(TagName.c_str());
UaAnalog->DataTypeId = Id_Double;
//UaAnalog->Value = TUaVariant::Create(0.0);
// Place this into three structures or lists:
// 1. the OPC Server’s Address Space
// 2. the internal TPsVar Tag List
// 3. UaVars sorted list for quick ref used for point updates
// 4. Tags sorted list for quick ref used for point updates
MyVars->AddComponent(UaAnalog);
MyTags->Add(Analog);
UaVars->AddObject(TagName,(TObject *)UaAnalog);
TagList->AddObject(TagName,(TObject *)Analog);
}
}
} //End of test for point count greater than zero for Analogs

[not able to show screen shot of app] The above loaded 68,277 points in 2141 seconds.

But if the variables are simply built but not added to the NodeManager via the Addcomponent() call as show here:

// Then create the Tag’s RTU & Area prefex name
RtuTag = Area + “_” + Strutils::RightStr((“0000″+RtuNum),4);

// Then create the Tag’s RTU & Area prefex name
RtuTag = Area + “_” + Strutils::RightStr((“0000″+RtuNum),4);
// Now check that the SDT is addressable
// Otherwise it will skip this RTU
// The SDT is used to determine the that the
// point exists via the RTUnum is in Pt Src RTUnum
SDT* Sdt = SdtGet( StrToInt(RtuNum) );
if( Sdt != NULL )
{
// Now select the point Class Analog (4)
ptcnt = RtuRow->Strings[7].ToInt();
if (ptcnt > 0)
{
// Create the Point Class tagname prefex and loop thru these
ClsTag = RtuTag + “04”;
for (int j = 0; j Algi( j );
if ( ( Algi != NULL ) && (Algi->SrcRtu() == StrToInt(RtuNum)) )
{
// build the name of the point tag name
TagName = ClsTag + RightStr((“0000″+IntToStr(j)),4);
// For analogs use a Float type variable – ODMS only uses
// two types of variables – so create and assign the tagname
TPsFloat *Analog = new TPsFloat(this);
Analog->Name = TagName.c_str();
UaAnalog = MyNodeManager->CreateVariable(TagName.c_str());
UaAnalog->DataTypeId = Id_Double;
//UaAnalog->Value = TUaVariant::Create(0.0);
// Place this into three structures or lists:
// 1. the OPC Server’s Address Space
// 2. the internal TPsVar Tag List
// 3. UaVars sorted list for quick ref used for point updates
// 4. Tags sorted list for quick ref used for point updates
//MyVars->AddComponent(UaAnalog);
MyTags->Add(Analog);
UaVars->AddObject(TagName,(TObject *)UaAnalog);
TagList->AddObject(TagName,(TObject *)Analog);
}
}
} //End of test for point count greater than zero for Analogs

[not able to show screen shot of app] The above shows that 68,277 points/TUavariables were built in 37 seconds.

Please advise any possible remedy, we are currently thinking living with the periodic restart of the old DA OPC Provider may be more acceptable. Just a bit of background, we are a large regional transmission and generation control facility in the USA covering an 8 state area, thus the large point count.

Thank you,

Randall Chilson

May 19, 2023
11:57, EEST
Avatar
Jouni Aro
Moderator
Moderators
Forum Posts: 1010
Member Since:
December 21, 2011
sp_UserOfflineSmall Offline

Thanks for a detailed report, Randall.

I can confirm that AddComponent can get very slow if you have a lot of components that you are adding. The references are kept in a list (to keep original order), and it’s verifying that the reference is not already in the list to avoid duplicates.

I can work on a more optimal algorithm to make sure that you can start your server faster. I will continue by email with you to provide you an update as soon as possible, as well.

You might also want to consider a more hierarchical layout for the components, so that they wouldn’t all be under the same folder. This might help the client applications also to browse and show all the variables – in several subfolders instead of one big folder.

June 21, 2023
16:50, EEST
Avatar
ChilsonR
Watertown, SD, USA
Member
Members
Forum Posts: 3
Member Since:
July 26, 2017
sp_UserOfflineSmall Offline

Thank you for the updates provided. Yes, I have added a hierarchical factor to the layout of the components which greatly assisted with using the browser in loading/mapping the points for the browser. The updates provided, greatly increased the add time for our 75,000 points, in fact the new component add now operates much quicker than even the older DA version of the product which typically took 5mins is now down to 100 seconds for the UA version of the product. We now have created connectivity to the client product that will be consuming the point data and now working with their support on items related to their product, but see an end insight. Thank you again so much for your assistance and we’ll look forward to the formal release of the updated product.

Randall Chilson

June 28, 2023
9:35, EEST
Avatar
Jouni Aro
Moderator
Moderators
Forum Posts: 1010
Member Since:
December 21, 2011
sp_UserOfflineSmall Offline

Yes, this performance improvement was released as part of Sentrol 7.6.0. Thank you for the good feedback and pushing us to make it Smile

Forum Timezone: Europe/Helsinki

Most Users Ever Online: 518

Currently Online:
19 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

hbrackel: 135

pramanj: 86

Francesco Zambon: 81

rocket science: 77

Ibrahim: 76

Sabari: 62

kapsl: 57

gjevremovic: 49

Xavier: 43

fred: 41

Member Stats:

Guest Posters: 0

Members: 681

Moderators: 16

Admins: 1

Forum Stats:

Groups: 3

Forums: 15

Topics: 1467

Posts: 6261

Newest Members:

graciela2073, sagarchau, elviralangwell4, Donnavek, Eddiefauth, DonaldPooma, fidelduke938316, Jan-Pfizer, DavidROunc, fen.pang@woodside.com

Moderators: Jouni Aro: 1010, Otso Palonen: 32, Tuomas Hiltunen: 5, Pyry: 1, Petri: 0, Bjarne Boström: 983, Heikki Tahvanainen: 402, Jukka Asikainen: 1, moldzh08: 0, Jimmy Ni: 26, Teppo Uimonen: 21, Markus Johansson: 42, Niklas Nurminen: 0, Matti Siponen: 321, Lusetti: 0, Ari-Pekka Soikkeli: 5

Administrators: admin: 1