Microsoft Word - Pega Offline Sync JMeter Framework.docx

Pega Offline Sync JMeter Framework

The Pega Offline Sync JMeter Framework offers the essential building blocks to construct a

JMeter test plan to simulate most Pega mobile offline-enabled applications. Such a test plan

allows load testing a particular Pega server environment to determine the sustainable peak

concurrent load for a particular usage scenario exercised with the mobile application.

Quick Start

These are the essential high level steps to put together such a JMeter test plan:

STEP 1: Using Charles Web Debugging Proxy or Fiddler to act as a proper proxy between a

mobile device and the Pega server, record the HTTP transactions that result after having

the device disconnected from the network (so it is in full offline mode), and exercising

the particular usage scenario that you wish to load test.

STEP 2: Use the provided Pega Sync Action Extractor tool to unencode (if necessary),

transform and extract the individual actions that were generated due to the offline

mobile application interactions.

STEP 3: Populate the Decomposed Actions component within a copy of the provided sample

Jmeter test plan by copying individual action content from the Pega Sync Action

Extractor tool and pasting into each incremental Action variable within the component.

STEP 4: Adjust the other configurable portions of the Jmeter Test Plan with server and

application specific values (as necessary).

That’s it. The complex JMeter test plan provided with the framework is coded to simulate

much of the Pega mobile offline client’s behavior--issuing the same HTTP sync transactions that

the device would generate, and properly dealing with the variable content within these HTTP

transactions. It also offers many knobs (via Jmeter variables) that may be adjusted to further

explore the load the server might sustain.

More detailed step explanations

STEP 1: Capturing all the mobile “Actions” generated from the key mobile application usage

scenario

As you open and manipulate a Pega Mobile Offline application, various actions are queued up

within the mobile device and eventually sent to the server when a network connection to the

Pega Server is present. The goal of this step is to capture all such actions so that specific

dynamic (varying) values within these payloads might be properly marked up so that they might

be substituted with appropriate dynamic runtime values when the completed JMeter test plan

is eventually executed.

If using a Mac desktop OS, Charles Web Debugging Proxy is the best tool to capture the HTTP

transactions to the Offline service package. To use Charles, make sure to read the “Help/SSL

Proxying/Install Charles Root Certificate on a Mobile Device or Remote Browser” section within

the Charles client. Also see:

https://community1.pega.com/community/pega-support/question/configuring-charles-web-

proxy-mobile-devices

If using a Windows development environment, Fiddler Web Debugging Proxy provides similar

capabilities. See this video within the Pega Community to help set this up for Windows:

https://community.pega.com/video-library/support-guide-using-fiddler-collect-network-traces-

ios-device-pega-mobile-app

Once the Web Debugging Proxy has been setup to properly capture network traces, confirm

that transactions to /prweb/PRRestService/offlinehttp/DataSync/pzSynchronizationService are

being captured. (If using Charles, you may need to set Proxy/Recording Settings and Proxy/SSL

Proxying Settings)

Once the web proxy is confirmed to be setup properly (able to see sync transactions and look at

the POSTed payloads), turn off the Wi-Fi on the mobile device. This will force the Pega

application into true offline mode. Now run through the single usage sequence you desire to

load test. After going thru the desired case manipulation sequence, enable Wi-Fi once again.

Bring the Pega mobile application to the foreground, and the synchronization transactions will

now be seen as they are sent to the Pega server with the various “createNewWork” and/or

“finishAssignment” Pega process engine actions within these transactions. If the application

was customized to utilize the pega.offline.DataSync.setQueueMaxSize API to adjust the queue

size to something other than the default “1”, it will be necessary to wait for the default flush

interval (default 5 mins) to make sure all the actions are flushed.

A Request Approval sample app generated by using Pega Express 7.4 is provided with this

framework posting. Two Charles Proxy capture files are also provided to show the actual

capture from the application from a Pega 7.4 as well as a Pega Infinity 8.1 system. These might

be used as the starting point for exploring the other steps in creating a JMeter test plan (if you

have difficulty capturing the network trace with your own app).

STEP 2: Transform and get ready to Extract the captured Actions

For this step, a web page tool is provided with the framework to inspect, transform and extract

the captured actions that are significant to your key application usage scenario.

To run the app, open the “PegaSyncActionExtractor.htm” from Chrome (or another browser).

(Note: The page has been primarily tested with Chrome but should work well with other

browsers as well)

When open the tool displays the following UI:

Area 1 is the area within which the raw Post payload content should be pasted (after copying

from the Web debugging proxy tool (e.g, Charles or Fiddler)).

Area 2 represents the set of regular expressions that would be run on the captured payload to

transform the content to insert variables for which values will be generated by the JMeter Sync

Framework test plan. This set of rules can be expanded and modified as appropriate for your

application. For example, the rules presently look for an operator value of “loadmobile.user”

and case prefix “ReqApp-“, but your application will likely have a different initial operator and

different case prefix. You can adjust the replacement rules directly within Area 2 and once they

are as desired can edit the core .htm file to include your modifications…so that they will be

present the next time you open the html file. The ones for “clientTimeCreated” and above

should be left alone.

Area 3 is where the scanned or transformed content will be displayed for each Action. The

mobile device at times (particularly when disconnected from Wi-Fi or configured to do so via

special javascript api) will send multiple actions per transaction. The header of this area will

display the total number of actions detected and allow inspecting each of the separate actions.

It will also display a “New” checkbox when content is detected within clientStoreContent.new

portion of the sync payload:

Clicking the New checkbox will display the content of the New payload and change the “Copy

Action” button to “Copy New”.

Buttons at the bottom of the form, provide the following capability:

Decode: Prior to Pega Infinity 8.1, the payload of the transactions were really application/x-

www-form-urlencoded (though the content-type header had the value as text/plain). The

Decode button is useful for decoding this encoded format so that it can be visually

inspected and transformed. Starting with Pega Infininty 8.1, the content-type of the sync

transactions is application/json (and the content is no longer encoded)

Encode: Provides a way to re-encode content that was decoded.

Transform: Transforms the content of the sync payload (in Area 1) by executing the series of

regular expression transformations specified in Area 2. The content in Area 1 is updated

with the result of this transformation. In addition, Area 3 will contain the content specific

to individual Action or the what was found within clientStoreContent.new.

Scan: Parses the payload content within Area 1 (without applying the transforms), populating

Area 2 (and the Area 2 header).

Copy All: Copies the entire contents of Area 1 to the clipboard

Copy Action: Copies the entire contents of currently selected Action or clientStoreContent.new

content displayed within Area 2 to the clipboard. Rather than trying to select the entire

contents of Area 2 and then invoking a copy keyboard syntax, this button provides a more

robust (less error-prone) way to achieve the same.

Clear: Clear the contents within the different Areas of the tool to prepare for the pasting of a

different synchronization payload transaction

When inspecting the captured transactions within Charles or Fiddler, you should notice 1 or

more transactions to pzSynchronizationService. For each such transaction:

a) Select the raw POST contents and paste it into Area 1 of the tool.

b) If the Pega server being utilized is prior to Pega Infinity 8.1, the content will be url

encoded and the “Decode” button will need to be clicked first to decode the contents.

c) Click the “Transform” button to Transform the contents and parse out the relevant

Actions array within this transactions.

d) If there are no Actions (or the only action within the transaction is a “reconciled”

action), click the “Clear” button to clear the payload contents within the tool and

proceed to paste in the next captured transaction from Charles or Fiddler within Area 1.

Once there are legitimate Actions to incorporate within the test plan, proceed to STEP 3.

STEP 3: Populate the Decomposed Actions area within the JMeter Test Plan

The Decomposed Actions component within the JMeter Test Plan is the primary portion of the

Framework Test Plan that needs to be modified to contain your application specific content.

Prior to starting this step, open the PegaSyncFramework(LoadMobile).jmx sample file using the

JMeter visual client and do a Save As to save it as a new name (effectively making a copy of the

file).

Note: The PegaSyncFramework(LoadMobile).jmx JMeter test plan was developed primarily

using JMeter 3.2, but should work fine with more recent versions. Version 3.3 or better is

recommended as it deals seamlessly with transactions that return a large response … as is

typically the case with the initial Full Sync transaction. If using Version 3.2 make sure to add the

following entry within your JMeter user.properties file:

httpsampler.max_bytes_to_store_per_request=41943040

Within the left component pane, make a duplicate of the Decomposed Actions segment you

wish to utilize (7.4 or 8.1) and enable the new copy created and disable all other Decomposed

Actions segments. (It is best to adjust the name of the original that is being disabled…perhaps

by adding “ (orig)” to the end of the Name)

The framework file is setup to support multiple “usage scenarios” and execute each one a

particular percentage of the time (for every simulated user). However, as in the LoadMobile

sample, focus on just one usage scenario initially. The first usage scenario will have variables

with the prefix “DA1” (for Decomposed Actions #1).

Clear out the current values within DA1_ACTIONXX and DA1_NEWXX variables to insure that

new relevant content from the recently transformed captured transactions is utilized (and stale

content specific to the LoadMobile sample app isn’t left behind).

For each action found within the Pega Sync Action Extractor tool (using the steps outlined at

the end of STEP 2):

a) Select each incremental Action using the Actions dropdown (start with 1 and select each

incremental one the next time this sub step is reached)

b) Click the “Copy Action” button to copy the transformed contents of this action to the

clipboard

c) Place the cursor within the next DA1_ACTIONXX variable value area and use the Paste

shortcut (Ctrl+v for Windows or Cmd+v for Mac)

d) Update the comments field for the action entry to uniquely identify this action/screen

transition (perhaps by the action value and/or the screen name this action relates to).

Each screen transition should generate a “finishAssignment” action.

After transferring all the Action contents into the JMeter test plan for a specific transformed

transaction, if the “New” checkbox is displayed between Areas 2 & 3 (it will be displayed only if

there was a “createNewAction” action within the actions array for the transaction and some

content was present within clientStoreContent.new):

a) Click the “New” checkbox to display the content of clientStoreContent.new within Area

b) Click the “Copy New” button to copy the transformed New content to the clipboard

c) Create a variable named DA1_NEWYY where YY is the same action number (conveyed by

XX of DA1_ACTIONXX). Place the cursor within the variable’s value area and use the

Paste shortcut (Ctrl+v for Windows or Cmd+v for Mac)

d) Update the comments field for the action entry if desired

(To summarize: A DA1_NEWYY variable should be created only for any DA1_ACTIONXX

variables that have a value containing a “createNewWork” action.)

Repeat the above procedures for the additional captured HTTP pzSynrchronizationService

transactions so that all the relevant actions for the scenario are incorporated into the test plan.

Enter the total number of unique non-“reconciled” actions to pzSynchronizationService that

were captured within the DA1_ACTIONCNT value. (This should be Integer value represented by

the XX for the last DA1_ACTIONXX variable with actual action payload)

The DA1_PW_ACTIONCNT integer value specifies the incremental sequence of actions to

execute when populating work within the worklist (PW stands for Populate Work). This is

relevant only if “1” is specified for the global variable “createNewWork” and “OPENCASES” is

set to a integer value greater than “0”. If relevant, it should be set to a value less than

DA1_PW_ACTIONCNT. In this manner, if there are less than the specified desired number of

OPENCASES, it will go thru and issue the actions in consecutive order but stop at this lower

action count value, so that the case would then be left in the open-assigned state (within the

current user’s worklist). This would be done for as many incremental cases are needed to reach

the desired “OPENCASES” count.

When moving captured actions to the Decomposed Actions area, remember that any actions

with the “action” property value of “reconciled” should be ignored. The logic within the test

plan will automatically generate the “reconciled” action when necessary.

STEP 4: Adjust other portions of the configurable test plan

The final step is to adjust various global server and application specific variables.

4.1 Specify proper “Server Vars” values

Enter the serverdomain, serverprotocol and serverport values for your Pega server which you

wish to load test (and has the mobile offline-enabled application deployed).

syncVersion should be set to “4” for a Pega 7.4 or earlier server and to “5” for any later servers.

There were several changes to the request payload in version 5 when compared to version 4.

The logic within the Framework test plan will simulate the proper version based on this setting.

Some of the changes include:

• MIME content-type is “application/json” (was previously application/x-www-form-

urlencoded and the POST content was URL encoded)

• Various top level synchronization attributes within the requestJSON structure moved to

be HTTP request headers

• Support for forced bundling of actions to minimize the number of transactions during

normal operation

• Support for chunking large accumulated set of actions when disconnected to be

separate transactions to avoid very, very, large transacctions

Update the WORKITEM-PREFIX, WORKITEM_pyID-PREFIX and WORKITEM-FINDPTN values to

reflect the proper values for your application. The WORKITEM-PREFIX is typically all upper case

characters, whereas the WORKITEM_pyID-PREFIX will typically be mixed case. The WORKITEM-

FINDPTN should contain a value such as:

pyCategory\":\"ASSIGN-\"\n,\"pzInsKey\":\"ASSIGN-WORKLIST OLO8KL-LOADMOBILE-WORK

The portion in yellow highlight will need to be updated to properly reflect your application.

This is used to find open assignments within the current user’s worklist. To get the proper

value look at the response to an initial full sync (when there is atleast one open work item in

the user’s worklist), and search for the portion before the yellow within the response.

useOAuth2 should be set to “1” if leveraging OAuth2 authentication, otherwise set to “0” to

leverage basic authentication. The remaining vars on this variable component are all related to

leveraging OAuth2. The OAuth2 authentication logic will be very specific to the type of identity

provider server that is being leveraged. The sample JMeter file includes an implementation

which works against a NetIQ Access Manager appliance, but will require modifications if going

against any other identity provider server.

(As the example file shows, there can be multiple Server Vars referencing different servers, but

only one should be enabled when attempting to run the test plan)

4.2 Specify proper “App Vars” component values

The appaccessgroupseg variable should be the initial application access group segment value

for the web application. The next four variables (platform, deviceid, nativeappid and

webviewid) are values found when a certificate with push notifications is utilized. These would

be found within the POST transaction which follows the login POST and specifies

pyActivity=pyRegisterPushNotificationsAndCategories. Capture this transaction using Charles

or Fiddler and extract these other POST values and update the appropriate corresponding

variable within the “App Vars” component. (Note: These four variables are not utilized if

usePushNotification is set to “1” within the “High Level Variables” component)

The OAUTHCLIENTNAME variable is posted as part of the pzGetMobileAppConfiguration POST

transaction when not configured to use OAuth2 and when not doing an initial sync

(initialClientStore is set to “0” witin the “High Level Variables” component). Extract this value

from a Charles or Fiddler trace of the initial sync.

4.3 Adjust the “High Level Variables” as necessary

Most of the global variables above are documented within the comments section of the test

plan. In most cases for Boolean variables, the values of “1” and “0” are used (rather than “true”

and “false”). Some of the significant High Level Variables include:

Var Name

Description

NUSERS

The number of concurrent users to simulate on any run

initialClientStore

Set to “1” to simulate an operator logging in to a device with

an already installed application. If set to “0”, the test plan

will need to always do an initial full sync. When set to “1”,

the “initialClientStore” component will need to be enabled, as

the vars there will be utilized to simulate the clientstore of

the installed app

logFullSyncClientStore

Set to “1” to have the script emit the values into the log

viewer so that they might be copied and pasted to populate

the clientstore variables. Once populated and up to date, this

can be set back to “0”.

LOOPCOUNT

The number of cases to process during the main “work” and

concurrent load test portion of the test plan.

OPENCASES

The number of items that should be created and left open

during the setup phase so that each operator has an expected

number of open items within their worklist. This is only

relevant if the “createNewWork” high level variable is also set

to “1”.

idleSyncInterval

Interval in ms to always issue a transaction. (0=disabled,

300000=5 min=default value). Any queued actions will be

transmitted to server if this much time expires after the last

transaction.

minimizeDelays

May be set to “1” to disable several other configured delays.

Useful for test plan development and avoids having to change

many separate variables to accomplish this.

skipFullSync

Provides a way to speed up each JMeter thread getting the

initial client store. It results in the first thread doing a full

sync and then communicating the clientstore variables to all

the subsequent threads. An additional deltasync is done to

remove any records that might be specific to the operator

assigned to the first thread and to load ones specific to the

current operator associated with the thread.

tracerSetupDelay,

tracerStartBeforeBundleTxn,

tracerStopAfterBundleTxn

debug settings useful to pause a run before and after a

particular transaction within the main load test loop to

facilitate using Designer Studio and tracer to capture the

details of what the requestor is doing for that particular

sequence of transactions.

scenarioPcts,

scenarioMinTimes

These settings allow multiple DecomposedActions (scenarios)

to be utilized and each one being utilized some percentage of

the time. The minimum time to complete each scenario can

also vary. Both these vars should have a consistent number

of comma separated values. (When multiple scenarios are

used, each variable within each additional active

“Decomposed Actions” component should begin with the

next proper integer value (e.g, “DA2” and then “DA3”))

createNewWork

The exercised case scenario includes creating a new case (as

opposed to the script only selecting and working on cases

within this user’s worklist.

gzipRequests

Set to 1 to simulate the Pega Infinity 8.1 ability to compress

posted content.

4.4 Enable the proper “Online” or “Offline” scenario and adjust variables as necessary

If the load test is meant to simulate typical server load when mobile devices have a network

connection, the “Online Scenario” component should be Enabled (and the “Offline Scenario”

component must be disabled).

If the load test is meant to simulate a scenario where the users do not have network

connectivity when performing the main usage scenario for the test, and then users establish a

network connection at the same time to synch all the completed work to the Pega server, then

the “Offline Scenario” component should be Enabled (and the “Online Scenario” component

shoud be disabled).

In most cases you likely want to simulate the “Online” (Connected) rather than the “Offline”

(Disconnected) scenario.

The variables within these two components have identical names, but the Description within

each of the components varies a bit. Prior to Pega 8.1, the bundleSize within the “Online

Scenario” is always “1”, and the setting for the “Offline Scenario” was always the largest

possible integer value (all offline generated actions would be sent together with the next HTTP

transaction).

The Offline Mobility Guidelines document references the critical APIs available to configure the

application to leverage a custom bundleSize for online and offline scenarios. See:

https://community.pega.com/knowledgebase/articles/offline-mobility-guidelines-0

The relevant APIs related to these variables are:

pega.offline.DataSync.setQueueMaxSize

pega.offline.DataSync.setQueueFlushInterval

pega.offline.DataSync.enableFlushOnLastStep

pega.offline.DataSync.setMaxNumberOfActionsPerRequest

Additional miscellaneous information about the provided framework

The framework is provided to help bootstrap your efforts to create a JMeter test plan for your

Pega offline-enabled mobile application. Feel free to experiment and modify it as you deem fit.

To understand the deeper details of how the framework works, greater working knowledge of

JMeter will be required. The framework leverages many standard JMeter components to

achieve the Pega Offline-enabled Mobile Application simulation. It leverages “JSR233

Samplers” and Groovy code for much of the implementation.

Many comments have been left within the code. Also various console output log statements

have been commented out but left so that they might be re-enabled if needed to debug

portions of the implementation.

The most significant JMeter components with Groovy code are the following:

“Request Shaping Manager” PreProcessor: establishes some key sync variables and sets up

the HTTP request properly based on whether it is version 5 (Pega Infinity 8.1) or earlier

version.

“clientStore Content Manager” PostProcessor: handles parsing the responses to every sync

transaction and manages the critical “clientStore”. It also detects any token expired errors

when OAuth2 is being utilized so that the token may be later renewed and the current HTTP

transaction be retried.

“Initialize case instance specific vars” Sampler: initializes key case-specific variables used by

the framework. A blank “Extension: Initialize case instance specific vars” Post Processor is

provided in case it becomes necessary to add and initialize various application specific

variables.

“Case Actions bundler ${dispCaseInfo}” Sampler: implements the complex logic that deals

with replacing variables within the Actions payload as well as determining how many

actions to include within each transaction.

The standard “View Results Tree” component is a great component to have enabled when

constructing and testing out the test plan. This is a great way to monitor the transactions that

are actually occurring. (Note: This component should be disabled when doing a real load test

with many users)

Additional information about the provided Sample Application (LoadMobile_74_RAP.zip)

The application includes a “UIPerfLoad” ruleset which includes some useful activities:

CloneUser – provides a way to easily duplicate an existing operator appending numeric

values to disambiguate the users. (If the operator being duplicated has a “@” in the name

the disambiguating numbers are inserted preceding the “@” character)

CloneUserCleanup – eliminate operators created via CloneUser

RemoveStressTestRequestors – kill requestors for operators that are to be used in a load

test (that begin with a common prefix)

The application archive includes the following operators (which will need to be enabled after

the application is imported):

loadmobile.author (with password “rules”)

loadmobile.user (with password “rules”)

Questions about the provided framework may be posted within the Pega Community.