Endeca Performance

Some Tips for Configuring Endeca for Optimal Production Performance with ATG

February 11, 2014 by Mark Scarton Leave a Comment

Introduction

The following tips will ensure that you are achieving optimum performance from your production Oracle Commerce Endeca environment.

Main Article

Make sure Preview isn’t enabled in production

/dyn/admin/nucleus/atg/endeca/assembler/cartridge/manager/AssemblerSettings/ : Make sure previewEnabled = false

Setting this to true will return back too much merch rule data with the results. You can confirm this by looking at the Dgraph.reqlog, and looking for merchdebug=true in the queries.

Make sure you have the latest Assembler patches

Ensure that you have the following patches applied. (These are included in version 11 of Oracle Commerce, so no patching is required on that release.)

• Patch 17342677 : This includes 2 fixes. First, it, reduces the number of supplemental objects that are returned with the queries, which helps performance. This is for ATG 10.1.2 and 10.2, but really it’s for Assembler 3.1.1 and 3.1.2.
• Second, it fixes an XML Parser locking problem that can be seen during performance testing.

• To verify that the patch was correctly installed, make sure the request logs start including the following with the requests: &merchrulefilter=endeca.internal.nonexistent

This patch should be considered critical and always applied.

Check the properties being returned by Endeca

In Assembler, you can select which properties are returned back with the search results.

If this is not configured, then all of the properties that are configured by Developer Studio will be returned (and this can be large).

If you have incorrectly spelled the names of properties, then an error message will be written to the Dgraph.log. This can cause a slight IO hit by the MDEX, and fill up that log very quickly.

This (by default) can be found at: /atg/endeca/assembler/cartridge/handler/config/ResultsListConfig.properties (by default, this is in /Store/Endeca/Assembler/config/config.jar)

Make sure your Assembler logging is configured correctly

Assembler will create an Endeca LogEntry for each page view and send that message to the Endeca logging server. This happens asynchronously, and the Endeca logging API will spawn a separate thread to handle the communication with the logging server.

If this is configured incorrectly, that separate thread will spawn, try, wait and fail to connect to the logging server. It will eventually go away, but they will build up in memory. If you are doing heap dumps and see problems for com.endeca.logging.LogEntry or LogConnection, this is probably the problem.

For Assembler, you would configure this as an assemblerEventListener on the NucleusAssemblerFactory configuration ( /atg/endeca/assembler/NucleusAssemblerFactory in /dyn/admin ).

Remove any comments from your Experience Manager cartridges and templates

In the XML files that you create for your cartridges and templates in Experience Manager, remove all of the XML comments. Especially remove any Oracle disclaimer text from the top. These comments are returned in the response back from the MDEX Engine and will increase the response size. (In the Discover Electronics application, this text will increase the response size by around 10%).

NOTE: The Experience Manager tool (XM) does not add these “copyright” comments. Any content item (or page) that is edited and saved by XM will not have any comments, so any page or content saved by the tool will not have this issue. The XML comments are only introduced to the system by importing initial content for Discover Electronics. The XML comments were added by hand outside of the tool, and they were only added to sample configuration files. Hopefully, you won’t run your application with this sample configuration.

Check your network latency

Depending on the number of cartridges (and the type of those cartridges) ATG might have to make a number of calls to the MDEX Engine. If the network isn’t fast, this can introduce latency into the application even if the MDEX Engine has to do very little processing.

Make sure the load balancer is working correctly and that the network connections are as fast as possible. If possible, use a network packet sniffer or a tool like wireshark to trace the round trip packets from ATG to your MDEX instances so that you can verify that the network incurred latency has been minimized.

Check your URL Patterns to ensure non-Endeca URLs don’t hit Assembler

http://docs.oracle.com/cd/E41069_01/Platform.11-0/ATGEndecaIntegrationGuide/html/s0706assemblerpipelineservlet01.html

In ATG, there is an AssemblerPipelineServlet. This will read in URLs and process them through Assembler. If it’s not correctly configured, too many URLs might flow through Assembler for no reason. We’ve seen this at several customer sites, where their ATG application was serving up the product images (as opposed to them being hosted externally at Akami or something). They had their URLs incorrectly configured, and for every image being served up, a useless call was being made to Assembler.

The “/atg/endeca/assembler/AssemblerPipelineServlet.ignoreRequestURIPattern” is the property to configure.

Records Per Aggregate Record set to all

Quick background: In Endeca, you can have SKU-level records that are aggregated together into an Aggregate record. You can choose how to bring back the SKU level information. Either 0 SKU’s, 1, or all. By default, this setting is set to bring back All SKU’s in an aggregate.

This isn’t necessary and creates a larger response sizes from Endeca and uses more memory in ATG.

This can be configured in the ResultsListHandler.

All Dimensions Compute Refinement Counts

By default, all dimensions generated via ATG schema are set to compute refinement statistics. You should turn this off if the site does not display them. This can be done by adding the dimensions to the schema.csv in the config/api_input directory and setting the attribute.dimension.compute_refinement_counts to false. You can also globally disable the refinement counts by setting the refineShowRecordCounts property to false in the dimensionTemplate bean in the fcm.context.xml file.

This will save processing time in the MDEX Engine, but won’t really affect response sizes or memory usage by ATG.

Ensure that custom handlers are scoped as Prototype

Incorrectly scoped handlers can cause ConcurrentModificationExceptions. If you should attempt to create/configure an ATG Nucleus Component based on an Endeca class/bean, you will need to set the $scope for that Component to prototype. The Assembler object, created via /atg/endeca/assembler/AssemblerTools, retrieves Objects from ECR|IFCR|Spring. With scope set to global, Assembler is essentially retrieving the specific Object from Spring, which begins to explain why multiple threads are sharing the sameobject. With scope set to prototype, Assembler is retrieving a copy of the Object or an iteration of the Object, which is not shared across multiple threads. EXAMPLE

./localconfig/foobar/endeca/assembler/custom/cartridge/handler/config/ContentSlotListConfig.properties

$class=com.endeca.infront.cartridge.ContentSlotListConfig
$scope=prototype

To clarify a little, the out-of-the-box Commerce Reference Store cartridge handler default configuration components happen to function correctly when configured with global scope because their cartridge handlers don’t attempt to modify them in any way. This is not true of the ContentSlotHandler (or of the cartridge handler contract in general). Thus the recommendation is to always use prototype scope for default configuration objects. Doing so, although conservative for some handlers, makes it easy to ensure correctness. The sample Spring files supplied with the Endeca v11.0.0 release conforms to this recommendation.