Customizing WebCenter Logging with WLST

Like many enterprises, the one I’m currently working with only provides developers with the Monitor role in production. This requires scripting tasks that are well-documented for manually performing but take some real thinking and testing to get right as a script.

One of my team members frequently says that code should be self-documenting, so rather a long blog entry I thought I would simply post the script along with some minimal comments and see if everyone finds that useful:

def configureAdfLogHandlers():
#The full script has multiple methods, so I print the currently running method to follow along
	print 'Start configureAdfLogHandlers()'
	managedServers=cmo.getServers()
	for managedServer in managedServers:
		sname=managedServer.getName()
		path = '/Servers/' + sname
		cd(path) 
		lh = listLogHandlers(returnMap='true')
		for l in lh:
			lname = l.get('name')
#Even with this script, the odl logs are quite verbose and we run an automated arciving process. 
#To simplify the process, we just put the logs in the archive path since the odl log handlers don't provide an archive location option
			odlfile = '${domain.home}/logs/LOGS/${weblogic.Name}-' + lname + '.log'
			print '%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%'
			print '   Setting log handler configuration for ' + lname
			print '%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%'
#The custom code rarely uses any levels between debug and warning, and the notice level for Oracle code is too verbose for most trouble-shooting needs
			configureLogHandler(target=sname,name=lname,path=odlfile,rotationFrequency='daily',retentionPeriod='day',maxLogSize='1g',maxFileSize='100m',level='WARNING:7')
		print '%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%'
		print '   Setting logger configuration for ' + sname
		print '%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%'
#The custom code used by the application sets of warnings in these packages that are not issues for the application, so we set the level to ERROR here
		setLogLevel(logger='oracle.webcenter.page',level='ERROR:1',target=sname,persist='1',addLogger='1')
		setLogLevel(logger='oracle.webcenter.framework.service.Utility',level='ERROR:1',target=sname,persist='1',addLogger='1')
		setLogLevel(logger='org.apache.myfaces.trinidadinternal.skin.SkinCSSDocumentHandler',level='ERROR:1',target=sname,persist='1',addLogger='1')

connect(os.getenv('WLS_USER'),os.getenv('WLS_PW'),os.getenv('ADMIN_T3'))
configureAdfLogHandlers()
disconnect()
exit()

 

If you found this interesting, please share.

© Scott S. Nelson

Speeding Up WebLogic Server Console First Access on Linux

It has been posted many times, but since I ran across it again and it made a significant improvement, I’ll repeat it for others.

Add the following to your WLS start up arguements:

-Djava.security.egd=file:/dev/./urandom

If you are interested in details, see https://bugs.openjdk.java.net/browse/JDK-6202721

If you found this interesting, please share.

© Scott S. Nelson

Fixing Silent Domain Builds with WebCenter Portal

Automating domain creation is important for enterprises to minimize maintenance cost and be prepared for efficient disaster recovery. One enterprise I worked with was struggling with the domains they were building through scripts as the Enterprise Manager was not working properly in the completed domains and the managed servers would come up in Admin mode.

After comparing the scripted domain to a manually created domain that worked correctly it was found that the paths for the libraries did not match. The paths in config.xml were missing the beginning of the path where the libraries were actually located. While the installation paths chosen were not the default paths, this did not entirely explain the issue, and internal processes made the custom paths highly desirable.

The correction for this issue was to add a function to the offline WLST script run on completion of the domain creation to correct the library paths:

def fixApplicationSourcePaths():
  appPathPrefix = os.getenv('DOMAIN_HOME')+'/applications/'
  cd('/AppDeployments/')
  deployed_apps = ls(returnMap='true')
  for app in deployed_apps:
    cd('/AppDeployments/'+app)
    apppath = get('SourcePath')
    if apppath.find('/app/')!=0:
      apppath = apppath.split('/')
      apppath = apppath[-1]
      set('SourcePath',appPathPrefix+apppath)
  cd('/Libraries/')
  deployed_libs = ls(returnMap='true')
  for lib in deployed_libs:
    cd('/Libraries/'+lib)
    apppath = get('SourcePath')
    if apppath.find('/app/')!=0:
      set('SourcePath',appPathPrefix+apppath)

With this change, domains were created consistently as desired.

If you found this interesting, please share.

© Scott S. Nelson

Correcting WLS Console Hang with SSH Node Manager Environment

I was having a frustrating situation yesterday where the WLS console would just hang after clicking the Servers link. Naturally I go to the Unix console to tail the logs and see what’s what, and these prompts for a password keep popping up. Entering the admin password is no help.

So I went to the next natural step of rebuilding the domain (the build is scripted, and yes, I backed up the domain first) and the problem is still there!

An email to a colleague more steeped in the dark arts of server administration than I results in the suggestion that this is due to a problem with SSH (the domain uses SSH Node Manager) and tells me test it with:

ssh username@hostname

to confirm connectivity and permissions.  What I get back is:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Permissions 0755 for '/app/home/username/.ssh/id_rsa' are too open.

It is required that your private key files are NOT accessible by others.

This private key will be ignored.

bad permissions: ignore key: /app/home/username/.ssh/id_rsa

Password:

With this new detail in mind, I went to an environment that was working, found it had the file permission set at 600, and preforming

chmod 600 /app/home/username/.ssh/id_rsa

Got me up and running again

If you found this interesting, please share.

© Scott S. Nelson

Fix for InvalidRegistrationException with a WLP Producer and WebCenter Portal Consumer

I have come across a case where I have deployed a WebCenter Portal EAR to a newly created domain that was copied in from another domain. When attempting to update the WLP producer registration with the new environment, I received a com.bea.wsrp.faults.InvalidRegistrationException.

I had done this sort of new domain configuration many times, with the only difference here being that the EAR I was deploying was copied from a running domain. While I don’t know the actual root cause of why it had a problem this time, I suspect that OEM is writing back to the EAR for some reason. Anyways…

After much head-banging, and successfully registering other domains to the same producer in a vain attempt to prove I was not insane, I re-read the error for the umpteenth time and finally realized what was causing the exception. Do you see it?:

<BEA-000000> <<Jun 16, 2014 3:08:28 PM PDT> <Error> <oracle.portlet.client.connection.wsrp.HTTPClientTransport> <WCS-40152> <A request to the producer URL "http://myhost5:8001/myAccount/producer/wsrp-2.0/markup" resulted in a status 500 response with fault string "The given registrationHandle [22001] is invalid.". The fault code given was "{urn:oasis:names:tc:wsrp:v2:types}InvalidRegistration". The producer generated a timestamp of 2014-06-16T15:08:28.584-07:00 and associated the following stack trace with the fault message: com.bea.wsrp.faults.InvalidRegistrationException: The given registrationHandle [22001] is invalid.

So, since it was trying to register for the first time with a handle that had not been created yet, it made sense that it would fail. The WLP schema is a bit too complex to allow just creating the one row, so instead I went back to JDeveloper, updated the registration handle there and built a new EAR. The new EAR had a new value for MDS so that when I tried for the last time to update the producer it decided to request a handle rather than insist it already had one.

Added May 20, 2015:

This still puzzled me, though. I ran across the same issue again and realized where the problem originates. The migration process being followed was to always deploy the same EAR successfully tested in a lower environment. When automated, this EAR comes from the build server. When done manually (and pressed for time) it is taken directly from the lower environment. For existing environments this has never been an issue. But it was an issue for new environments. So when it happened again, I undeployed the EAR copied from the lower environment and deployed the EAR from the build server. The error went away.

It seems that something is written back to the EAR after deployment. Not what I would expect, but at least I know that in the future it is best to always deploy the EAR from the build server rather than one that has been pushed from the build server to another environment.

If you found this interesting, please share.

© Scott S. Nelson