Experiences with SOAP, HTTPS and Ohio SIF Agents.

I'm posting this as a blog entry until I hear for sure how/if Edustructures is going to fix the Agent and Console configuration process.  When I know for sure, then I'll transfer this into the Wiki.

A number of ITC's have had trouble configuring the Ohio USAS and USPS SIF Agents to use HTTPS for communicating with the USPS and USAS SOAP Services.  I had this working a few months ago, but when I went back to reconfigure NWOCA'S test environment, I couldn't get it to work.

Here are a few things I learned (or re-learned) while getting this to work again:

  1. The SIF agents do not use the standard Java certificate truststore that installs with the JRE (this is basically the same list of CA's your web browser would trust). Rather, they use only the "trustStore" that is configured in the agent.cfg.  This means that even if your SOAP server has a signed certificate you must load the certificate (or the cert of the signing CA) into the Agent's truststore.  I consider this a problem with the Agent but I'm not sure yet if Edustructures does.
  2. The current version of the Ohio SIF Agents (1.0.0.28) do a better job of trapping connection and SOAP errors (see SIF Trouble Shooting).  The agent's now report errors to the ZIS rather returning an empty response and exceptions are logged to the Agent's console.  But not all information from the SOAP failure is logged to the agent's console or the Agent.log.  This makes it difficult to determine exactly why the SOAP connection is failing.  You might be dealing with an SSL certificate problem or other problems like authentication errors, SETUPENV problems, etc.  
  3. The Agent GUI Console may not use the same directory as the Agent when using the Console's Transport->Certificate Manager. I found that if I launched the Console GUI by double-clicking the RunConsole.jar, that the current working directory would be my "home" directory, not the directory where the jar lives. But the running Agent uses it's install directory as the working directory.  This means that changes made to "Trusted.ks" by the Console's Certificate Manager could be placed in the wrong directory.  So even though it appeared I had loaded my SSL certificate into the Agent, the Agent was not using the same file and couldn't connect to the SOAP service.  You can avoid this by running the Console from the command line from the install directory, or use abolute paths (see below).

Recommendations

Based on the above, here are my current recomendations for configuring and debugging connections between the SIF Agents and USPS/USAS SOAP agents.

Don't Trust the Agent.log

When debugging the Agent, run the agent from a command window and not in background.  In my experience, the console log from the agent contains more/better information than the Agent.log, especially for errors related to SOAP connections.  Running the agent from the command line is easy just do:

    cd {agent-install-directory} 
    java -jar RunAgentStandalone.jar 

Use an absolute path for Trusted.ks

If you use an abosolute path for the trust store, then you can share the store with all agents on the same machine and you can avoid the Console's working directory problem mentioned above.  Do to this, create a directory to contain the keystores (outside the SIF Agent's directories), like "/data/SIFAgents/keystores" or c:\sif\keystores.    If you have a Trusted.ks already built with your certs, then copy it to this directory.

Next, edit the agent.cfg for each SIF Agent and change the line:

     <property value="Trusted.ks" name="trustStore"></property>

To contain the full path to the file:

     <property name="trustStore" value="/data/SIFAgents/keystores/Trusted.ks"></property> 

You can do the above with the GUI console under Agent Settings->Transports if you prefer. Stop and restart the agents.

With this done, all the Agents and Consoles on the same machine will share the same keystore and you can use any of the Agent Consoles to import SSL certificates.

Importing Certificates

Once you have your shared Trusted.ks configured, then you must import your SOAP server's certificate using an Agent Console's Certificate Manager (Agent Settings->Transports->Certificate Manager). When importing the certificate, be sure to import it as a "Server Certificate" (not an Agent Certificate).  The "Server Certificate" is the same as the Trusted.ks store.

You have three choices when picking which certificate to import depending on what type of certificate is on your SOAP server:

  • You can load the server's public key certificate directly. This is the only choice for unsigned certificates but also works for signed certs. 
  • If the certificate is signed by a trusted CA, you can import the certificate of the signing CA. 
  • If your ITC runs it's own local CA, you can load the certificate of the local CA.

The advantage of loading the CA's cert is you only need to do it for once for all certs signed by the same CA.

BTW, the easiest way to get a server's certificate or signing CA cert is to use Firefox. Just browse to your web server, inspect certificate (double-click the "Lock" icon in the status bar), view the certitificate details. Select the server certificate or the signing CA cert and then use the Export button to export it as an X.509 certificate.