Determine the problem

  1. In the console, navigate to Symphony Workload > Monitor Workload. There are three types of errors to look for to quickly narrow down which compute host experienced service problems:
    • Service instance startup failures

      1. Look for your application in the application list.

      2. Look for service instance hostname and process ID pairs in the SI Startup Failures column for your application. If there are any service instance hostname and process pairs, the service application experienced problems in either the Register or CreateService method on each of the service instances listed in the column.

        If none of your service instances can start successfully, no service instance is available to run any of your tasks. If your client is hung waiting for task results, this may be the reason.

    • Binding failures

      1. Click on your application name to drill into more details about your workload.

      2. Look for service instance hostname and process ID pairs in the Binding Failures column for your session. If there are any service instance hostname and process ID pairs, the service application experienced problems in the SessionEnter method.

    • Task failures

      1. Navigate to the Sessions page. You can see whether any tasks ended in the ERROR state. It is possible your service experienced an error, but was unable to rerun and finish successfully.

      2. Click on a session to drill into more details about that session. Look for comments in the Failure Reason column of your Tasks table.

      3. Find out the host and process ID for the service instance on which the task ran. By default, the hostname is displayed in the Tasks table. You can configure the process ID to be displayed by modifying the Instance ID in Preferences at the bottom of the Tasks table.

    Make note of one hostname and process ID to troubleshoot a particular service instance to see what happened in the next steps.

  2. Narrow down the detailed reason why service startup failed.

    In Symphony DE, look at the service instance manager logs for the application on the host for more information. For example:

    %SOAM_HOME%\logs\sim.hostname.appName.log_file_number.log

    In Symphony, if you trap SNMP events, you receive event notifications for the service errors that occur.

    You can also use log retrieval to retrieve the service instance manager logs on the host where your service failed to start, as follows:

    1. Navigate to the Resources > Hosts (List View) tab in the console.
    2. Click on the host where service startup failures occurred.

      A dialog displays.

    3. Click on the Host Logs tab in the dialog.
    4. Check sim.log.
    5. Optional. Check User specified log or file if your service generates its own log files and you want to retrieve them.

      A text box appears.

      In the text box, type the file pattern to retrieve service logs for your application.

    6. Click on Retrieve Log List.
    From the log information, you can determine the following:
    • Which method caused the error (i.e. CreateService, Invoke, and so on)

    • Roughly what caused the error (process exit, FatalException, FailureException, unexpected exception, method timeout)

    • Where the service event replay log file is located

    The service instance manager reports something similar to the following example:

    2007-11-14 12:30:43.843 Eastern Standard Time ERROR [3100:5188] sim.backend.ServiceBroker - Code[S20070]: e:\symphonyde\de40\4.0\src\soamservicecontainer.cpp : 137 IException  Domain <Application>: Unexpected service exception in method onCreateService(). If the additional details attached do not provide enough information, change your code to use SoamException. Additional Details: Unexpected Exception Caught in onCreateService()..
    2007-11-14 12:30:43.843 Eastern Standard Time ERROR [3100:5188] soam.common.EventAgent - Code[S75052]: Application <ServiceReplayDebuggerCPP>, service <DebugService>: Failure exception thrown on method <createService>, control code <0>. Action on service instance: action <blockHost> taken on service instance process <5672>, host <achin2>, service <DebugService>.
    2007-11-14 12:30:43.890 Eastern Standard Time WARN [3100:5188] soam.common.EventAgent - Code[S75064]: Application <FailureException with control code 0>: SIM detected <FailureException with control code 0> for method <createService>.  Debug action is <writeServiceEventReplayFiles>. Serl file is <E:\SymphonyDE\DE40\work\serl\ServiceReplayDebuggerCPP\DebugService\ServiceReplayDebuggerCPP.DebugService.achin2.5672.2007-11-14.12h.30m.43s.859ms.serl>.
Related concepts
About debugging a service
Related reference
Service replay debugging