This page applies to on-premise deployments.

This folio provides troubleshooting data for issues that may arise during Controller installation and operation.

Controller Server Log

The primary log file for the Controller at the following location:

<controller_home>/logs/server.log

The offset stepin troubleshooting Controller problems typically involves checking the log file. Search the log for errors that may correspond to the consequence you lot are encountering. If found, an fault log may assistance you place and resolve the issue.

Too, run across installation troubleshooting data in Custom Install.

Place Controller Performance Issues

The following are indications of Controller functioning problems:

  1. The Controller UI performs slowly. For brusque time ranges, such as 15 or 30 minutes, responses that take longer than ten to xx seconds can signal that your Controller is under stress.
  2. When the Controller's metric reporting lags vii to 10 minutes behind the current time, it can exist an indication that your Controller is under stress. A lag of about 3 to five minutes is normal.
  3. When monitoring the Controller environment, you run into that CPU, memory, and deejay metrics are at well-nigh 75% capacity.

If you lot observe deposition in Controller performance, it may be due to one of the following:

  • The hardware resource for the Controller might not match the correct Controller profile.
  • The Controller performance contour may be incorrectly configured.
To troubleshoot Controller functioning problems:
  1. Ostend that the hardware matches the Controller contour you use. For details see Controller System Requirements.
  2. Ostend that your disk functioning matches the recommended thresholds for minimum disk performance. For details encounter Controller Organisation Requirements.
  3. Confirm that the Java SDK version is exactly the aforementioned every bit the Java version on the Controller. To display the version of Java used by the Controller:
    • Open the control-line utility.
    • Become to <Controller_Installation_Directory>/jre/bin
    • Run coffee -version.

Monitor heap usage

  • On Windows, use the Task Managing director to measure the memory usage for the Controller.

  • On Linux, use thetop control to go statistics for the memory data.

                    ps -elf (expect to see a "java" process and a "mysql" procedure)  elevation (expect to run into java and mysql with cpu greater then 0)

    BASH

Timeout errors during Controller installation

While installing the Controller, the Enterprise Console attempts to starting time up the Controller application server and database. At first database startup, the application attempts to create the database schema, tables, and other artifacts needed by the Controller.

Past default, the Enterprise Console waits 45 minutes for the Controller app server or database to showtime. When installing a medium or large profile Controller or into certain types of environments such as virtual machines, the time information technology takes to get-go up the system can exceed the default startup timeout period.

Controller does not get-go properly on Windows

Your Controller may not be starting due to file extensions of transaction logs created by Glassfish. Excluding the Controller information directory from beingness scanned by virus scanners equally specified on Fix Windows for the Controller does not account for these extent files establish in the <AppDynamicsInstall>\Controller\appserver\glassfish\domains\domain1\logs\server\tx directory. When your antivirus detects these extensions, such as WRY, information technology may mistakenly terminate the process of using these files and then the Controller ultimately does non start.

These transaction logs are used to recover any failed Glassfish transactions, so deleting these logs on startup is not advised. Instead, configure your virus scanners to ignore the unabridged Controller directory.

No information in the Metrics Browser

This may betoken that the agents are not correctly configured. Begin troubleshooting past looking at the server.log file.

All log files for Controller are located in the <Controller_Installation_Directory>/logs folder.

Error Bulletin

Solution

Mistake receiving metrics (node non properly modeled yet: Could non find component for node.

This fault means the app agent tried to upload metric information for a specific node, but the node does not vest to whatever tier. Nodes must belong to tiers and these tiers must belong to a business concern awarding in order to receive metric data for that node. See Overview of Application Monitoring.

Received Metric Registration asking for a machine that is NOT registered to whatever nodes. Sending dorsum zero!

This error indicates that the Controller received a registration request for metrics for a Auto Amanuensis that listed a machine ID non yet associated with any node. Configure the Machine Agent to associate with the right application, tier, and node. See Install the Machine Agent.
Agent upload blocked, as its reporting a time well into the future.

The App Agents attempt to study metric data using Controller time. The agents retrieve the time from the Controller every five minutes and report times using a skew of the local auto time, if different.

If for some reason the App Agent reports metrics that are fourth dimension-stamped ahead of the Controller time, the Controller rejects the metrics. To avoid this event, ensure that the system times for the machine on which the Controller is running and the machines for the app agents are in synchronization.

Controller shutdown does not increase gratis retentivity on Linux

You lot practice not mostly demand to be concerned about the "complimentary memory" value, equally it will always tendency towards cipher. The Linux kernel tries to continue its cache as large as possible. As a result, the Linux kernel does not release the retentiveness even later on process termination. The memory is freed only if it is required by another process.

Controller process unexpectedly close down

On Linux, memory allocation failures may crusade the Controller process to be shut down unexpectedly by the Linux Out-of-Retention (OOM) Killer. The Controller log, server.log, does not provide information about the shutdown. Instead, to diagnose this outcome, check the system log (normally /var/log/messages) for "out of memory" entries written by the OOM killer, for example, as follows: 

grep -i "Out of memory" /var/log/letters

If you encounter this log entry, brand sure that you have allocated sufficient bandy space on the Controller machine. AppDynamics recommends allocating a minimum of 10 GB of swap infinite.

Controller server swapping too oft

If you run across unexpected swapping on the Controller machine, you can configure how aggressive the operating system swaps by configuring the swappiness parameter. The swappiness parameter controls how oft the Linux kernel moves processes out of concrete memory and onto the bandy disk.   The default value for the parameter is normally 60.  When you decrease the value, you lower the tendency of the operating system to swap.  This results in less default file caching.

Run into the documentation for your Linux distribution for recommendations on the value for theswappiness parameter. For case, RedHat recommends setting swappiness to 10 for CentOS and RedHat kernels version ii.vi.32-303 or later if you encounter OOM issues even though bandy space is still bachelor.

Before you configure theswappiness parameter though, ensure that the car has sufficient RAM and that the buffer pool size for MySQL is properly configured.

To configure swappiness

  1. Bank check the current value forswappiness.

                    /sbin/sysctl -a | grep swappiness

    BASH

  2. Set theswappiness parameter.

    For example, add the following line to set theswappiness parameter to 10.

                    echo 10 > /proc/sys/vm/swappiness

    BASH

  3. Prepare theswappiness parameter in the/etc/sysctl.conf file to the same value you used in footstep two.

    For example, add together the post-obit line to the /etc/sysct1.conf file:

Could not determine the IP address of this host error during installation

During the installation process, the Enterprise Console attempts to ping the Controller by the hostname or IP accost you lot enter. If the ping is unsuccessful during the user input validation, the post-obit fault message appears: "Could not determine the IP accost of this host. Please ensure that the IP address of the Controller host resolves to its hostname or to localhost. Y'all may need to add an entry in the hosts file on the Controller host and retry the operation."

To brand the hostname resolvable, add together an entry for information technology to the hosts file on the machine on which you lot are installing the Controller.On Linux, the hosts file is typically at /etc/hosts. On Windows, look for the file at the following location, C:\Windows\System32\Drivers\etc\hosts, or the location appropriate for your version of Windows.

Add together the entry in the form of the post-obit example: 

127.0.0.i localhost myhostname

Use the IP accost and hostnames appropriate for your arrangement.

For case, the following shows the entry added as the third line of the default RedHat hosts file:

127.0.0.1    localhost.localdomain localhost ::1          localhost6.localdomain6 localhost6 198.51.100.2 myhost myhost.example.org

Controller Cannot Connect to the MySQL Database

The following exception message in server.log file indicates that the Controller cannot connect to its embedded database.

*Server log exception:* "Caused past: java.cyberspace.ConnectException: Connection refused"

If you encounter this error, verify that the Controller database is running properly. On Linux, you tin do then using ane of the following commands:

Linux

Windows

Clarification

lsof -i:3388

SysInternals Process Explorer, will provide a list of files opened by procedure with pid 3388.

List open files opened by process with pid 3388.

netstat -anp | grep 3388

netstat -ano | find "3388"

List all networking ports opened past procedure with pid 3388.

ps -aef | grep mysql

tasklist /5 | observe "mysql"

Lists all processes and then checks if the procedure with name "mysql" is active and alive.

If no processes are establish, information technology indicates that the Controller database was incorrectly terminated. Beginning the Controller database once again and verify the Controller server.log file for any error letters.

Stack overflow exception when installing the Controller installation on Windows

This exception is ordinarily caused when you prepare the -Xss option to a lower value. We recommend changing this value to 96000.

Triggering automatic collection of Controller logs

Utilise the following console commands to trigger automatic capture of Controller log files:

  • On Linux, run:

    bin/platform-admin.sh submit-job --platform-proper noun test --service controller --job retrieve-log

  • On Windows, open an elevated command prompt (in the Windows start menu, right-click the Command Prompt icon and cull Run equally Administrator) and run:

    bin/platform-admin.exe cli submit-job --platform-proper name test --service controller --task call back-log

    The logs will exist copied in the Enterprise Panel host under platform-admin/ logs-controller-<platform-proper noun>-<date-time-postage stamp>.nothing.

Collecting Troubleshooting Information for the Controller

If opening a support case for Controller troubleshooting, y'all can facilitate the diagnosis of the trouble past providing the following information:

  • Submit all platform-admin/logs/* and platform-admin/logs-controller-*.aught, in detail the server.log files. You tin also utilise the log file utility described in Triggering automated collection of Controller logs to collect logs.
  • If the Controller runs out of memory, it generates a heap dump. Submit all files in <controller_home>/appserver/glassfish/domains/domain1/config/hprof.
  • Submit all <controller_home>/appserver/glassfish/domains/domain1/config/gc.log files.
  • Submit information about the hardware and operating system configuration of the machine that is currently hosting the Controller, including operating system, bit version, CPU cores, clock speed, disk configuration, and RAM.

  • Point the Performance profile of Controller. Run the controller diagnosis command which captures the information in platform-admin-server.log: 

                      bin/platform-admin.sh submit-job --platform-name <platform_name> --job diagnosis --service controller

    Bash 

                      bin/platform-admin.exe cli submit-job --platform-name <platform_name> --chore diagnosis --service controller

    Bash

    Refer to the Controller diagnostic information in the platform-admin-server.log. Run across a sample Controller diagnostic data on Manage a Loftier Availability Deployment page.

When the Controller upgrade is complete, audit reports may not work immediately. The inspect database tabular array is getting migrated just after the upgrade process and the migration takes at to the lowest degree an hour to complete. If inspect reports are run earlier completing the migration process, inspect table migration messages are logged in the server.log file.

No deportment are required, attempt running the audit reports again after an hr.