Let me start by defining the current problem-detection process in most enterprises. An issue arises in the production environment for a J2EE application running on WebLogic, information is captured via logs, and WebLogic server is rebooted. From this point, application developers and administrators are tasked with analysis of the information and finding the root cause.
If they are skilled and fortunate enough they will have all of the information. However in most situations this isn't the outcome. You rarely find the details needed for root-cause analysis from production failures. You frequently have to reproduce the problem in a test environment to be able to get the proper data to determine "root cause." You get the picture: all of this can require significant time and effort.
After facing these problems in the field for many years as a WebLogic consultant and finding available tools lacking in this area, I decided to do something about it. Arcturus (www.arcturustech.com) AutoPilot for WebLogic is the result of that decision.
AutoPilot addresses these issues with a unique approach. AutoPilot is an expert system-based tool with many person years of expert knowledge embedded within. AutoPilot automates extremely tedious tasks such as Proactive Monitoring, Instant Root Cause Analysis, and WebLogic Tuning, WebLogic environment review for best practices. AutoPilot results in significant savings in terms of man hours and hardware/software licenses.
AutoPilot proactively monitors WebLogic using its IntelliCheck Technology for patterns that can cause an outage and warns administrators before they actually do. AutoPilot doesn't require any instrumentation to discover unusual patterns. AutoPilot uses information already available in WebLogic and has extremely low overhead.
AutoPilot completely automates the process of root-cause analysis. AutoPilot automatically detects WebLogic failure conditions in production and auto-analyzes WebLogic state (thousands of matrices) to generate a root-cause analysis report instantly. Additionally, AutoPilot will generate an e-mail that not only tells you that WebLogic server failed, but that also tells you why.
AutoPilot is an all JAVA and J2EE solution that plugs right into and seamlessly coexists with BEA WebLogic Server and Console (see Figure 1). You can get an evaluation version of AutoPilot at http://support.arcturustech.com/downloadpage.do.
The installation process for AutoPilot is straightforward. The AutoPilot installation wizard installs AutoPilot in fewer than 10 minutes. You can obtain more details on AutoPilot installation at http://support.arcturustech.com/APHelp/installingautopilot.htm.
Although WebLogic does provide all JMX information, it is not in an easy-to-use format. AutoPilot provides an easy-to-use interface for browsing MBeans information. With AutoPilot explorer, views are available for exploring configuration and run-time MBeans properties. AutoPilot explorers make JMX information accessible directly within WebLogic Console, without causing you to have to browse through many JSP pages to obtain the same.
One of the features that I missed with WebLogic was the lack of persistence capabilities for monitoring data. You do get a nice chart of heap and queue size in WebLogic, but once the information falls off the screen it is gone forever. What that means is if you are not in front of the Console when something bad happens, you are out of luck. AutoPilot automatically persists key information about WebLogic state for historical analysis. It provides WebLogic Administrators with extremely easy-to-use data-mining capabilities. For example, you can start with a 30-day view of WebLogic server's performance and drill down to a particular time frame in a matter of a few clicks. AutoPilot Analyzers help you spot problems & trends within the WebLogic Server (see Figure 2).
AutoPilot allows you to create sensors on any JMX property. A sensor is itself a JMX property whose value is persisted every time AutoPilot takes the pulse of WebLogic Server. AutoPilot provides an easy-to-use interface to graphically analyze data accumulated by AutoPilot sensors.
One of the unique and very useful features of AutoPilot is Blackbox. AutoPilot Blackbox (Figure 5) allows you to take a snapshot of the WebLogic Server state at any point in time. Blackbox is the view of the internal state of WebLogic Server and the applications running on it. Blackbox saves the state of the server at the time when it was created. Blackbox data can be analyzed with an easy-to-use explorer view, similar to the one used by AutoPilot Config and Run-Time MBeans Explorers. You can create a Blackbox at any time just with the click of a button. It comes in handy when you are in the middle of an analysis and want to preserve the state.
From my experience, it is much easier to find the root cause of the problem if you see it happening while you are there. Unfortunately however, you can't have an expert sitting in front of WebLogic Console waiting for a problem to occur. AutoPilot solves this problem. A key feature of AutoPilot is the auto-detection of failure conditions in WebLogic Server. AutoPilot automatically detects when WebLogic Server gets into a bad state. Once such a condition is detected, AutoPilot automatically creates a Blackbox for WebLogic. In most cases once WebLogic gets into this bad state, administrators bounce the faulty instance of WebLogic Server quickly to bring the production environment back to the desired capacity. Hence even if an expert is available to analyze the server and find the cause for the issue, the person doesn't have enough time for troubleshooting. Blackbox provides these experts with the complete state of the server at that instance, even after the server is brought down. AutoPilot just doesn't stop there. It auto analyzes the state of the WebLogic Server to generate a "root cause" report that gets e-mailed to the administrator(s) automatically.
In this way experts get all of the information they need even though the incident may have happened in the middle of the night while they were catching up on their sleep. AutoPilot can analyze thousands of matrices almost instantly, versus an expert who would take significant time to analyze the same. AutoPilot drills down to transaction level in determining the root cause and identifies the culprit transaction(s). This is what makes AutoPilot for WebLogic an extremely useful tool.
The AutoPilot Detector that determines the root cause can also be manually executed to analyze the run-time state of WebLogic Servers at any point in time. It is extremely efficient and it's the least intrusive way of finding out the problematic patterns developing in your environment. AutoPilot detector doesn't make any changes to the environment and hence can be safely executed in a production environment.
AutoPilot adds sought-after self-detection capabilities to WebLogic 8.1. Using its IntelliCheck Technology, AutoPilot monitors and analyzes internal WebLogic state to detect conditions that can lead to an outage. Once such a condition is detected, AutoPilot raises alerts for administrators.
As a WebLogic consultant, every customer I visited asked me to review their environment and guide the on where there is room for improvement. It is a time-consuming task to certify that there are no issues with the environment and advise about best practices. With AutoPilot Advisor we have automated this task. AutoPilot Advisor analyzes your whole WebLogic environment and generates a report of recommendations. AutoPilot Advisor tells you where you are not leveraging your WebLogic right and guides you on how to resolve those issues. Advisor also guides by revealing best practices and where in your environment you can benefit from them. This is equivalent to having an expert evaluate your environment and provide feedback on how well you are leveraging your infrastructure. The AutoPilot advantage is that it applies many person years of expert knowledge at the touch of a button and produces results in a matter of minutes.
Many times you have a production outage and you work with BEA Support to find out that there is a patch (CR) available for that. Wouldn't it be good to know of these patches that are applicable to your environment before you actually encounter the issue in production? With AutoPilot you can. Advisor checks for all the applicable patches to your environment and provides you with details on each. It doesn't just blindly give you a list of patches that are available from BEA in newer service packs, it goes and checks for relevance. This way we are able to filter the list of patches to the ones you must have.
Another problem that enterprises are facing is that extensive time and effort are required to align SOA/J2EE applications and WebLogic Servers for optimum performance. Currently, tuning WebLogic is considered an art and not science. Although more and more people are becoming familiar with it, the tuning process "tune one thing, see the impact, and repeat until balanced configuration is achieved," still remains an extremely time-consuming and laborious task. Because performance tuning is so complex, many enterprises are running WebLogic with configurations that are not fully optimal. Another factor to keep in mind here is that WebLogic tuning is extremely fragile. Every time you change you application you can throw the whole environment completely off balance, and previously tuned configurations may not be optimum due to the recent change to the application. I compare WebLogic tuning to the analogy of car wheels. Every time you change car tires you need to do a few other things such as wheel alignment, balancing, etc. to get the best performance. Similarly in the case of WebLogic, every time there is a change in the application, WebLogic and the application both need to be realigned to get the optimum performance. During my experience in the field I often got this response: "We did tuning last year, it should still be good, right?" Well if nothing changed in the environment and application then that is true, but with today's constantly changing business requirements, applications and environments change more often than once a year. With every change you should evaluate the performance impact. It may be that not all changes require change in WebLogic or application configuration, but at the very least every change should be looked at from a performance standpoint.
AutoPilot comes to the rescue for WebLogic tuning and capacity determination. AutoPilot changes WebLogic tuning from an extremely complex task to on that entails just a few clicks. AutoPilot automates the proven WebLogic tuning process using its IntelliTune technology built upon AutoPilot's inherent knowledge and decision-making capabilities. With AutoPilot, the process of tuning WebLogic remains the same - it's just that AutoPilot does all of the work of running loads, analyzing performance, making appropriate changes, and restarting WebLogic Server whenever needed. AutoPilot uses a user-provided load-generation script and runs various scenarios on the WebLogic application. It analyzes the application and WebLogic behavior during each load. At the end of each load and analysis, AutoPilot makes appropriate adjustments to the right tunables. After adjustments it reruns the load and compares. It follows this process until a perfect balance is achieved. During the tuning process AutoPilot identifies any application bottlenecks. You can start the WebLogic tuning process before you leave for the evening and the next morning you get a perfectly tuned WebLogic. Your involvement in the whole tuning process takes place only during the kickoff. AutoPilot Tune Wizard simplifies the kickoff process to just a few clicks. You can also schedule the kickoff to start tuning at nighttime (or any other time) if needed.
At the end of the tuning, you can view the performance impact using AutoPilot Tuning Analyzers. You get the before and after picture to compare the results of each tuning. Since AutoPilot tunes and detects problem at the WebLogic-engine level, it helps the whole suite of products based on WebLogic Server.
AutoPilot compliments self-tuning capabilities offered by WebLogic 9.0. In my opinion, self-tuning is a step in the right direction, but it will take time to mature. Dynamically changing the settings in production can cause unforeseen conditions and can result in outages. Until enterprises get comfortable with dynamically changing the settings in production, AutoPilot will help auto-tune applications and WebLogic in preproduction environments. This way you are pushing a well-tuned configuration to production and not doing tuning in production. Even in the future when self-tuning matures and enterprises become comfortable with self-tuning, it would make sense to use self-tuning features only as "traction control" or "stability assist," i.e., go to production with a well-tuned configuration and let self-tuning help in certain conditions not accounted for in preproduction tuning.
In a nutshell, AutoPilot compliments WebLogic 8.1, the most widely used version of WebLogic in production, with well-sought-after capabilities. Key differentiators for AutoPilot are its intelligence that is built upon many person years worth of embedded expert knowledge, the way AutoPilot leverages that to streamline the root-cause analysis and WebLogic tuning process, proactive guidance, and how AutoPilot results in significant reduction in total cost of ownership for the BEA WebLogic Platform.