Tomcat Restarts: Is it a Big Deal?

motif

While we like to believe that our application servers and web applications are flawless, the reality is that applications have bugs. Sometimes, they have nasty bugs, such as holding onto references and thus causing larger memory consumption over time. As a result, many IT operations have put in place processes to restart the application servers and web applications on a periodic basis. Some have written scripts to do this, and some rely on an administrator to wake up in the middle of the night to login remotely to the server and do the restarts. Even if you have flawless web applications, you still need to restart your application server as a result of configuration changes and/or to deploy new versions of your web applications.

We know that Apache Tomcat is a proven, rock-solid application server; however, as many have found out, restarting a Tomcat instance is not an exact science. Sometimes a shutdown will end up leaving the process running and holding onto the ports, causing subsequent starts to fail. The reasons why this happens are many and often vary per platform. It is not uncommon for administrators to fall back on kill -9 (or the ever popular Ctrl-C on Windows) to kill the process. In some cases where the Tomcat instance is started as a daemon, administrators first have to find out the process ID and then issue a kill command. Kill -9 is like a nuclear option. If you use the nuclear option before the diplomatic options run out (for example, giving Tomcat an opportunity to properly shut down), bad things happen. You may end up causing your web app persisted data to be in an unstable state. Therefore, I do not advise that administrators use the nuclear option without waiting for Tomcat to shut down properly. (This applies equally to the head of states 🙂 ).

So, what should an IT organization do in this circumstance? One option would be to hire a smart shell script developer to develop a solid init script that can handle the proper shutdown and restart of Tomcat. This assumes that you are going to maintain the script on the same basis as you maintain your other software. And we know that maintaining software and enhancing it is often an expensive proposition for IT. Do it for those custom applications that are highly domain-specific and require specialized skill sets that an IT vendor cannot deliver. However, for something as simple as restarting a Tomcat server, you do not need to take on the expense of writing a script and maintaining it for the next decade. If you already have developed a script, you can still reduce the cost of maintaining it.

Our in-house Tomcat expert Jason Brittain had to solve this exact problem when he was managing Tomcat instances and happened to spend several years of his life developing a solid init script. We adopted his script and incorporated it into Tcat Server. I will describe how restarts work with Tcat Server below.

To restart Tomcat Server, you have two options:

Option 1: When viewing a server’s details, click Restart on its summary screen.

Restarting a Tomcat is as easy as clicking a button

Option 2: On the Servers tab, select one or more servers and click Restart.

Tcat Server will then reliably shut down Tomcat (yes, it will explore all diplomatic options to get Tomcat to shut down properly before resorting to any other option). This is much easier and more time-efficient than waiting in front of your monitor for Tomcat to shut down before issuing a start command, and you don’t have to write and maintain your own custom script to handle it.

You can download Tcat Server here. It’s free to download and use in development and pre-production.


We'd love to hear your opinion on this post


5 Responses to “Tomcat Restarts: Is it a Big Deal?”

  1. How long does a restart (shutdown and startup) take under ideal circumstances. What effect does the number of applications (Mules?) have on this time?

    • This depends on the server on which Tomcat is running and as you say number of applications as well.

      We have seen as short as 1.5 second to probably 20 seconds or so. The advantage of using a solution like Tcat Server is that you are not checking process state repeatedly, as you can just select Restart and Tcat Server will do the required work to reliably restart the Tomcat instance.

  2. We dozens of applications on hundreds of Tomcat instances. Our solution is to run them (and any other Java deamon) inside a Java Service Wrapper. When an application gets into an instable state (no response according to some criteria, out of memory etc.), the service wrapper simply restarts it. Monitoring will pick this up so we can fix the problem, but for the time being the servers keep running.

    • Hi Matt,

      Thank you for your comment. I assume the solution you described is working good for your environment.

      Couple of things to consider: 1. Give Tomcat a chance to shutdown gracefully before issuing a kill -9. 2. Auto restarts based on a condition – do you find that you get false positives and do restarts when it was just a long running request for the app?.

      With Tcat Server, it is an administrator’s decision to restart Tomcat server and the logic in Tcat Server does give Tomcat a chance to shutdown properly. Give it a shot for few of your test Tomcat servers and see how that works for you. ( you can use it for free in dev and pre-production).

      Best Regards,
      Sateesh

  3. Interesting article, i’ve been looking for something like this to explain why it’s important to not restart tomcat when deploying apps and instead use the manager…