Class QoSHandler

java.lang.Object
org.eclipse.jetty.util.component.AbstractLifeCycle
org.eclipse.jetty.util.component.ContainerLifeCycle
org.eclipse.jetty.server.Handler.Abstract
org.eclipse.jetty.server.Handler.AbstractContainer
org.eclipse.jetty.server.Handler.Wrapper
org.eclipse.jetty.server.handler.ConditionalHandler
org.eclipse.jetty.server.handler.ConditionalHandler.Abstract
org.eclipse.jetty.server.handler.QoSHandler
All Implemented Interfaces:
Handler, Handler.Container, Handler.Singleton, Request.Handler, Container, Destroyable, Dumpable, Dumpable.DumpableContainer, LifeCycle, Invocable
@ManagedObject public class QoSHandler extends ConditionalHandler.Abstract

A quality of service Handler that conditionally limits the number of concurrent requests, to provide more predictable end-user experience in case descendant Handlers have limited capacity.

This Handler limits the number of concurrent requests to the number configured via setMaxRequestCount(int). If more requests are received, they are suspended (that is, not forwarded to the child Handler) and stored in a priority queue.

The maximum number of suspended request can be set with setMaxSuspendedRequestCount(int) to avoid out of memory errors. When this limit is reached, the request will fail fast with status code 503 (not available).

Priorities are determined via getPriority(Request), that should return values between 0 (the lowest priority) and positive numbers, typically in the range 0-10.

When a request that is being processed completes, the suspended request that current has the highest priority is resumed.

This Handler is ideal to avoid contending on slow/limited resources such as a JDBC connection pool, avoiding the situation where all server threads blocked contending on the limited resource, therefore leaving threads free to process other requests that do not require access to the limited resource.

Requests are resumed in priority order, so that when the server is under load, and there are many requests suspended to be processed, high priority request are processed first. For example, load balancer "ping" requests may have the highest priority, followed by requests performed by admin users, etc. so that regardless of the load, "ping" and "admin" requests will always be able to access the web application.

  • Constructor Details

    • QoSHandler

      public QoSHandler()

    • QoSHandler

      public QoSHandler(Handler handler)

  • Method Details

    • getMaxRequestCount

      @ManagedAttribute(value="The maximum number of concurrent requests", readonly=true) public int getMaxRequestCount()
      Returns:
      the max number of concurrent requests

    • setMaxRequestCount

      public void setMaxRequestCount(int maxRequests)

      Sets the max number of concurrent requests.

      A negative or zero value indicates to calculate a value based on heuristics, drawn from the number the size of the server thread pool and/or the number of CPU cores.

      Parameters:
      maxRequests - the max number of concurrent requests

    • getMaxSuspendedRequestCount

      @ManagedAttribute(value="The maximum number of suspended requests", readonly=true) public int getMaxSuspendedRequestCount()
      Returns:
      the max number of suspended requests

    • setMaxSuspendedRequestCount

      public void setMaxSuspendedRequestCount(int maxSuspendedRequests)

      Sets the max number of suspended requests.

      Once the max suspended request limit is reached, the request is failed with a HTTP status of 503 Service unavailable.

      A negative value indicate an unlimited number of suspended requests.

      Parameters:
      maxSuspendedRequests - the max number of suspended requests

    • getMaxSuspend

      public Duration getMaxSuspend()
      Get the max duration of time a request may stay suspended.
      Returns:
      the max duration of time a request may stay suspended

    • setMaxSuspend

      public void setMaxSuspend(Duration maxSuspend)

      Sets the max duration of time a request may stay suspended.

      Once the duration expires, the request is failed with an HTTP status of 503 Service Unavailable.

      Duration.ZERO means that the request may stay suspended forever.

      Parameters:
      maxSuspend - the max duration of time a request may stay suspended

    • getSuspendedRequestCount

      @ManagedAttribute("The number of suspended requests") public int getSuspendedRequestCount()

    • doStart

      protected void doStart() throws Exception
      Description copied from class: ContainerLifeCycle
      Starts the managed lifecycle beans in the order they were added.
      Overrides:
      doStart in class ConditionalHandler
      Throws:
      Exception - If there was a problem starting. Will cause a transition to FAILED state

    • doStop

      protected void doStop() throws Exception
      Description copied from class: ContainerLifeCycle
      Stops the managed lifecycle beans in the reverse order they were added.
      Overrides:
      doStop in class Handler.Abstract
      Throws:
      Exception - If there was a problem stopping. Will cause a transition to FAILED state

    • onConditionsMet

      public boolean onConditionsMet(Request request, Response response, Callback callback) throws Exception
      Description copied from class: ConditionalHandler
      Handle a request that has met the conditions. Typically, the implementation will provide optional handling and then call the ConditionalHandler.nextHandler(Request, Response, Callback) method to continue handling.
      Specified by:
      onConditionsMet in class ConditionalHandler
      Parameters:
      request - The request to handle
      response - The response to generate
      callback - The callback for completion
      Returns:
      True if this handler will complete the callback
      Throws:
      Exception - If there is a problem handling
      See Also:

    • onConditionsNotMet

      protected boolean onConditionsNotMet(Request request, Response response, Callback callback) throws Exception
      Description copied from class: ConditionalHandler
      This method is called when the request has not met the conditions and is not to be handled by this handler. Implementations may return false; send an error response; or handle the request differently.
      Specified by:
      onConditionsNotMet in class ConditionalHandler
      Parameters:
      request - The request to handle
      response - The response to generate
      callback - The callback for completion
      Returns:
      True if this handler will complete the callback
      Throws:
      Exception - If there is a problem handling
      See Also:

    • getPriority

      protected int getPriority(Request request)

      Returns the priority of the given suspended request, a value greater than or equal to 0.

      Priority 0 is the lowest priority.

      The set of returned priorities should be stable over time, typically constrained in the range 0-10.

      Parameters:
      request - the suspended request to compute the priority for
      Returns:
      the priority of the given suspended request, a value >= 0

    • failSuspended

      protected void failSuspended(Request request, Response response, Callback callback, int status, Throwable failure)

      Fails the given suspended request/response with the given error code and failure.

      This method is called only for suspended requests, in case of timeout while suspended, or in case of failure when trying to handle a resumed request.

      Parameters:
      request - the request to fail
      response - the response to fail
      callback - the callback to complete
      status - the failure status code
      failure - the failure