Cron framework options in cloud environment

Category: Cron

This blog post is for all developers, who need to start fresh or refactor existing cron implementation. In this post, we will detail insights about 3 of popular cron frameworks that we have gone through for cloud environment.

At Endurance, we empower more than 15 million domain names.

We were faced with the task of building a service which is responsible for all activities related to provisioning and management of domain names. A critical part of this service was to run background jobs/crons and audit logging was advisable for these crons.

Development teams typically want their crons to run on a separate Instance/Cluster, keeping scale and separation of concern in mind, even having their own codebase.

In our case,

  • We did not have a separate code
  • A lot of common code written in the service itself
  • We did not anticipate resource-intensive tasks
  • We did not foresee crons affecting production traffic
  • Thus, no requirement was there to spin off a separate cron cluster.

The service was going to be deployed on AWS. Every instance in the cluster would have both the service code and the cron logic embedded within it. A few of the crons had an additional requirement that at any given time, we had to ensure that only a single instance of that particular cron could execute. Considering that the codebase was in spring boot, any solutions which had a direct spring boot integration would be an added bonus.

To implement the requirements defined above, we evaluated three options

  • Spring boot @Scheduled: Spring native support for task scheduling
  • AWS CloudWatch Events + Lambda/API: Scheduling tasks on AWS with minimal resources. This would mean that we would have the scheduling done via AWS lambdas, which would trigger the cron jobs on the service instances via the load balancer.
  • Quartz framework: Popular + Widely used open-source framework for job scheduling

We compared each of the alternatives with our desired features to identify the best fit.

Spring boot @Scheduled:

This is the simplest approach that spring provides out of the box. The simple rules that need to be followed to annotate a method with @Scheduled are:

  • The method should have void return type
  • The method should not accept any parameters

Once methods are annotated as such, they will get executed based on the cron expression defined in the annotation. Unfortunately, this approach does not allow us to constrain the execution in a way to restrict only execution of a single cron at any given time across all instances.

AWS CloudWatch Events + Lambda/API

 

aws setup with crons via apiThis is a hybrid approach where we could have scheduled events to trigger lambda which could, in turn, invoke the API on the load balancer, which would, in turn, hit the load balancer. These APIs would be responsible for the actual cron execution. Since the API would be invoked via the load balancer, it would automatically ensure that the actual cron job is executed on only 1 of the service instances. The lambda’s however, typically have an execution limit of ~15 minutes. So the caller would not be able to record whether a given cron job has executed successfully or not if it took more time to run. As a result, we would have to implement more functionality to guarantee the execution of these cron jobs.

 

 

Quartz framework

Quartz is a feature-rich scheduling library that can be easily integrated within Spring boot. It has out of the box support for features such a clustering  – fail-over, load- balancing, listeners and plugins which could help with maintaining history. This seemed like the ideal solution for our use-case since it provided a solution for almost all of our constraints.

 

Added below is a comparison table on the various aspects that we evaluated these approaches on

 

Feature Quartz Spring boot @Scheduled AWS CloudWatch Events + Lambda/API call
Job triggering Maximum number of options compared to the rest (including cron expressions) Cron expressions, fixed-rate or fixed-delay based Cron expressions
Job definition Any java class that implements the Job interface

eg:

public class CronJOb implements Job{

@Override
public void execute (JobExecutionContext context){..}

}

Any method with the @Scheduled annotation

eg:

@Scheduled(cron=”0 0 * * * *”)
public void cronJob () {… }

Two parts :

  • Lambda function which needs to be triggered based on an event. This will be responsible for invoking the API
  • API call defined on a service which will have the core implementation of the cron job
Clustering support Provides load-balancing, automatic fail-over out of the box using JDBC persistence. No support out of the box We won’t need separate clustering support in this case, as only a single instance of the API will get invoked via the lambda. However, it would need additional effort to build load balancing and fail-over.
Listeners and plugin support A number of listeners and plugins are available to support audit logging,  loading job definitions via xml etc No We can add Listeners like triggering an AWS SNS event at the completion of execution and use it however we want. This can also be used to keep the history of job execution.
Cons No support for ensuring a single instance of cron job executes at a given time. No persistence option out of the box. The feature set is quite limited.

Maximum Execution for lambda is 15 minutes. So we cannot invoke crons synchronously.

Also, this approach is tightly coupled with AWS services hence it would make moving out of AWS more difficult in the future.

References:

http://www.quartz-scheduler.org/
https://spring.io/guides/gs/scheduling-tasks/
https://blog.readme.io/writing-a-cron-job-microservice-with-serverless-and-aws-lambda/
https://docs.aws.amazon.com/lambda/latest/dg/limits.html