Back to Spring Boot Course

Spring Boot Course

Batch Scheduler

We can schedule the batch in SpringBoot using below ways.

  1. SpringBoot Batch Scheduler.
  2. Shedlock

SpringBoot Batch Scheduler

Lets checkout how to create an app using SpringBoot Batch Scheduler.

Add the batch dependency in pom.xml

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-batch</artifactId>
</dependency>

Cron expression

Cron expression is represented by six fields:

second, minute, hour, day of month, month, day(s) of week

screenshot of the app

From the above cron expression, we give you some example patterns:

"0 0 * * * *" = the top of every hour of every day.
"*/10 * * * * *" = every ten seconds.
"0 0 8-10 * * *" = 8, 9 and 10 o'clock of every day.
"0 0 8,10 * * *" = 8 and 10 o'clock of every day.
"0 0/30 8-10 * * *" = 8:00, 8:30, 9:00, 9:30 and 10 o'clock every day.
"0 0 9-17 * * MON-FRI" = on the hour nine-to-five weekdays.
"0 0 0 25 12 ?" = every Christmas Day at midnight.

Configure batch job scheduler with spring boot

Configuring batch job scheduler very easy with spring boot, we just do two steps:

  1. Enable scheduling with @EnableScheduling annotation.
  2. Create a method annotated with @Scheduled and a cron job pattern.

Lets see the code implementation

BatchSchedulerApplication


@SpringBootApplication
@EnableScheduling
public class BatchSchedulerApplication {

	public static void main(String[] args) {
		SpringApplication.run(BatchSchedulerApplication.class);
	}
}

BatchScheduler


@Component
@Slf4j
public class BatchScheduler {

	private Environment env;

	public BatchScheduler(Environment env) {
		this.env = env;
	}

	@Scheduled(cron = "app.scheduler.configCornJob", zone = "app.scheduler.timezone")
	public void quoteGeneratorScheduler() {
		log.info("quoteGeneratorScheduler Starts");
		String runnableEnv = env.getProperty("app.scheduler.runnableEnvironment");
		String randomQuote = Constants.quotes[(int) (Math.random() * Constants.quotes.length)];
		if (runnableEnv.equalsIgnoreCase("dev")) {
			System.out.println(randomQuote);
		}
		log.info("quoteGeneratorScheduler Ends");
	}
}

Constants

public class Constants {

	public static final String[] quotes = new String[] { "You will face many defeats in life, but never let yourself be defeated.",
			"The greatest glory in living lies not in never falling, but in rising every time we fall.",
			"In the end, it's not the years in your life that count" };

}

application.yml

app:
 scheduler:
   configCornJob: 0 0/1 * * * ?
   runnableEnvironment: dev
   timezone: IST

If you see the above implementation, We have created quote generated scheduler which runs every 1 minute.

Lets see the Logs now.

{"@timestamp":"2020-11-08T00:23:00.016+05:30","severity":"INFO","service":"praveenoruganti-springboot-batch-scheduler","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.praveen.batch.scheduler.BatchScheduler","methodName":"quoteGeneratorScheduler","lineNo":"24","message":"quoteGeneratorScheduler Starts","stacktrace":""}
You will face many defeats in life, but never let yourself be defeated.
{"@timestamp":"2020-11-08T00:23:00.016+05:30","severity":"INFO","service":"praveenoruganti-springboot-batch-scheduler","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.praveen.batch.scheduler.BatchScheduler","methodName":"quoteGeneratorScheduler","lineNo":"35","message":"quoteGeneratorScheduler Ends","stacktrace":""}
{"@timestamp":"2020-11-08T00:24:00.010+05:30","severity":"INFO","service":"praveenoruganti-springboot-batch-scheduler","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.praveen.batch.scheduler.BatchScheduler","methodName":"quoteGeneratorScheduler","lineNo":"24","message":"quoteGeneratorScheduler Starts","stacktrace":""}
The greatest glory in living lies not in never falling, but in rising every time we fall.
{"@timestamp":"2020-11-08T00:24:00.010+05:30","severity":"INFO","service":"praveenoruganti-springboot-batch-scheduler","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.praveen.batch.scheduler.BatchScheduler","methodName":"quoteGeneratorScheduler","lineNo":"35","message":"quoteGeneratorScheduler Ends","stacktrace":""}

Drawbacks of SpringBoot Batch Scheduler

We need to create seperate app for batch scheduler as it needs to run in single instance which is a drawback in terms of maintenance which can be eliminated by using Shedlock.

Shedlock with MySQL

Lets see how to use Shedlock which provides a distributed lock mechanism for cron jobs. Well for this we also need to have a database or redis so I will use MySQL in this example to run locally.


<dependency>
	<groupId>net.javacrumbs.shedlock</groupId>
	<artifactId>shedlock-spring</artifactId>
	<version>4.15.1</version>
</dependency>
<dependency>
	<groupId>net.javacrumbs.shedlock</groupId>
	<artifactId>shedlock-provider-jdbc-template</artifactId>
	<version>4.15.1</version>
</dependency>

Enabling Shedlock

First step is creating a table for locks to be used by Shedlock.


CREATE TABLE shedlock(
    name VARCHAR(64),
    lock_until TIMESTAMP(3) NULL,
    locked_at TIMESTAMP(3) NULL,
    locked_by  VARCHAR(255),
    PRIMARY KEY (name)
)

Later on, we have to add one annotation and a bean producer for LockProvider for enabling Shedlock. So, our Application class will look like.

ShedlockJobHandler

@Component
@Slf4j
@EnableScheduling
@EnableSchedulerLock(defaultLockAtMostFor = "app.shedlock.defaultLockAtMostFor")
public class ShedlockJobHandler {

	@Bean
	public LockProvider lockProvider(DataSource dataSource) {
	    return new JdbcTemplateLockProvider(dataSource);
	}
}
  • EnableSchedulerLock annotation enables Shedlock in Spring context so that we can use it for our jobs. defaultLockAtMostFor attribute is required but it can be overridden by individual tasks.
  • We need to create a Bean for LockProvider which will be used by Shedlock to update lock table we created.

Running a Job with Shedlock

All jobs which need distributed lock should be annotated with @SchedulerLock annotation, otherwise, jobs will run in parallel. There are three main parameters to configure.

  • name - Every job has to have a unique name. Remember that name field from shedlock table was the primary key
  • lockAtLeastFor - This attribute is for ensuring the current job will hold the lock at least given amount of time. For example; if we configure it to 20 seconds then the lock won't released even though job finished before 20 seconds.
  • lockAtMostFor - This attribute is for making sure lock is released in case of executing instance dies. As it is suggested from the documentation itself it will be better to set this attribute larger than the maximum estimated execution time. If a task takes longer than this value than an unexpected behaviour might happen because some other instance will also run the same job since lock will be released.

I will create a job called QuoteGeneratorJob with lockAtLeastFor attribute set to 15 seconds and lockAtMostFor attribute set to 20 seconds. The actual time it will take to run will be just 5 seconds and it will run at every minute.

ShedlockJobHandler

@Component
@Slf4j
@EnableScheduling
@EnableSchedulerLock(defaultLockAtMostFor = "app.shedlock.defaultLockAtMostFor")
public class ShedlockJobHandler {

	private Environment env;

	public ShedlockJobHandler(Environment env) {
		this.env = env;
	}

	@Scheduled(cron = "app.shedlock.scheduler.configCornJob", zone = "app.shedlock.scheduler.timezone")
	@SchedulerLock(name = "app.shedlock.name", lockAtLeastFor = "app.shedlock.lockAtLeastFor", lockAtMostFor = "app.shedlock.lockAtMostFor")
	public void quoteGeneratorJob() throws InterruptedException {
		log.info("quoteGeneratorJob Starts");
		String runnableEnv = env.getProperty("app.shedlock.scheduler.runnableEnvironment");
		String randomQuote = Constants.quotes[(int) (Math.random() * Constants.quotes.length)];
		if (runnableEnv.equalsIgnoreCase("dev")) {
			System.out.println(randomQuote);
		}
		log.info("quoteGeneratorJob Ends");
	}

	@Bean
	public LockProvider lockProvider(DataSource dataSource) {
	    return new JdbcTemplateLockProvider(dataSource);
	}
}

Constants

public class Constants {
	public static final String[] quotes = new String[] {
			"You will face many defeats in life, but never let yourself be defeated.",
			"The greatest glory in living lies not in never falling, but in rising every time we fall.",
			"In the end, it's not the years in your life that count" };
}

application.yml

server:
  port: 8080
spring:
  application:
    name: praveenoruganti-springboot-shedlock
  jpa:
  database: MYSQL
  show-sql: true
  generate-ddl: false

  datasource:
    type: com.zaxxer.hikari.HikariDataSource
    driver-class-name: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://root:password@localhost:3306/praveendb?reconnect=true
    username: root
    password: password
    hikari:
      connectionTimeout : 30000
      idleTimeout : 600000
      maxLifetime : 1800000
      maximumPoolSize : 5
management:
  endpoints:
   enable-by-default: false
   web:
     base-path: /
     exposure:
       include: info,refresh,logger
  endpoint:
    info:
      enabled: true
    refresh:
      enabled: true
    loggers:
      enabled: true
apidocs:
  info:
   description: |

       Praveen Oruganti SpringBoot Shedlock

   version: '1.0'
   title: Praveen Oruganti SpringBoot Shedlock
   termsOfService: 'urn:tos'
   contact:
     name: Praveen Oruganti
     url: https://www.praveenoruganti.in
     email: praveenoruganti@gmail.com

app:
 disable:
  tryOutOption:false
 serviceName: Praveen Oruganti SpringBoot Shedlock
 shedlock:
   name: QuoteGeneratorJob
   defaultLockAtMostFor: 30s
   lockAtLeastFor: 15s
   lockAtMostFor: 20s
   scheduler:
     configCornJob: 0 0/1 * * * ?
     runnableEnvironment: dev
     timezone: IST

Here with the application logs

"@timestamp":"2020-11-08T01:03:00.455+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"29","message":"quoteGeneratorJob Starts","stacktrace":""}
You will face many defeats in life, but never let yourself be defeated.
{"@timestamp":"2020-11-08T01:03:00.459+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"35","message":"quoteGeneratorJob Ends","stacktrace":""}
{"@timestamp":"2020-11-08T01:04:00.126+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"29","message":"quoteGeneratorJob Starts","stacktrace":""}
In the end, it's not the years in your life that count
{"@timestamp":"2020-11-08T01:04:00.127+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"35","message":"quoteGeneratorJob Ends","stacktrace":""}
{"@timestamp":"2020-11-08T01:05:00.155+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"29","message":"quoteGeneratorJob Starts","stacktrace":""}
The greatest glory in living lies not in never falling, but in rising every time we fall.
{"@timestamp":"2020-11-08T01:05:00.157+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","trace":"","span":"","parent":"","exportable":"","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"35","message":"quoteGeneratorJob Ends","stacktrace":""}

If you see the table only one instance will hold the lock.

screenshot of the app

Shedlock with Redis

We have seen Shedlock with MySQL before but we can also use Redis for Shedlock.

Below changes we need to do to achieve this

  1. pom.xml
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>
<dependency>
  <groupId>net.javacrumbs.shedlock</groupId>
  <artifactId>shedlock-spring</artifactId>
  <version>4.29.0</version>
</dependency>
  <dependency>
  <groupId>net.javacrumbs.shedlock</groupId>
  <artifactId>shedlock-provider-redis-spring</artifactId>
  <version>4.29.0</version>
</dependency>
  1. Include the redis lock provider bean
@Bean
public LockProvider lockProvider(RedisTemplate<String, String> redisTemplate) {
  return new RedisLockProvider(redisTemplate.getConnectionFactory());
}
  1. application.yml
spring:
 application:
  name: praveenoruganti-springboot-shedlock
 redis:
  host: redis-19158.c212.ap-south-1-1.ec2.cloud.redislabs.com
  port: 19158
  password: flyL2EpWKG4YuFehTzzzg8VnnKzmUOjC
 datasource:
     url: jdbc:mysql://root:password-1@localhost:3306/praveenoruganti?reconnect=true 
     driverClassName: com.mysql.cj.jdbc.Driver 
     
 jpa:
    hibernate:
     ddl-auto: update 

 
  
springdoc:
   use-fqn: true
   swagger-ui:
    doc-expansion: none
    operationsSorter: method
#   supportedSubmitMethods: []


app:
 openApiTitle: SpringBoot Shedlock
 openApiDescription: |
   
       SpringBoot Shedlock
 shedlock: 
   name: QuoteGeneratorJob
   defaultLockAtMostFor: 30s
   lockAtLeastFor: 15s
   lockAtMostFor: 20s
   scheduler:
     configCornJob: 0 0/1 * * * ?
     runnableEnvironment: dev
     timezone: IST

Here with the application logs

{"@timestamp":"2020-11-08T01:29:00.883+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"33","message":"quoteGeneratorJob Starts","stacktrace":"","traceId":"d2724292fe8256c7","spanId":"d2724292fe8256c7"}
The greatest glory in living lies not in never falling, but in rising every time we fall.
{"@timestamp":"2020-11-08T01:29:00.883+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"39","message":"quoteGeneratorJob Ends","stacktrace":"","traceId":"d2724292fe8256c7","spanId":"d2724292fe8256c7"}
{"@timestamp":"2020-11-08T01:30:00.081+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"33","message":"quoteGeneratorJob Starts","stacktrace":"","traceId":"71466c57c961c376","spanId":"71466c57c961c376"}
In the end, it's not the years in your life that count
{"@timestamp":"2020-11-08T01:30:00.082+05:30","severity":"INFO","service":"praveenoruganti-springboot-shedlock","thread":"scheduling-1","class":"c.p.shedlock.handler.ShedlockJobHandler","methodName":"quoteGeneratorJob","lineNo":"39","message":"quoteGeneratorJob Ends","stacktrace":"","traceId":"71466c57c961c376","spanId":"71466c57c961c376"}

If your Spring Boot application is a standalone application, i.e., it will not handle HTTP requests (such as a batch application), you can disable the embedded web server in the configuration.

spring:
  main:
   web-application-type: none