Introduction
In a distributed system, logging is critical for tracking the flow of requests and responses between microservices. In this article, we will describe a distributed logger built using Spring Boot and Microsoft Application Insights. This logger provides extensible logging, correlation IDs, and flexible logging formats.
Library Setup
In this article, we’ll be setting up a logger that streamlines the logging process, making it easy to track the flow of requests and responses between micro-services. Our logger uses a correlation ID to keep track of transactions and ensure that logs are properly formatted and organized. The logger is composed of three key components: the logUtils class, which creates a new log appender; the HeaderRequestMapper class, which provides utility methods for working with HTTP requests; and the CorrelationIdFilterBase class, which can be extended in your application to add correlation IDs to incoming requests.
Azure Dependencies
Maven:
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>spring-cloud-azure-starter</artifactId>
</dependency>
<properties>
<version.spring.cloud>2021.0.3</version.spring.cloud>
<version.spring.cloud.azure>4.3.0</version.spring.cloud.azure>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${version.spring.cloud}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.azure.spring</groupId>
<artifactId>spring-cloud-azure-dependencies</artifactId>
<version>${version.spring.cloud.azure}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Gradle:
ext {
set('springBootCloudVersion', "2021.0.4")
set('springBootStarterVersion', "2.7.2")
}
dependencies {
implementation "org.springframework.cloud:spring-cloud-starter:${springBootCloudVersion}"
implementation "org.springframework.boot:spring-boot-starter-web:${springBootStarterVersion}"
}
CorrelationId Filter
The CorrelationIdFilterBase.java class is a custom request filter that can be extended in your application to add Correlation IDs to requests.
public class CorrelationIdFilterBase extends OncePerRequestFilter {
public static final String CORRELATION_ID_HEADER_NAME = "x-correlation-id";
public static final String CORRELATION_ID_MDC_KEY = "CorrelationId";
@Override
public void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
List<String> headers = Collections.list(request.getHeaderNames());
if (headers.contains(CORRELATION_ID_HEADER_NAME)) {
MDC.put(CORRELATION_ID_MDC_KEY, request.getHeader(CORRELATION_ID_HEADER_NAME));
filterChain.doFilter(request, response);
} else {
String id = UUID.randomUUID().toString();
MDC.put(CORRELATION_ID_MDC_KEY, id);
HeaderRequestMapper headerRequestMapper = new HeaderRequestMapper(request);
headerRequestMapper.addHeader(CORRELATION_ID_HEADER_NAME, id);
response.addHeader(CORRELATION_ID_HEADER_NAME, id);
filterChain.doFilter(headerRequestMapper, response);
}
}
}
Request Header Wrapper
The HeaderRequestMapper.java class provides convenient methods for fetching and setting Http Requests.
public class HeaderRequestMapper extends HttpServletRequestWrapper {
public HeaderRequestMapper(HttpServletRequest request) {
super(request);
}
private Map<String, String> headerMap = new HashMap<String, String>();
public void addHeader(String name, String value) {
headerMap.put(name, value);
}
@Override
public String getHeader(String name) {
String headerValue = super.getHeader(name);
if (headerMap.containsKey(name)) {
headerValue = headerMap.get(name);
}
return headerValue;
}
@Override
public Enumeration<String> getHeaderNames() {
List<String> names = Collections.list(super.getHeaderNames());
for (String name : headerMap.keySet()) {
names.add(name);
}
return Collections.enumeration(names);
}
@Override
public Enumeration<String> getHeaders(String name) {
List<String> values = Collections.list(super.getHeaders(name));
if (headerMap.containsKey(name)) {
values.add(headerMap.get(name));
}
return Collections.enumeration(values);
}
}
Logging Utility
The LogUtils.java class is a utility class that will create a new Log Appender.
public class LogUtils {
public static Logger getConsoleLogger(String loggerName, Level level) {
// add the pid (process id) to the MDC context
RuntimeMXBean bean = ManagementFactory.getRuntimeMXBean();
MDC.put("pid", bean.getName().split("@")[0]);
LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
PatternLayoutEncoder ple = configurePatterLayout(context);
ConsoleAppender<ILoggingEvent> consoleAppender = configureConsoleAppender(context, ple);
Logger logger = configureLogger(consoleAppender, level, loggerName);
return logger;
}
private static Logger configureLogger(ConsoleAppender<ILoggingEvent> consoleAppender, Level level, String loggerName) {
Logger logger = (Logger) LoggerFactory.getLogger(loggerName);
logger.addAppender(consoleAppender);
logger.setLevel(level);
logger.setAdditive(false);
return logger;
}
private static ConsoleAppender<ILoggingEvent> configureConsoleAppender(LoggerContext context, PatternLayoutEncoder ple) {
ConsoleAppender<ILoggingEvent> consoleAppender = new ConsoleAppender<>();
consoleAppender.setEncoder(ple);
consoleAppender.setWithJansi(true);
consoleAppender.setContext(context);
consoleAppender.start();
return consoleAppender;
}
private static PatternLayoutEncoder configurePatterLayout(LoggerContext context) {
PatternLayoutEncoder ple = new PatternLayoutEncoder();
ple.setPattern("%-3d{dd-MM-yyyy HH:mm:ss.SSS} %highlight(%level) %magenta(%mdc{pid}) --- [%mdc{CorrelationId}] %cyan(%C{40}).%M : %msg%n");
ple.setContext(context);
ple.start();
return ple;
}
}
Library Usage
To use the Console Appender logger library we just created in your Spring Boot project, you first need to add it as a dependency. This can be done with Maven or Gradle.
Maven:
<dependency>
<groupId>com.examplecompany.azure</groupId>
<artifactId>logback-console-appender</artifactId>
<version>current version</version>
</dependency>
Gradle:
implementation 'com.examplecompany.azure:logback-console-appender:current version';
Next, create a new logger in the class where you want to use the Console Appender logger:
private static final Logger logger = LogUtils.getConsoleLogger(MyAppName.class.getName(), Level.ALL);
To log messages, use the following code:
logger.info("My log message");
To use the correlation ID filter, extend the CorrelationIdFilterBase class and add it to the filter chain.
@Component
@Order(1)
public class CorrelationIdFilter extends CorrelationIdFilterBase {
@Override public void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
super.doFilterInternal(request, response, filterChain);
}
}
If your service makes calls to other APIs, you can maintain the correlation ID header by adding the following code:
WebClient webClient = new WebClient();
return webClient
.put()
.uri(uriPath)
.attributes(oauth2AuthorizedClient(client))
.header(CorrelationIdFilterBase.CORRELATION_ID_HEADER_NAME, MDC.get(CorrelationIdFilterBase.CORRELATION_ID_MDC_KEY))
.header(HttpHeaders.CONTENT_TYPE, "application/json")
.bodyValue(resourceDTO.getJson())
.retrieve()
.bodyToMono(String.class);
Integration with Microsoft Application Insights
The logging library described in the article can be integrated with Application Insights, allowing you to use the tool to monitor the logs produced by the library. This can help you identify issues and trends in the logs and use that information to improve the performance and reliability of your applications.
Additionally, Application Insights supports a variety of ways to visualize and analyze the data, including built-in dashboards, custom charts and graphs, and integration with other tools such as Power BI. By leveraging the capabilities of Application Insights, you can gain deeper visibility into your applications and services, allowing you to make informed decisions about how to optimize their performance.
Application Insights Integration
In your projects Application class add the following line to your main() method:
ApplicationInsights.attach();
Create a configuration json file in your /resources folder named applicationinsights.json
Add the following to the applicationinsights.json file adding the InstrumentationKey value of your Application Insights instance:
{
"connectionString": "InstrumentationKey=[InstrumentationKey value]"
}
You can change defaults of what level to log by adding it to the json file, the default level is “INFO”. For example, to log all debug messages to Application Insights, change the json to be:
{
"connectionString": "InstrumentationKey=[InstrumentationKey value]",
"instrumentation": {
"logging": {
"level": "DEBUG"
}
}
}
Additional configuration options using the json file can be found here.
Conclusion
In conclusion, the distributed logger using Spring Boot and Microsoft’s Application Insights is a flexible and extensible logging library. With the ability to log messages to the console, maintain correlation IDs, and log to Application Insights, it provides a comprehensive solution for logging in a distributed environment. Additionally, by extending the CorrelationIdFilterBase class, you can easily customize the logging format to meet your specific needs.
