The Stepdown Rule

In Clean Code, Robert discusses a concept called the stepdown rule, which we’ll be discussing today. The stepdown rule says that each function should read as a top-down narrative: starting with the bigger picture, then working down to the specific details. In practice, this means breaking down a method that contains multiple steps using descriptive helper functions that describe what each step does. This makes the code read like:

public void myMethod() {
    stepOne();
    var items = stepTwo();
    return stepThree(items);
}

This keeps myMethod small and focused on its higher-level logic while abstracting the finer details of each step into helper methods. This should help myMethod read more like a list of easy-to-understand steps, and when you want to delve deeper into how each step works, you can look at each of the helper methods, which stay focused on the logic of that step. You can think of this as similar to how we create new service interfaces to abstract business logic and make other classes more coherent, just on the method level.

Practicing The Stepdown Rule

To show this rule more in detail, I prepared a simple program that we can refactor to showcase how this style cleans up our code:

package io.john.amiscaray.service;

import io.john.amiscaray.domain.Item;
import lombok.AllArgsConstructor;

import java.io.PrintStream;
import java.util.List;

@AllArgsConstructor
public class ItemPrinter {

    private static final Integer TABLE_ITEM_SIZE = 20;
    private PrintStream printer;

    public void printItems(List<Item> items) throws IllegalAccessException {
        var itemFields = Item.class.getDeclaredFields();
        var separatorLength = itemFields.length * 21 + 1;
        for (var field : itemFields) {
            printTableCell(field.getName());
        }

        printNextTableLine(separatorLength);
        for (var item : items) {
            for (var field : itemFields) {
                field.setAccessible(true);
                var fieldValue = field.get(item);
                if (fieldValue instanceof Float moneyValue) {
                    printTableCell("$" + moneyValue);
                } else {
                    printTableCell(fieldValue.toString());
                }
            }
            printNextTableLine(separatorLength);
        }
    }

    private void printTableCell(String item) {
        var padding = TABLE_ITEM_SIZE - item.length();
        var paddingLeft = padding / 2;
        var paddingRight = padding - paddingLeft;
        printer.printf("|" + " ".repeat(paddingLeft) + "%s" + " ".repeat(paddingRight), item);
    }

    public void printNextTableLine(int length) {
        printer.println("|");
        printer.println("_".repeat(Math.max(0, length)));
    }

}

package io.john.amiscaray.domain;

import lombok.AllArgsConstructor;
import lombok.Data;

@AllArgsConstructor
@Data
public class Item {

    private String name;
    private String description;
    private Float cost;

}

package io.john.amiscaray;

import io.john.amiscaray.domain.Item;
import io.john.amiscaray.service.ItemPrinter;

import java.util.List;

public class Main {

    public static void main(String[] args) throws IllegalAccessException {
        var itemPrinter = new ItemPrinter(System.out);
        itemPrinter.printItems(List.of(
                new Item("Chicken Breast", "10lbs", 13.99f),
                new Item("Salad", "Caesar Salad", 10.99f)
        ));
    }

}

The class we will be refactoring is the ItemPrinter class. This contains a print method that logs a list of Item objects as a table. To do this, first, it prints the field names of the Item class as the table header (fetching them using reflection), then it prints the field values of each object in the table rows. As you can see, this method violates the stepdown rule by showing specific details of each of its steps within the method body. While this may not be a huge deal with a simpler method like this, for a more complex method, violating this rule can make it very difficult to understand and maintain. Leveraging this rule, we can start to make the method cleaner. First, let's start by separating the steps of the printItems method into helpers:

package io.john.amiscaray.service;

import io.john.amiscaray.domain.Item;
import lombok.AllArgsConstructor;

import java.io.PrintStream;
import java.lang.reflect.Field;
import java.util.List;

@AllArgsConstructor
public class ItemPrinter {

    private static final Integer TABLE_ITEM_SIZE = 20;
    private PrintStream printer;

    public void printItems(List<Item> items) throws IllegalAccessException {
        var itemFields = getItemFields();
        var separatorLength = calculateLineSeparatorLength(itemFields);

        printTableHeader(itemFields);
        printNextTableLine(separatorLength);
        printTableItems(items, itemFields, separatorLength);
    }

    private static Field[] getItemFields() {
        return Item.class.getDeclaredFields();
    }

    private static int calculateLineSeparatorLength(Field[] itemFields) {
        return itemFields.length * 21 + 1;
    }

    private void printTableHeader(Field[] itemFields) {
        for (var field : itemFields) {
            printTableCell(field.getName());
        }
    }

    private void printNextTableLine(int length) {
        printer.println("|");
        printer.println("_".repeat(Math.max(0, length)));
    }

    private void printTableItems(List<Item> items, Field[] itemFields, int separatorLength) throws IllegalAccessException {
        for (var item : items) {
            for (var field : itemFields) {
                field.setAccessible(true);
                var fieldValue = field.get(item);
                if (fieldValue instanceof Float moneyValue) {
                    printTableCell("$" + moneyValue);
                } else {
                    printTableCell(fieldValue.toString());
                }
            }
            printNextTableLine(separatorLength);
        }
    }

    private void printTableCell(String item) {
        var padding = TABLE_ITEM_SIZE - item.length();
        var paddingLeft = padding / 2;
        var paddingRight = padding - paddingLeft;
        printer.printf("|" + " ".repeat(paddingLeft) + "%s" + " ".repeat(paddingRight), item);
    }

}

Now, after that refactor, the ItemPrinter#printItems method starts to follow the list of steps format I mentioned above. The inner helper methods help us describe the steps the printItems method takes, making it easier to understand at a glance how it works, while making sure each method focuses on one thing (another rule that Robert advocates for). By making sure each method does one thing, it makes it easier to understand how each of them works, making the whole class easier to understand.

While that one refactor helped a ton, I still have my eye on the printTableItems method for us to refactor. The logic within the loops of checking the field type and printing the appropriate value seems like something we can break down further using this rule:

package io.john.amiscaray.service;

import io.john.amiscaray.domain.Item;
import lombok.AllArgsConstructor;

import java.io.PrintStream;
import java.lang.reflect.Field;
import java.util.List;

@AllArgsConstructor
public class ItemPrinter {

    private static final Integer TABLE_ITEM_SIZE = 20;
    private PrintStream printer;

    public void printItems(List<Item> items) throws IllegalAccessException {
        var itemFields = getItemFields();
        var separatorLength = calculateLineSeparatorLength(itemFields);

        printTableHeader(itemFields);
        printNextTableLine(separatorLength);
        printTableItems(items, itemFields, separatorLength);
    }

    private static Field[] getItemFields() {
        return Item.class.getDeclaredFields();
    }

    private static int calculateLineSeparatorLength(Field[] itemFields) {
        return itemFields.length * 21 + 1;
    }

    private void printTableHeader(Field[] itemFields) {
        for (var field : itemFields) {
            printTableCell(field.getName());
        }
    }

    private void printNextTableLine(int length) {
        printer.println("|");
        printer.println("_".repeat(Math.max(0, length)));
    }

    private void printTableItems(List<Item> items, Field[] itemFields, int separatorLength) throws IllegalAccessException {
        for (var item : items) {
            printItemRow(itemFields, item);
            printNextTableLine(separatorLength);
        }
    }

    private void printItemRow(Field[] itemFields, Item item) throws IllegalAccessException {
        for (var field : itemFields) {
            printItemFieldValue(item, field);
        }
    }

    private void printItemFieldValue(Item item, Field field) throws IllegalAccessException {
        field.setAccessible(true);
        var fieldValue = field.get(item);
        if (fieldValue instanceof Float moneyValue) {
            printTableCell("$" + moneyValue);
        } else {
            printTableCell(fieldValue.toString());
        }
    }

    private void printTableCell(String item) {
        var padding = TABLE_ITEM_SIZE - item.length();
        var paddingLeft = padding / 2;
        var paddingRight = padding - paddingLeft;
        printer.printf("|" + " ".repeat(paddingLeft) + "%s" + " ".repeat(paddingRight), item);
    }

}

The printTableItems method is much simpler to understand now that it reads as: "for each item in the items, print the row and a new table line".

One thing that felt unnatural at first is how we are using so many small methods that are not really reusable (a bit of a mindset shift of methods being reusable chunks of logic). However, being able to use these small methods almost as documentation of larger methods makes this mindset shift worth it in my opinion.

Conclusion

With that, I hope you got a good understanding of the stepdown rule and how you can use it to improve your code style. After reading this, I challenge you to start implementing this rule yourself and see how much of a difference it can make in making your code as pretty as it can be. Happy coding!

Bringing the Pieces Together

In part 1 of this series, I focused on why I started this migration and how I separated my code generation services within my API server into AWS Lambda functions. At that point, what I didn't figure out yet was how to bring everything together by aggregating their results. This would involve waiting for Kafka messages containing the generated code, saving the generated code to a zip file, and implementing retry and timeout mechanisms in case something goes wrong with the code generation.

Awaiting the Kafka Messages

To await the Kafka messages containing generated code, I implemented a strategy using CountDownLatch objects. These allow you to block while awaiting a countdown to finish. The algorithm goes as follows:

1. From the user's CodeGenerationRequest, get the number of classes we need to generate using our Lambdas.

2. Using this number, initialize a CountDownLatch for us to count down all the classes we generate. This will be stored in a map with a correlation ID for the project as the key, and the CountDownLatch as the value.

3. Send requests to generate the entities, controller advice, and repositories with the correlation ID in the message. The Lambdas handling the entity and repository generation will produce Kafka messages to generate the other classes (i.e., DTOs, services, controllers). All generated classes will echo back the correlation ID, so we can group them as part of the same project.

4. Await for the countdown to finish and return the generated classes using CompleteableFuture#supplyAsync. This completable future will return the generated classes once they are all completed.

5. Consume messages from the generated-class topic on the API server. For each class we receive, we fetch the appropriate CountDownLatch (based on the correlation ID) and call the CountDownLatch#countDown method to advance the countdown. We'll also store the generated classes into map that we can use to keep track of and return the final result.

6. Once the countdown is over, fetch the generated classes from the Map.

7. Write the generated classes into the zip file and return it.

Each time a user requests a project be generated, we run this algorithm on a separate thread that writes the zip file to a temporary storage location. While this is going on, they can call a GET /status/{correlationID} endpoint to get the status of the project generation and later call a GET /project/{correlationID} endpoint to get the final zip contents.

Resilience And Error Handling

With that algorithm figured out, now comes the issue of error handling, which can be tricky in this distributed environment. This is where Resilience4J can come to play. As mentioned above, we await the final code generation result in the API server using a CompletableFuture. Using Resilience4J, we can configure retry logic if the CompletableFuture times out with a TimeoutException.

Aside from that, we need a way to handle errors from the code generation Lambdas. Each of them sends messages (containing a project correlation ID) to an error topic if any exception happens. When the API server consumes a message from this topic, it will complete the countdown early and clear the generated classes. When the CompletableFuture completes (since the countdown is finished), we check if the expected number of classes is in the result. If not, then we throw an exception for us to retry on using our Resilience4J retry configuration.

Idempotency

While these two approaches sound great and straightforward, one thing we need to be aware of with this retry logic is the possibility of duplicate class entries. Let's say that we get an error from one of our Lambdas and need to retry the entire code generation flow. At the same time, there may be a Lambda still processing for the previous attempt that sends a message with its generated class result. In that case, because of the timing of things, there's a possibility of getting duplicate classes: one from the previous code generation attempt, and one from the retry.

To prevent this from happening, I decided to store the generated classes in a Map<String, Map<String, GeneratedClass>> data structure where the key is the project correlation ID, the key of the inner map is the name of the class, and the value of the inner map is the generated class itself. This way, we can ensure idempotency in that when we insert generated classes for a project, if we find a duplicate class name, we will override the previously saved result.

Re-deploying Our API Server

With all these changes added, one further problem arises when it comes to our API Server deployment. Previously, I had deployed my API server using Heroku. While it worked fairly well for the previous architecture, because Heroku doesn't keep the API server always-on, we might not always be ready to consume our Kafka messages. Thus, I decided I need to migrate my API Server to an always-on VPS. For this purpose, I chose Contabo as an affordable option that I can easily set up and connect to. From there, to deploy the API server, I:

1. Wrote a new Dockerfile to build the Docker image on the VPS

2. Logged in to the Contabo VPS as the root user

3. Cloned my Git repository containing my backend

4. Created a .env file containing the production secrets (ensuring it is ignored by Git)

5. Wrote a Docker Compose file to set the .env file, set the volumes, and handle the port mapping

6. Started the API server using the Docker Compose CLI tool.

7. Added a DNS A record for the VPS using my domain provider's management tool. This will map the backend to an api.springforgeapp.com domain.

8. Set up an Nginx reverse proxy on the VPS to forward HTTP requests to my API.

9. Used the nginx certbot tool to set up SSL.

What I Could Still Improve

While this migration was officially a success, there are still some things I can improve on from this point:

• Creating Proper CI/CD Pipelines: Instead of manually building and uploading jars to AWS Lambda for my code generation services, I should find a way to use GitHub actions to automate this when I merge a branch into main. I could also automate copying my backend code to the VPS instead of logging in and pulling from GitHub every time.

• Cold Start Optimization: While the new architecture gives me scalability and maintainability gains, when the AWS Lambdas cold start, the code generation process is noticeably a lot slower (although on warm starts it performs quite well). To counteract this, I need to research configuration options to reduce the impact of cold starts while staying on budget.

• Consider Using Redis: For the temporary storage of generated classes of a project, I could potentially use Redis to avoid using up server memory. Although I should only consider this in the future as a possible optimization if I have more of a user base.

• Research and Profile Other Resilience Strategies: While I have a working resilience strategy, I need to see if it works well in production and if there are other resilience mechanisms I can add, like a Bulkhead.

Conclusion

In this blog post, we went over how I completed my monolith-to-microservice journey by creating a working and resilient algorithm for aggregating distributed processing from my microservices. Using CountDownLatch objects, we can await the expected number of results we need from our microservices. With Resilience4J, we can create retry logic to retry our distributed processing if any error or unexpected result occurs. Lastly, using Docker, we can easily set up an API Server deployment on a cheap, always-on VPS like those provided by Contabo. With that, I hope you found my monolith-to-microservice journey interesting and useful for your future projects. Happy coding!

Uncovering the Monolith

To understand the methods and motivations behind this architectural shift, I first need you to understand how the current monolithic approach to my application works. The magic behind SpringForge involves a group of “generator” service classes, each responsible for generating code for different components of a Spring Application (i.e., controllers, services, entities, etc). These service classes operate together under the same Spring Application, are dependent on code generated by each other, and are orchestrated by a single APIGenerationService. The architecture behind this approach looks like so (with some details being left out for simplicity):

Monolithic Struggles

From the architectural diagram, it becomes immediately clear that orchestrating all this in code (especially in a single service) is cumbersome and messy. I can say that the code for this confirms it, with there being many moving parts and tight coupling that make it hard to write cleanly and maintain.

If we consider scalability and parallelism, there are further issues this design brings. For context, I noticed that this workflow can be parallelized per entity. From the architecture, you’ll see that classes get generated from Entity → DTO → Repository → Service → Controller for each entity being generated. Therefore, we can run this flow in parallel per entity in the project. The issue with this, though, is that since all these services fall under the same Spring application, I can see there being limits to parallelism based on the servlet thread pool used to handle HTTP requests. Aside from that issue, we also can’t scale the services independently from the controller entry point and each other, depending on the load on each of these components.

In Comes Microservices

From these issues, I got the idea of creating an event-driven microservice architecture as my solution. Each service will be its own deployable unit, responsible for generating classes and requesting that other classes be generated based on its results. When the user wants to create a project, events will be sent to generate the entities, DTOs, repositories, services, and lastly, controllers. To group these generated classes as a single user’s project, each message will contain a correlation ID that the API server will use to aggregate the results into a project zip. Finally, to handle the exchange of these messages, I’ve decided to use Apache Kafka (with Avro for message schemas) for durable and reliable message queueing, and to improve my skills with it for my current job.

With this new approach, we can directly solve many of our issues in the monolithic architecture. By having separately deployable services per code generator, each can have its own threads/processors, greatly improving scalability and flexibility. With separate codebases per service, we also have better modularity and development flexibility on how these services work (e.g., if I wanted to migrate them to a different language). Lastly, with some orchestration handled within each microservice, I can potentially clean up much of that messy logic in the API.

The new architecture would look more like this:

The Implementation

Tools

In my work-in-progress implementation of this new architecture, I had to learn and use many tools and services, including:

• AWS Lambda for easy microservice deployments with auto-scaling functionality

• A Kafka Cluster hosted on Cloud Clusters for a reasonably priced messaging solution

• Apache Avro for message formats, serialization, and deserialization

• Digital Ocean for hosting a VM running an open-source Avro schema registry

The Refactoring Processes

The refactoring process has been a long journey, but with some good processes to guide me along the way, it has been fairly smooth despite the tedious effort required. For each generator service, I:

1. Created a new Maven module

2. Copied the service class to the new module along with its tests

3. Created Avro files with new event messages containing the required data

4. Generated classes from my Avro files using a Maven plugin

5. Refactored the code to use the new Avro-generated classes

6. Ensured the tests for the service still passed

7. Added message handling logic for the new microservice

8. Deployed the service to AWS Lambda with the necessary Kafka connection properties

9. Tested sending and receiving messages from the service.

To help with this process, I created a generator-common module, which provides abstractions for many of the common message handling tasks for the microservices. Additionally, I wrote simple classes to test sending and receiving Kafka messages from them.

What's Next

To finish this refactor, I still have some work to do when it comes to connecting my microservices through messages containing their processing results. I also need to put it all together in the original Spring application by sending the initial Kafka message and aggregating the final results. Lastly, I need to add some forms of fault tolerance (possibly with something like Resilience4j) to this entire workflow. For example, if there is a transient communication error or an exception, I need to have an error topic for the API to report errors or retry some of the code generation.

Conclusion

With that, I hope I gave you great insights from my microservice migration journey, including the architectural decisions, tools, and workflow that’s making this migration successful so far. Stay tuned for more about this migration in a future blog post. Happy coding!

Project Inspiration

After my project, Quak Framework, a Spring Framework clone with some of my own twists on things, I reflected a lot on the potential impact of the project. One thing that bothered me was that I felt like I couldn’t make a significant impact with Quak since it was unrealistic for me to create a viable alternative to Spring. Instead, I wanted to build something that could be a useful tool for Spring developers, which seemed like a more approachable angle to build something with greater impact.

The Eureka

I was considering the code-generation aspect of Quak Framework and thought of doing something similar, more sophisticated, and for the Spring Framework instead. The initial idea was to build a CLI tool (similar to the Angular CLI) for generating Spring CRUD logic (pretty close to the final idea). However, a few things caused me to pivot on that idea:

1. It’s been a while since I’ve worked with Angular, and I wanted to build something nice to refresh on it and show on my portfolio.

2. I recently took a software engineering course in the final year of my undergraduate degree, and was fascinated by the idea of model-driven engineering, which was brought up.

3. A couple of years ago, I worked on a school project involving editing graphs on a web page, and thought I could reuse some of that experience.

That's when I got my final idea for SpringForge: create an editor for ER-like diagrams, serialize it as JSON for a backend, and generate Spring CRUD apps from it!

Introducing SpringForge 🚀

From that eureka, my idea for SpringForge was born. In the initial release, the app featured:

🧠 Smart code generation for entities, DTOs, JPA repositories, and REST controllers;

🤖 AI-powered unit test generation;

🖼️ and a polished and intuitive UI.

Through that, I hope to automate some of the annoying boilerplate of coding CRUD application logic with Spring.

Tools & Tech Stack

To build the application, we used the following tech stack and tools:

Backend

• Spring Framework for implementing the backend endpoints

• Firebase for user authentication, NoSQL document-based database storage, and file storage buckets

• JavaPoet for code generation

Frontend

• Angular for implementing the frontend logic and UI

Deployment

• Heroku for the backend deployment platform

• Docker for easier deployment setup on Heroku

• Firebase for the frontend deployment

CI

• GitHub actions for running our CI pipelines

• Codecov for managing code coverage reports and displaying them within GitHub

Challenges

While the four months of development for the initial release have been a fun experience, it wasn't without major challenges. I thought I'd share two of the major ones so you can avoid some of the mistakes I’ve made and learn from what went well here.

Unit Testing

One of the biggest challenges of building this project was unit testing the code generation functionality. While I was able to achieve around 80% backend code coverage, I think I could have made some of the tests more useful than they were. A good portion of the assertions involve creating expected TypeSpec instances with JavaPoet using repeated logic as the code itself. In the future, I'd like to refactor these tests to something more useful, like asserting a string with the expected code instead. As time went by, however, I would say I wrote some better tests with this improvement in mind.

Outside of the backend tests for code generation, I also wrote tests for components using the Firebase Admin API. These posed their own unique challenges of a complex mock setup to avoid actually invoking Firebase. This involved using static mocking for singleton calls, setting up a ton of mocks for chained calls involving collections, documents, and data snapshots, and verifying method calls were made on those mocks. While this was often tedious and complicated, I'm really happy with how those tests turned out.

Lastly, for frontend unit testing, the biggest regret was not writing frontend tests sooner. Here, I was a little eager working on the frontend, not too experienced with frontend testing, and intimidated by the thought of writing frontend tests for this, so I put it off for a lot longer than I should have. After a while, I decided to take a big leap at adding unit tests, realizing that it wasn't as complex as I thought and that I should take the correctness of the frontend more seriously. While I made good progress on that, the frontend code coverage is still not ideal, and if I had started sooner (i.e., with test-driven development or at least loosely following it), it would be a lot better.

UI Development

One of the biggest challenges that Elli, my UI/UX designer and frontend developer, faced was working with the Angular Material UI library. Initially, I had built prototypes for the UI using this library because of my prior experience with it and its relative simplicity. However, when she started refining the UI and customizing it, she came across challenges working with it. Angular Material, while fairly easy to use for pretty components out of the box, can be fairly hard to customize at times, and the documentation has its own problems. In hindsight, it might have been better if we explored our options first when it came to ready-made components to avoid difficulties like these.

Biggest Takeaways

Finally, I'd like to share the biggest takeaways from this project that you can use to bring your future ideas to life:

1. Build Something That Pushes Your Skills

One thing I always strive for when developing a project idea is to choose something that can further develop my skills. Whether it’s by choosing technologies you want experience with or pushing for something more ambitious than ever, always try to build something that will grow you as a developer, not necessarily something you already know you can build.

2. Inspiration Can Come From Anywhere

As I touched on in the section about my inspiration for the project, you can find inspiration from many places. By reflecting hard on what interests you, what you've done previously, and what you can do to make an impact, you can come up with some awesome ideas for your next project, too.

3. Try to Code More With The Future In Mind

Some of the major mistakes I've made in this project came from just committing to the first solution that came to mind and not stopping to evaluate or improve things first. Had I taken the extra effort in the short term, I could have potentially saved myself from technical debt, future headaches, and wasted time.

What’s Next For SpringForge

While I haven’t touched the project in quite some time, I can promise that changes will come this year. As of now, I’ve already put some major work forward for:

1. Refactoring the backend to a microservice architecture (more on that soon)

2. Support for request validation constraints

3. A new AI-related feature

Conclusion

With that, I hope that gave you good insight into my project, inspired you on your project journey, and gave you some knowledge to consider in your current or future side projects. If you want to try my application for yourself, you can find it at www.springforgeapp.com. Happy coding!

What Is JavaPoet?

JavaPoet is a library that makes programmatically generating Java source files easy and has been incredibly nice to use in a project I’m currently working on. Using mostly builders to form the different language constructs, you can build complex Java code using semantically elegant statements and fairly limited string formatting. Because of this, a lot of the code should be straightforward once I start giving you examples of it in practice!

NOTE: the original JavaPoet library by Square Inc. is now deprecated so it won’t be actively maintained and updated with the latest Java features. However, this library is still very usable and there is a fork being maintained by Palantir which contains some of the more recent Java features like records.

What We’ll Do With It

To give you a good feel for the library, we’ll generate a class that’s simple yet contains a variety of Java constructs for us to build. We’ll be writing code to replicate this class below:

package io.john.amiscaray;

import java.util.ArrayList;
import java.util.List;

public class ShoppingList {

    private final List<String> items = new ArrayList<>();
    private int capacity;

    public ShoppingList(int capacity) {
        this.capacity = capacity;
    }

    public List<String> getItems() {
        return items;
    }

    public void setCapacity(int capacity) {
        this.capacity = capacity;
    }

    public int getCapacity() {
        return capacity;
    }

    public void addItem(String item) {
        if (items.size() >= capacity) {
            return;
        }
        items.add(item);
    }

    public void printItems() {
        for (var item : items) {
            System.out.println(item);
        }
    }
}

With different methods and block scopes, this should have different things for us to try out, yet be pretty straightforward for you to pick up.

Creating the Class

First, let me show you how we can create the class declaration without any fields or methods and print out the output. As I alluded to earlier, JavaPoet provides us with a lot of builders for generating Java code constructs, including classes:

package io.john.amiscaray;

import com.squareup.javapoet.TypeSpec;
import com.squareup.javapoet.JavaFile;

import javax.lang.model.element.Modifier;

public class Main {

    public static void main(String[] args) {
        var shoppingListClass = TypeSpec.classBuilder("ShoppingList")
                .addModifiers(Modifier.PUBLIC)
                .build();

        System.out.println(JavaFile.builder("io.john.amiscaray", shoppingListClass).build());
    }

}

This code would have the following output:

package io.john.amiscaray;

public class ShoppingList {
}

Adding Fields

Now, we can add our fields to this class using the additional builder methods of the TypeSpec.Builder class:

package io.john.amiscaray;

import com.squareup.javapoet.*;

import javax.lang.model.element.Modifier;
import java.util.ArrayList;
import java.util.List;

public class Main {

    public static void main(String[] args) {
        var shoppingListClass = TypeSpec.classBuilder("ShoppingList")
                .addField(
                        FieldSpec.builder(ParameterizedTypeName.get(List.class, String.class), "items")
                                .addModifiers(Modifier.PRIVATE, Modifier.FINAL)
                                .initializer("new $T()", ArrayList.class)
                                .build()
                )
                .addField(
                        FieldSpec.builder(ClassName.INT, "capacity")
                                .addModifiers(Modifier.PRIVATE)
                                .build()
                )
                .addModifiers(Modifier.PUBLIC)
                .build();

        System.out.println(JavaFile.builder("io.john.amiscaray", shoppingListClass).build());
    }

}

Looking at the first TypeSpec.Builder#addField call, we create a new field using the FieldSpec.Builder class returned from the FieldSpec.Builder#builder method call. This method takes the type of the field and its name. In this case, we are specifying a List<String> field named items. Then we add the private and final keywords to the field and initialize it as a new ArrayList<>(). The FieldSpec.Builder#initializer method takes a format string for the initial value of the field along with format arguments. In this case, the $T token means a type name format argument, and the ArrayList.class statement is the value we are using for it. At this point, the output looks like this:

package io.john.amiscaray;

import java.lang.String;
import java.util.ArrayList;
import java.util.List;

public class ShoppingList {
  private final List<String> items = new ArrayList();

  private int capacity;
}

Adding a Constructor, Getters, And Setters

Now, let’s add a constructor:

package io.john.amiscaray;

import com.squareup.javapoet.*;

import javax.lang.model.element.Modifier;
import java.util.ArrayList;
import java.util.List;

public class Main {

    public static void main(String[] args) {
        var shoppingListClass = TypeSpec.classBuilder("ShoppingList")
                .addField(
                        FieldSpec.builder(ParameterizedTypeName.get(List.class, String.class), "items")
                                .addModifiers(Modifier.PRIVATE, Modifier.FINAL)
                                .initializer("new $T()", ArrayList.class)
                                .build()
                )
                .addField(
                        FieldSpec.builder(ClassName.INT, "capacity")
                                .addModifiers(Modifier.PRIVATE)
                                .build()
                )
                .addMethod(MethodSpec
                        .constructorBuilder()
                        .addModifiers(Modifier.PUBLIC)
                        .addParameter(ClassName.INT, "capacity")
                        .addStatement("this.capacity = capacity")
                        .build()
                )
                .addModifiers(Modifier.PUBLIC)
                .build();

        System.out.println(JavaFile.builder("io.john.amiscaray", shoppingListClass).build());
    }

}

Here, we call TypeSpec.Builder#addMethod with a MethodSpec created from a builder. You’ll notice that with the MethodSpec.Builder we call MethodSpec.Builder#addStatement to add code within the constructor body.

Similarly, we can make additional TypeSpec.Builder#addMethod calls to generate our getters and setters:

package io.john.amiscaray;

import com.squareup.javapoet.*;

import javax.lang.model.element.Modifier;
import java.util.ArrayList;
import java.util.List;

public class Main {

    public static void main(String[] args) {
        var shoppingListClass = TypeSpec.classBuilder("ShoppingList")
                .addField(
                        FieldSpec.builder(ParameterizedTypeName.get(List.class, String.class), "items")
                                .addModifiers(Modifier.PRIVATE, Modifier.FINAL)
                                .initializer("new $T()", ArrayList.class)
                                .build()
                )
                .addField(
                        FieldSpec.builder(ClassName.INT, "capacity")
                                .addModifiers(Modifier.PRIVATE)
                                .build()
                )
                .addMethod(MethodSpec
                        .constructorBuilder()
                        .addModifiers(Modifier.PUBLIC)
                        .addParameter(ClassName.INT, "capacity")
                        .addStatement("this.capacity = capacity")
                        .build()
                )
                .addMethod(MethodSpec
                        .methodBuilder("getItems")
                        .addModifiers(Modifier.PUBLIC)
                        .returns(ParameterizedTypeName.get(List.class, String.class))
                        .addStatement("return items")
                        .build()
                )
                .addMethod(MethodSpec.methodBuilder("getCapacity")
                        .addModifiers(Modifier.PUBLIC)
                        .returns(ClassName.INT)
                        .addStatement("return capacity")
                        .build())
                .addMethod(MethodSpec.methodBuilder("setCapacity")
                        .addModifiers(Modifier.PUBLIC)
                        .addParameter(TypeName.INT, "capacity")
                        .addStatement("this.capacity = capacity")
                        .build())
                .addModifiers(Modifier.PUBLIC)
                .build();

        System.out.println(JavaFile.builder("io.john.amiscaray", shoppingListClass).build());
    }

}

Now the code should output the following:

package io.john.amiscaray;

import java.lang.String;
import java.util.ArrayList;
import java.util.List;

public class ShoppingList {
  private final List<String> items = new ArrayList();

  private int capacity;

  public ShoppingList(int capacity) {
    this.capacity = capacity;
  }

  public List<String> getItems() {
    return items;
  }

  public int getCapacity() {
    return capacity;
  }

  public void setCapacity(int capacity) {
    this.capacity = capacity;
  }
}

Implementing Our Last Methods

Now, all we have to implement are the ShoppingList#addItem and ShoppingList#printItems methods. The following is the final code after implementing them:

package io.john.amiscaray;

import com.squareup.javapoet.*;

import javax.lang.model.element.Modifier;
import java.util.ArrayList;
import java.util.List;

public class Main {

    public static void main(String[] args) {
        var shoppingListClass = TypeSpec.classBuilder("ShoppingList")
                .addField(
                        FieldSpec.builder(ParameterizedTypeName.get(List.class, String.class), "items")
                                .addModifiers(Modifier.PRIVATE, Modifier.FINAL)
                                .initializer("new $T()", ArrayList.class)
                                .build()
                )
                .addField(
                        FieldSpec.builder(ClassName.INT, "capacity")
                                .addModifiers(Modifier.PRIVATE)
                                .build()
                )
                .addMethod(MethodSpec
                        .constructorBuilder()
                        .addModifiers(Modifier.PUBLIC)
                        .addParameter(ClassName.INT, "capacity")
                        .addStatement("this.capacity = capacity")
                        .build()
                )
                .addMethod(MethodSpec
                        .methodBuilder("getItems")
                        .addModifiers(Modifier.PUBLIC)
                        .returns(ParameterizedTypeName.get(List.class, String.class))
                        .addStatement("return items")
                        .build()
                )
                .addMethod(MethodSpec.methodBuilder("getCapacity")
                        .addModifiers(Modifier.PUBLIC)
                        .returns(ClassName.INT)
                        .addStatement("return capacity")
                        .build())
                .addMethod(MethodSpec.methodBuilder("setCapacity")
                        .addModifiers(Modifier.PUBLIC)
                        .addParameter(TypeName.INT, "capacity")
                        .addStatement("this.capacity = capacity")
                        .build())
                .addMethod(MethodSpec.methodBuilder("addItem")
                        .addModifiers(Modifier.PUBLIC)
                        .addParameter(String.class, "item")
                        .beginControlFlow("if (items.size() >= capacity)")
                        .addStatement("return")
                        .endControlFlow()
                        .addStatement("items.add(item)")
                        .build())
                .addMethod(MethodSpec.methodBuilder("printItems")
                        .addModifiers(Modifier.PUBLIC)
                        .beginControlFlow("for (var item : items)")
                        .addStatement("System.out.println(item)")
                        .endControlFlow()
                        .build())
                .addModifiers(Modifier.PUBLIC)
                .build();

        System.out.println(JavaFile.builder("io.john.amiscaray", shoppingListClass).build());
    }

}

You’ll notice that we can create control flows (i.e., if or for statements) using the MethodSpec.Builder#beginControlFlow method. Any statements after this method call get added within the new block scope until we call MethodSpec.Builder#endControlFlow. With that, the final code should look like so:

package io.john.amiscaray;

import java.lang.String;
import java.util.ArrayList;
import java.util.List;

public class ShoppingList {
  private final List<String> items = new ArrayList();

  private int capacity;

  public ShoppingList(int capacity) {
    this.capacity = capacity;
  }

  public List<String> getItems() {
    return items;
  }

  public int getCapacity() {
    return capacity;
  }

  public void setCapacity(int capacity) {
    this.capacity = capacity;
  }

  public void addItem(String item) {
    if (items.size() >= capacity) {
      return;
    }
    items.add(item);
  }

  public void printItems() {
    for (var item : items) {
      System.out.println(item);
    }
  }
}

Code Indentation

Lastly, you probably realized that the code we’ve been outputting isn’t well indented. We can quickly fix this by altering the line where we output the file:

System.out.println(JavaFile.builder("io.john.amiscaray", shoppingListClass).indent("    ").build());

Conclusion

With that, you should have a very good idea of how you can use the JavaPoet library to write beautiful code that generates Java files. Try it out yourself and build something awesome with it!

Prerequisites

To follow along with this guide, you should:

1. Be able to follow along with the previous microservice tutorials in this blog (and ideally have read them)

2. Understand the basics of microservices

3. Understand the concept of configuration servers in a microservice architecture

4. Know how JWT works, including its structure and the theory behind it

5. Know about HTTP basic authentication

For this guide, we’ll be starting from the code we left off at the previous blog post. If you need a refresher on the code we’ll be working with, check it out on GitHub.

Background

If you remember back to my first post, we had implemented some communication between microservices to sync replicated data between services:

Course Service

@RestController
@RequestMapping("student")
@AllArgsConstructor
public class StudentController {

    private StudentService studentService;

    @PostMapping
    public ResponseEntity<Void> addStudent(@RequestBody SaveStudentRequest student) {
        studentService.saveStudent(student);

        return ResponseEntity.noContent().build();
    }

}

Single endpoint for the Student Service microservice to sync its student records with the Course Service

Student Service

@RestController
@RequestMapping("/student")
@AllArgsConstructor
public class StudentController {

    // Fields and other endpoints up here...

    @PostMapping("")
    public ResponseEntity<Void> saveStudent(@RequestBody StudentDTO studentDTO) throws URISyntaxException {
        var studentID = studentService.saveStudent(studentDTO);

        courseServiceClient.saveNewStudent(new CourseSaveStudentRequest(studentID))
                .then()
                .subscribe(); // Do nothing when the request goes through.

        return ResponseEntity.created(new URI("/student/" + studentID)).build();
    }

}

Student Service sends the request to the Course Service after saving a new student

One issue with this code is security; how can we enforce that only our Student Service can call this endpoint? To do this, we need to implement some form of authentication and authorization to verify that the client is our Student Service. This is where an authentication server will come into play.

Our Authentication Server

In our updated application, we'll make use of the functionality Spring provides to create an authentication server. Using the spring-boot-starter-oauth2-authorization-server dependency, we can set up a microservice for managing JWT-based authorization. This includes:

1. configuring registered clients, their passwords (secrets), authorities, and more;

2. generating JWTs with a private key;

3. issuing JWTs; and

4. providing public keys for microservices to verify issued JWTs.

Much of this functionality would be implemented for us by Spring, and all we'd need to do is write configurations. After it’s all set up, we can then configure our Course Service to require authentication for the /student endpoint. When our Student Service wants to send a request to this endpoint, it would first need to send a request to the Auth Service to obtain a JWT it will add to the authorization header.

Implementing the Auth Service

Now that you understand how our Auth Service will work, let's dive into the implementation! First, let's add a new Maven module to our application. The pom.xml for the new module should look like so:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>io.john.amiscaray</groupId>
        <artifactId>MicroservicesClassroomExample</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <artifactId>AuthService</artifactId>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-security</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-oauth2-authorization-server</artifactId>
        </dependency>
    </dependencies>

    <properties>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

</project>

From there, we can add the basic properties we need for our microservice to register itself with Consul (our service registry) and connect to the config server:

spring.application.name=auth-service
server.port=8083
spring.cloud.consul.host=localhost
spring.cloud.consul.port=8500
spring.config.import=configserver:
spring.cloud.config.discovery.service-id=config-service
spring.cloud.config.discovery.enabled=true

and add our main method:

package io.john.amiscaray;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class AuthServerApplication {

    public static void main(String[] args) {
        SpringApplication.run(AuthServerApplication.class, args);
    }

}

With that, we need to configure this microservice as an authentication server. We'll first create the required configuration class and set up the public/private key pair we discussed above:

@Configuration
public class AuthorizationServerConfig {
    @Bean
    public JWKSource<SecurityContext> jwkSource() {
        RSAKey rsaKey = generateRsa(); // generate a key pair
        JWKSet jwkSet = new JWKSet(rsaKey);
        return (jwkSelector, context) -> jwkSelector.select(jwkSet);
    }

    private RSAKey generateRsa() {
        KeyPair keyPair = KeyGeneratorUtils.generateRsaKey();
        return new RSAKey.Builder((RSAPublicKey) keyPair.getPublic())
                .privateKey((RSAPrivateKey) keyPair.getPrivate())
                .keyID(UUID.randomUUID().toString())
                .build();
    }
}

package io.john.amiscaray.util;

import java.security.KeyPair;
import java.security.KeyPairGenerator;

public final class KeyGeneratorUtils {

    private KeyGeneratorUtils() {}

    public static KeyPair generateRsaKey() {
        try {
            KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance("RSA");
            keyPairGenerator.initialize(2048);
            return keyPairGenerator.generateKeyPair();
        } catch (Exception ex) {
            throw new IllegalStateException(ex);
        }
    }
}

This code generates an RSA public/private key pair and uses it to create a JWKSource bean. Spring will use this bean to generate JWTs using the private key and share the public key using an endpoint of this Auth Service.

Next, we need to create another bean for some configuration for our Auth Server:

@Bean
public AuthorizationServerSettings authorizationServerSettings(@Value("${auth.issuer-uri}") String issuerUri) {
    return AuthorizationServerSettings.builder()
            .issuer(issuerUri)
            .build();
}

along with some new configuration:

# Other properties go here...
# Add the property below to our application.properties; this is the value of the issuerUri argument for our bean
auth.issuer-uri=http://localhost:8083

This sets the issuer field for our JWTs to the URI of our Auth Server. Without this configuration, you'd run into 401 unauthorized errors due to an invalid issuer.

Lastly, we need a couple more beans for a repository of registered clients and a password encoder to encode their passwords. Let's start with the easier of the two, the password encoder:

@Bean
public PasswordEncoder secretEncoder() {
    return new BCryptPasswordEncoder();
}

Then, we can define a RegisteredClientRepository bean where we can register our Student Service as a client:

@Bean
public RegisteredClientRepository registeredClientRepository(
        PasswordEncoder passwordEncoder,
        @Value("${client.student-service.client-secret}") String rawSecret
) {
    var encodedSecret = passwordEncoder.encode(rawSecret);

    var registeredClient = RegisteredClient.withId(UUID.randomUUID().toString())
            .clientId("student-service")
            .clientSecret(encodedSecret)
            .clientAuthenticationMethod(ClientAuthenticationMethod.CLIENT_SECRET_BASIC)
            .authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
            .scope("write:student")
            .build();

    return new InMemoryRegisteredClientRepository(registeredClient);
}

add auth-service.properties in our config repository (for our config server) with the client secret:

# auth-service/auth-service.properties from the configuration git repository
client.student-service.client-secret=${CLIENT_SECRET}

and update our config server configuration with a search path for the location of the auth-service.properties files

# Config Server application.properties
# Other properties above...
# Updated Config Server search path:
spring.cloud.config.server.git.search-paths[2]=auth-service

Above, in our registeredClientRepository method, we register our Student Service as a client with a random UUID, password/secret (added in our configuration repository as an environment variable), and a write:student permission/scope. Additionally, we configured it so that the Student Service can authenticate with this service using HTTP basic authentication to retrieve the JWT as client credentials.

With that configuration, our Student Service would have to send a POST request to this Auth Service to a generated /oauth2/token endpoint to request the JWT token. In the body, the request will have information that the Student service wants client credentials (a JWT) and to request the write:student permission for the /student endpoint. To authenticate the request, the Student Service would add an Authorization header (following the HTTP basic authentication scheme) with the above client ID and client secret as the username and password, respectively.

For your reference, the final configuration class should look like so:

package io.john.amiscaray.config;

import com.nimbusds.jose.jwk.JWKSet;
import com.nimbusds.jose.jwk.RSAKey;
import com.nimbusds.jose.jwk.source.JWKSource;
import com.nimbusds.jose.proc.SecurityContext;
import io.john.amiscaray.util.KeyGeneratorUtils;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
import org.springframework.security.crypto.password.PasswordEncoder;
import org.springframework.security.oauth2.core.AuthorizationGrantType;
import org.springframework.security.oauth2.core.ClientAuthenticationMethod;
import org.springframework.security.oauth2.server.authorization.client.InMemoryRegisteredClientRepository;
import org.springframework.security.oauth2.server.authorization.client.RegisteredClient;
import org.springframework.security.oauth2.server.authorization.client.RegisteredClientRepository;
import org.springframework.security.oauth2.server.authorization.settings.AuthorizationServerSettings;

import java.security.KeyPair;
import java.security.interfaces.RSAPrivateKey;
import java.security.interfaces.RSAPublicKey;
import java.util.UUID;

@Configuration
public class AuthorizationServerConfig {

    @Bean
    public RegisteredClientRepository registeredClientRepository(
            PasswordEncoder passwordEncoder,
            @Value("${client.student-service.client-secret}") String rawSecret
    ) {
        var encodedSecret = passwordEncoder.encode(rawSecret);

        var registeredClient = RegisteredClient.withId(UUID.randomUUID().toString())
                .clientId("student-service")
                .clientSecret(encodedSecret)
                .clientAuthenticationMethod(ClientAuthenticationMethod.CLIENT_SECRET_BASIC)
                .authorizationGrantType(AuthorizationGrantType.CLIENT_CREDENTIALS)
                .scope("write:student")
                .build();

        return new InMemoryRegisteredClientRepository(registeredClient);
    }

    @Bean
    public JWKSource<SecurityContext> jwkSource() {
        RSAKey rsaKey = generateRsa(); // generate a key pair
        JWKSet jwkSet = new JWKSet(rsaKey);
        return (jwkSelector, context) -> jwkSelector.select(jwkSet);
    }

    private RSAKey generateRsa() {
        KeyPair keyPair = KeyGeneratorUtils.generateRsaKey();
        return new RSAKey.Builder((RSAPublicKey) keyPair.getPublic())
                .privateKey((RSAPrivateKey) keyPair.getPrivate())
                .keyID(UUID.randomUUID().toString())
                .build();
    }

    @Bean
    public AuthorizationServerSettings authorizationServerSettings(@Value("${auth.issuer-uri}") String issuerUri) {
        return AuthorizationServerSettings.builder()
                .issuer(issuerUri)
                .build();
    }

    @Bean
    public PasswordEncoder secretEncoder() {
        return new BCryptPasswordEncoder();
    }

}

Note above that with the CLIENT_SECRET environment variable, the property would be retrieved from the machine that the Auth Service is running on, not from the config server.

Updating the Course Service

Now that our Auth Service is complete, we need to make some changes to our Course Service to secure the /student endpoint. To begin, we need to add the following dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-security</artifactId>
</dependency>
 <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-oauth2-resource-server</artifactId>
</dependency>

As you'd expect, we now need Spring Security to secure our /student endpoint. However, what's more interesting here is the spring-boot-starter-oauth2-resource-server dependency. This is required for us to use the Auth Service.

Next, we can configure Spring Security for our service to require authentication for our /student endpoint:

package io.john.amiscaray.cfg;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.security.config.Customizer;
import org.springframework.security.config.web.server.ServerHttpSecurity;
import org.springframework.security.web.server.SecurityWebFilterChain;

@Configuration
public class SecurityConfig {

    @Bean
    public SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) {
        http.authorizeExchange(ex -> ex
                        .pathMatchers("/student").authenticated()
                        .anyExchange().permitAll()
                ).oauth2ResourceServer(oauth2 -> oauth2.jwt(Customizer.withDefaults()));

        return http.build();
    }

}

This configuration will require clients calling the /student endpoint to be authenticated using a JWT in their Authorization header while permitting all other requests. Afterward, we need to update our application.properties to configure the URI of our Auth Server:

# Other properties up here...
spring.security.oauth2.resourceserver.jwt.issuer-uri=http://localhost:8083

Here, we need to use the actual hostname and port of our Auth Server since the JWT decoder functionality can't resolve the hostname and port by service name, as we've done in the previous blog post.

Finally, we can update our /student endpoint to require the write:student permission that our Student Service is allowed to have:

@PreAuthorize("hasAuthority('SCOPE_write:student')") // Require the write:student permission
@PostMapping
public ResponseEntity<Void> addStudent(@RequestBody SaveStudentRequest student) {
    studentService.saveStudent(student);

    return ResponseEntity.noContent().build();
}

Updating Our Student Service

Lastly, we need to update our Student Service to retrieve a JWT and apply it as an Authorization header for the /student request. First, we need to create a new AuthServiceClient class to call the Auth Service:

package io.john.amiscaray.http;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import org.springframework.web.reactive.function.BodyInserters;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.core.publisher.Mono;

import java.util.Map;

@Service
public class AuthServiceClient {

    private WebClient.Builder webClientBuilder;
    @Value("${auth.client-secret}")
    private String clientSecret;

    public AuthServiceClient(WebClient.Builder webClientBuilder) {
        this.webClientBuilder = webClientBuilder;
    }

    public Mono<String> getAuthToken() {
        return webClientBuilder.build().post()
                .uri("http://auth-service/oauth2/token")
                .headers(headers -> headers.setBasicAuth("student-service", clientSecret))
                .body(BodyInserters.fromFormData("grant_type", "client_credentials").with("scope", "write:student"))
                .retrieve()
                .bodyToMono(Map.class)
                .map(response -> (String) response.get("access_token"));
    }

}

In the getAuthToken method, we send the post request to the /oauth2/token. Here the http://auth-service URI gets resolved as the host and port of the Auth Service using service discovery. Then, in the headers, we have an HTTP basic authorization header with our service name as the username and client secret as the password. In the body, we have form data specifying that we want to get client credentials (the JWT) with the write:student scope. Finally, we retrieve the body as a Mono (async container) and map it to a Mono<String> containing the JWT from the response body. For the auth.client-secret property, we add it to the git repository that our Config Server fetches from:

auth.client-secret=${CLIENT_SECRET}

Again, like last time, the CLIENT_SECRET environment variable gets fetched from the machine that the Student Service is running on.

Lastly, using this new method, we can update the CourseServiceClient class to add a JWT to the header of the /student request:

package io.john.amiscaray.http;

import io.john.amiscaray.dto.CourseSaveStudentRequest;
import lombok.AllArgsConstructor;
import org.springframework.http.HttpStatusCode;
import org.springframework.stereotype.Service;
import org.springframework.web.reactive.function.client.*;
import reactor.core.publisher.Mono;

import java.util.function.Consumer;

@Service
@AllArgsConstructor
public class CourseServiceClient {

    private WebClient.Builder webClientBuilder;
    private AuthServiceClient authServiceClient;

    public Mono<Void> saveNewStudent(CourseSaveStudentRequest saveStudentRequest) {
        return authServiceClient.getAuthToken()
                .flatMap(authToken -> webClientBuilder
                        .build()
                        .post()
                        .uri("http://course-service/student")
                        .headers(headers -> headers.setBearerAuth(authToken))
                        .bodyValue(saveStudentRequest)
                        .retrieve()
                        .bodyToMono(Void.class));
    }
}

First, we are calling the AuthServiceClient#getAuthToken method to retrieve the Mono containing the JWT. Then we call the Mono#flatMap method to retrieve this JWT and call the /student endpoint with the JWT in the authorization header, returning a new Mono as the result of the API call.

Conclusion

With that, we have successfully implemented authentication and authorization in our microservices application to secure our inter-service communication! I hope you found this interesting and can use this to develop new skills or awesome projects in the future. For future reference, check out the final code here on GitHub.

Prerequisites

To follow along with this guide you need to know the following:

• How to build a simple REST API with Spring Framework
• Basic software testing concepts such as unit testing, mocking, and integration testing
• How to write simple tests with JUnit and Mockito

Also note that throughout the code snippets, we'll be using a lot of static imports as a bit of "syntactic sugar" for the readability of our tests (as commonly done using these test frameworks). If you're ever confused about where a method is from, it's probably from a static import omitted from the code snippet for simplicity. At the end of each section, I'll share the full code for each test class so you can see the static imports.

The Application We Will Be Testing

For this guide, we'll be testing a simple REST API for managing grocery products. Below are the classes of this application that we'll be testing:

ProductController

package io.john.amiscaray.productapi.controller;

import io.john.amiscaray.productapi.dto.ProductDTO;
import io.john.amiscaray.productapi.service.ProductService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

import java.net.URI;
import java.util.Set;

@RestController
@RequestMapping("products")
public class ProductController {

    private final ProductService productService;

    @Autowired
    public ProductController(ProductService productService) {
        this.productService = productService;
    }

    @GetMapping
    public ResponseEntity<Set<ProductDTO>> getAllProducts() {
        return ResponseEntity.ok(productService.getAllProducts());
    }

    @GetMapping("{id}")
    public ResponseEntity<ProductDTO> getProductById(@PathVariable Long id) {
        return ResponseEntity.ok(productService.getProductById(id));
    }

    @GetMapping("apples")
    public ResponseEntity<Set<ProductDTO>> getAppleProducts() {
        return ResponseEntity.ok(productService.getAppleProducts());
    }

    @PostMapping
    public ResponseEntity<Void> createProduct(@RequestBody ProductDTO productDTO) {
        var newProductID = productService.createProduct(productDTO);

        return ResponseEntity.created(URI.create("/products/" + newProductID))
                .build();
    }

    @DeleteMapping("{id}")
    public ResponseEntity<Void> deleteProduct(@PathVariable Long id) {
        productService.deleteProduct(id);

        return ResponseEntity.noContent().build();
    }

}

ControllerErrorHandler

package io.john.amiscaray.productapi.controller;

import lombok.extern.java.Log;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.servlet.resource.NoResourceFoundException;

import java.util.NoSuchElementException;

@ControllerAdvice
@Log
public class ControllerErrorHandler {

    @ExceptionHandler(Exception.class)
    public ResponseEntity<String> handleGenericException(Exception e) {
        log.severe(e.getMessage());

        return new ResponseEntity<>("Unexpected server error", HttpStatus.INTERNAL_SERVER_ERROR);
    }

    @ExceptionHandler(NoResourceFoundException.class)
    public ResponseEntity<String> handleNoResourceFoundException(NoResourceFoundException e) {

        return ResponseEntity.notFound().build();
    }

    @ExceptionHandler(NoSuchElementException.class)
    public ResponseEntity<String> handleNoSuchElementException(NoSuchElementException e) {
        return new ResponseEntity<>(e.getMessage(), HttpStatus.NOT_FOUND);
    }

}

ProductRepository

package io.john.amiscaray.productapi.dao;

import io.john.amiscaray.productapi.data.Product;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

import java.util.Set;

@Repository
public interface ProductRepository extends JpaRepository<Product, Long> {

    Set<Product> findAllByDescriptionContainsIgnoreCase(String substring);

}

ProductService

package io.john.amiscaray.productapi.service;

import io.john.amiscaray.productapi.dao.ProductRepository;
import io.john.amiscaray.productapi.data.Product;
import io.john.amiscaray.productapi.dto.ProductDTO;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.NoSuchElementException;
import java.util.Set;
import java.util.stream.Collectors;

@Service
public class ProductService {

    private final ProductRepository productRepository;

    @Autowired
    public ProductService(ProductRepository productRepository) {
        this.productRepository = productRepository;
    }

    public Set<ProductDTO> getAllProducts() {
        return productRepository.findAll()
                .stream()
                .map(ProductDTO::new)
                .collect(Collectors.toSet());
    }

    public ProductDTO getProductById(Long id) {
        return new ProductDTO(productRepository.findById(id)
                .orElseThrow());
    }

    public Set<ProductDTO> getAppleProducts() {
        return productRepository.findAllByDescriptionContainsIgnoreCase("Apple")
                .stream()
                .map(ProductDTO::new)
                .collect(Collectors.toSet());
    }

    public Long createProduct(ProductDTO productDTO) {
        var detachedProduct = new Product(productDTO);
        var savedProduct = productRepository.save(detachedProduct);

        return savedProduct.getId();
    }

    public void deleteProduct(Long id) {
        if (!productRepository.existsById(id)) {
            throw new NoSuchElementException("Could not find product with id: " + id);
        }

        productRepository.deleteById(id);
    }

}

Unit Testing The Controller Layer

First, let's begin by testing the controller layer (i.e., the ProductController). Our goal for this is to create a light-weight unit test class. Thus, we should mock all dependencies of our controller and test it without actually sending any HTTP requests. To do this, first, we need to figure out the test setup to make this possible.

Test Setup

To begin, let's define our test class. We'll annotate it with @ExtendWith(MockitoExtension.class) to allow us to use Mockito's annotation functionality. As for its fields, we'll need a ProductController along with the ProductService it depends on. Using Mockito's annotations, we can inject a mock for our ProductService and inject it into a constructor for a new ProductController:

@ExtendWith(MockitoExtension.class)
public class ProductControllerTest {

    @InjectMocks // Creates a new ProductController with the mock ProductService added to the constructor
    private ProductController productController;
    @Mock
    private ProductService productService;

}

From there, we'll also need to set up a MockMvc instance to help us mock HTTP requests for our tests. Using the static MockMvcBuilders#standaloneSetup method, we can set up MockMvc to work with only our ProductController and our ControllerErrorHandler for when we test error conditions:

@ExtendWith(MockitoExtension.class)
public class ProductControllerTest {

    @InjectMocks
    private ProductController productController;
    @Mock
    private ProductService productService;
    private MockMvc mockMvc;

    @BeforeEach
    public void setUp() {
        ControllerErrorHandler controllerErrorHandler = new ControllerErrorHandler();
        mockMvc = MockMvcBuilders.standaloneSetup(productController, controllerErrorHandler)
                .build();
    }
}

Note that when using the MockMvcBuilders#standaloneSetup method, only the controllers added can be used. If we had another controller in our project, we wouldn't be able to mock HTTP requests to it using this MockMvc instance.

Writing Our First Test

Now, let's write our first test for this class to demonstrate how we can use MockMVC to write a semantic test mocking an HTTP request:

@Test
public void whenRetrievingAllProductsThenExpectListOfProducts() throws Exception {
    var mockProducts = Set.of(new ProductDTO(1L, "Apple", "Fresh Apples", 5.0f));

    when(productService.getAllProducts())
            .thenReturn(mockProducts);

    mockMvc.perform(get("/products"))
            .andExpect(content().json(new ObjectMapper().writeValueAsString(mockProducts)));
}

First, we define a set containing the mock products that will be the returned from our ProductService. With Mockito, we can then make our mock ProductService return this set of products using the Mockito#when method. Next, MockMvc allows us to create a method chaining pattern to semantically send a mock HTTP request and assert the expected response. Here, we are sending a mock GET request using the MockMvc#perform method and with the MockMvcRequestBuilders#get method we specify the GET request to be to the "/products" endpoint. Finally, by calling the MockMvc#andExpect method, we assert that the response should be a JSON string matching the mockProducts set serialized as JSON.

Writing Tests For Other GET Requests

From there, we can write similarly structured tests for other GET endpoints:

@Test
public void whenRetrievingAppleProductsThenExpectListOfAppleRelatedProducts() throws Exception {
    var mockProducts = Set.of(
            new ProductDTO(1L, "Apple", "Fresh Apples", 5.0f),
            new ProductDTO(2L, "Apple Pie", "Fresh Apple Pie", 15.0f),
            new ProductDTO(3L, "Apple turnover", "Fresh pastries", 8.0f)
    );

    when(productService.getAppleProducts())
            .thenReturn(mockProducts);

    mockMvc.perform(get("/products/apples"))
            .andExpect(content().json(new ObjectMapper().writeValueAsString(mockProducts)));
}

@Test
public void whenRetrievingProductByIdThenExpectProduct() throws Exception {
    var mockProduct = new ProductDTO(1L, "Oranges", "Fresh Oranges", 5.0f);

    when(productService.getProductById(1L))
        .thenReturn(mockProduct);

    mockMvc.perform(get("/products/1"))
            .andExpect(content().json(new ObjectMapper().writeValueAsString(mockProduct)));
}

Asserting Status Codes

With MockMVC we can also assert the expected HTTP response status code. As a perfect example of this, we can test for a 404 error:

@Test
public void whenRetrievingProductByIdThatDoesNotExistThenExpectProductNotFound() throws Exception {
    when(productService.getProductById(1L))
            .thenThrow(NoSuchElementException.class);

    mockMvc.perform(get("/products/1"))
            .andExpect(status().isNotFound());
}

Final Test Class

With that, we can share the final test class including some additional tests for full test coverage:

package io.john.amiscaray.productapi.controller;

import com.fasterxml.jackson.databind.ObjectMapper;
import io.john.amiscaray.productapi.dto.ProductDTO;
import io.john.amiscaray.productapi.service.ProductService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;

import java.util.NoSuchElementException;
import java.util.Set;

import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.delete;
import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;

@ExtendWith(MockitoExtension.class)
public class ProductControllerTest {

    @InjectMocks
    private ProductController productController;
    @Mock
    private ProductService productService;
    private MockMvc mockMvc;

    @BeforeEach
    public void setUp() {
        ControllerErrorHandler controllerErrorHandler = new ControllerErrorHandler();
        mockMvc = MockMvcBuilders.standaloneSetup(productController, controllerErrorHandler)
                .build();
    }

    @Test
    public void whenRetrievingAllProductsThenExpectListOfProducts() throws Exception {
        var mockProducts = Set.of(new ProductDTO(1L, "Apple", "Fresh Apples", 5.0f));

        when(productService.getAllProducts())
                .thenReturn(mockProducts);

        mockMvc.perform(get("/products"))
                .andExpect(content().json(new ObjectMapper().writeValueAsString(mockProducts)));
    }

    @Test
    public void whenRetrievingAppleProductsThenExpectListOfAppleRelatedProducts() throws Exception {
        var mockProducts = Set.of(
                new ProductDTO(1L, "Apple", "Fresh Apples", 5.0f),
                new ProductDTO(2L, "Apple Pie", "Fresh Apple Pie", 15.0f),
                new ProductDTO(3L, "Apple turnover", "Fresh pastries", 8.0f)
        );

        when(productService.getAppleProducts())
                .thenReturn(mockProducts);

        mockMvc.perform(get("/products/apples"))
                .andExpect(content().json(new ObjectMapper().writeValueAsString(mockProducts)));
    }

    @Test
    public void whenRetrievingProductByIdThenExpectProduct() throws Exception {
        var mockProduct = new ProductDTO(1L, "Oranges", "Fresh Oranges", 5.0f);

        when(productService.getProductById(1L))
            .thenReturn(mockProduct);

        mockMvc.perform(get("/products/1"))
                .andExpect(content().json(new ObjectMapper().writeValueAsString(mockProduct)));
    }

    @Test
    public void whenRetrievingProductByIdThatDoesNotExistThenExpectProductNotFound() throws Exception {
        when(productService.getProductById(1L))
                .thenThrow(NoSuchElementException.class);

        mockMvc.perform(get("/products/1"))
                .andExpect(status().isNotFound());
    }

    @Test
    public void whenDeletingProductThenExpectNoContentResponse() throws Exception {
        mockMvc.perform(delete("/products/1"))
                .andExpect(status().isNoContent());
    }

    @Test
    public void whenDeletingProductThatDoesNotExistThenExpectProductNotFound() throws Exception {
        doThrow(NoSuchElementException.class)
                .when(productService).deleteProduct(1L);

        mockMvc.perform(delete("/products/1"))
                .andExpect(status().isNotFound());
    }

}

Testing Our ProductService

Now, going down another layer, we can test our ProductService. Similar to the controller class, we can use the Mockito annotations to mock our dependencies (the ProductRepository) and inject these mock dependencies into a new instance of our ProductService:

@ExtendWith(MockitoExtension.class)
public class ProductServiceTest {

    @InjectMocks
    private ProductService productService;
    @Mock
    private ProductRepository productRepository;

}

Writing Our Tests

Writing our tests for the ProductService should be pretty straightforward. This service class deals with simple CRUD operations on products and only has ProductRepository as a dependency. Unlike the ProductController, we don't need to use anything fancy like MockMvc to mock complicated operations like HTTP communication. Thus, all of our tests can follow a simple structure where we have a given behavior of our ProductRepository, perform some action with our service, and then assert the result of the operation. Below I'll give the final test class for our ProductService and explain the few parts that might be confusing:

package io.john.amiscaray.productapi.service;

import io.john.amiscaray.productapi.dao.ProductRepository;
import io.john.amiscaray.productapi.data.Product;
import io.john.amiscaray.productapi.dto.ProductDTO;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;

import java.util.List;
import java.util.NoSuchElementException;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.equalTo;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.*;

@ExtendWith(MockitoExtension.class)
public class ProductServiceTest {

    @InjectMocks
    private ProductService productService;
    @Mock
    private ProductRepository productRepository;

    @Test
    public void whenGetAllProductsExpectAllProductsAsDTOs() {
        // Given
        var sampleProducts = List.of(
                new Product("Apple Pie", "Fresh Apple Pie", 15.0f),
                new Product("Oranges", "Fresh Oranges", 6.0f),
                new Product("Bananas", "Fresh Bananas", 6.0f)
        );
        when(productRepository.findAll()).thenReturn(sampleProducts);

        // When
        var productDTOs = productService.getAllProducts();

        // Assert
        assertThat(productDTOs, equalTo(sampleProducts.stream().map(ProductDTO::new).collect(Collectors.toSet())));
    }

    @Test
    public void whenLoadProductByIDExpectProductAsDTO() {
        // Given
        var sampleProduct = new Product("Raspberries", "Fresh Raspberries", 6.0f);
        when(productRepository.findById(1L)).thenReturn(Optional.of(sampleProduct));

        // When
        var productDTO = productService.getProductById(1L);

        // Assert
        assertThat(productDTO, equalTo(new ProductDTO(sampleProduct)));
    }

    @Test
    public void whenGetAppleProductExpectProductsWithAppleSubstring() {
        // Given
        var sampleProducts = Set.of(
                new Product("Apples", "Fresh Apples", 6.0f),
                new Product("Apple Pie", "Fresh Apple Pie", 15.0f)
        );
        when(productRepository.findAllByDescriptionContainsIgnoreCase("Apple"))
                .thenReturn(sampleProducts);

        // When
        var appleProducts = productService.getAppleProducts();

        // Assert
        assertThat(appleProducts, equalTo(sampleProducts.stream().map(ProductDTO::new).collect(Collectors.toSet())));
    }

    @Test
    public void whenCreateProductExpectIDOfNewProduct() {
        var newProduct = new ProductDTO("Bananas", "Fresh Bananas", 6.0f);
        var savedProduct = new Product(newProduct);
        savedProduct.setId(1L);
        when(productRepository.save(any()))
                .thenReturn(savedProduct);

        var productID = productService.createProduct(newProduct);

        assertThat(productID, equalTo(savedProduct.getId()));
    }

    @Test
    public void whenDeleteProductByIDExpectRepositoryDeleteByIDIsCalled() {
        when(productRepository.existsById(1L))
                .thenReturn(true);

        productService.deleteProduct(1L);

        verify(productRepository).deleteById(1L);
    }

    @Test
    public void whenDeleteProductByIDThatDoesNotExistExpectNoSuchElementExceptionAndRepositoryNotCalled() {
        when(productRepository.existsById(1L))
                .thenReturn(false);

        assertThrows(NoSuchElementException.class, () -> productService.deleteProduct(1L));
        verify(productRepository, never()).deleteById(1L);
    }
}

From the above class, I have a few notes about things that might be of interest to you:

1. Many of our classes use the Hamcrest library for their assertions. This allows us to make semantic assertions mainly using the assertThat and equalTo methods. There are many other ways you can use this library for semantic assertions if you wish to explore it further.
2. Some of our tests use the Mockito#verify method for their assertions. We can invoke methods of our ProductRepository class on the return value of this method to assert whether or not the invoked method gets called on our mock at any point. For instance, in the whenDeleteProductByIDExpectRepositoryDeleteByIDIsCalled method, we assert that the method gets called at least once while in the whenDeleteProductByIDThatDoesNotExistExpectNoSuchElementExceptionAndRepositoryNotCalled method we assert that it never gets called.
3. In the whenCreateProductExpectIDOfNewProduct method, we use the ArgumentMatchers.any method to enforce that when we pass any value to the save method of our mock ProductRepository, it will return the given product.

Testing Our ProductRepository

After testing our service, our next step is the ProductRepository. With that, the setup gets slightly more involved than when we were testing the ProductService.

Test Configuration

First, we need to set up a mock database which we will later populate before starting our tests. In our test/java/resources folder, we need to add a application-test.properties file with configuration for our test database:

spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.show-sql=true
spring.jpa.hibernate.ddl-auto=create-drop
logging.level.org.hibernate.SQL=DEBUG
logging.level.org.hibernate.tool.hbm2ddl=TRACE

These properties will be activated when setting a test Spring profile.

Setting Up ProductRepositoryTest

From there, we can set up our ProductRepositoryTest class:

@DataJpaTest
@ActiveProfiles("test")
public class ProductRepositoryTest {

    @Autowired
    private TestEntityManager entityManager;

    @Autowired
    private ProductRepository productRepository;

    private List<Product> products;

}

Here, we use the @DataJpaTest annotation to set up the Spring Context with all the JPA Repositories, and the @ActiveProfiles annotation to activate our test profile with the above configuration. From there, we auto-wire a TestEntityManager for populating our test database and a ProductRepository we are about to test. Lastly, we declare a products list of Product entities that we can add to the test database for our tests like so:

@BeforeEach
public void setUpDataSet() {
    products = List.of(
            new Product("Apples", "Fresh Apples", 6.0f),
            new Product("Oranges", "Fresh Oranges", 6.0f),
            new Product("Bananas", "Fresh Bananas", 6.0f),
            new Product("Apple Pie", "Fresh Apple Pie", 15.0f),
            new Product("Raspberries", "Fresh Raspberries", 6.0f),
            new Product("Apple Tarts", "Apple Pastries", 10.0f)
    );

    for (var product : products) {
        entityManager.persist(product);
    }

    entityManager.flush();
}

You may be wondering why we are saving these products before every test and if that will result in many duplicate rows but with different IDs. With the TestEntityManager, the database should be reset back to its initial state after each test, i.e., all the data will be reset. Thus, using this setup, we should have the same database state at the start of every test.

With that, we can write a couple simple tests for retrieving data with our repository:

package io.john.amiscaray.productapi.dao;

import io.john.amiscaray.productapi.data.Product;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest;
import org.springframework.boot.test.autoconfigure.orm.jpa.TestEntityManager;
import org.springframework.test.context.ActiveProfiles;

import java.util.List;
import java.util.stream.Collectors;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.equalTo;

@DataJpaTest
@ActiveProfiles("test")
public class ProductRepositoryTest {

    @Autowired
    private TestEntityManager entityManager;

    @Autowired
    private ProductRepository productRepository;

    private List<Product> products;

    @BeforeEach
    public void setUpDataSet() {
        products = List.of(
                new Product("Apples", "Fresh Apples", 6.0f),
                new Product("Oranges", "Fresh Oranges", 6.0f),
                new Product("Bananas", "Fresh Bananas", 6.0f),
                new Product("Apple Pie", "Fresh Apple Pie", 15.0f),
                new Product("Raspberries", "Fresh Raspberries", 6.0f),
                new Product("Apple Tarts", "Apple Pastries", 10.0f)
        );

        for (var product : products) {
            entityManager.persist(product);
        }

        entityManager.flush();
    }

    @Test
    public void whenFindAllProductsExpectListOfAllProducts() {
        var foundProducts = productRepository.findAll();

        assertThat(foundProducts, equalTo(products));
    }

    @Test
    public void whenFindAllByDescriptionContainsAppleIgnoreCaseExpectProductsWithAppleSubstring() {
        var foundProducts = productRepository.findAllByDescriptionContainsIgnoreCase("Apple");

        assertThat(foundProducts, equalTo(
                products.stream()
                        .filter(product -> product.getName().toLowerCase().contains("apple"))
                        .collect(Collectors.toSet())
        ));
    }

}

Since much of the work of implementing CRUD operations in our repository gets done for us by Spring, I think it's fine to leave the tests at that.

Integration Testing Our ProductController

Finally, we can put all the pieces together and write integration tests for our ProductController. This will send actual HTTP requests to our ProductController and go down through the ProductService, ProductRepository, and test database.

Setting Up Our Integration Test Class

First, let's set up the test class for our integration tests:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ActiveProfiles("test")
public class ProductControllerIT {

    @LocalServerPort
    private int port;
    @Autowired
    private TestRestTemplate restTemplate;
    @Autowired
    private ProductRepository productRepository;
    private List<Product> products;

}

Here, we use the @SpringBootTest annotation to set up our Spring Boot application on a random port and the @ActiveProfiles annotation to activate our test profile to use the same database as our ProductRepository tests. From there, we can define an int field annotated with @LocalServerPort to get the random port. Also, beneath that, we have an auto-wired TestRestTemplate to send HTTP requests, an auto-wired ProductRepository that we'll use to set the initial data, and a list of products we'll use as the data for the tests. From there, let's write a @BeforeEach method setting up our test data:

@BeforeEach
public void setUpDataSet() {
    productRepository.deleteAll();
    products = List.of(
            new Product("Apples", "Fresh Apples", 6.0f),
            new Product("Oranges", "Fresh Oranges", 6.0f),
            new Product("Bananas", "Fresh Bananas", 6.0f),
            new Product("Apple Pie", "Fresh Apple Pie", 15.0f),
            new Product("Raspberries", "Fresh Raspberries", 6.0f),
            new Product("Apple Tarts", "Apple Pastries", 10.0f)
    );
    productRepository.saveAll(products);
}

Afterward, we can write our first test for sending a GET request for all the products:

@Test
public void whenGetAllProductsExpectOkResponseWithListOfProducts() {
    var url = "http://localhost:" + port + "/products";

    ResponseEntity<Set<ProductDTO>> productsResponse = restTemplate.exchange(
            url,
            HttpMethod.GET,
            null,
            new ParameterizedTypeReference<Set<ProductDTO>>() {  }
    );

    assertThat(productsResponse.getStatusCode(), equalTo(HttpStatus.OK));
    assertThat(productsResponse.getBody(), equalTo(products.stream().map(ProductDTO::new).collect(Collectors.toSet())));
}

Using the TestRestTemplate#exchange method, we can send an HTTP GET request to the /products endpoint. The third argument to that method would be a RequestEntity where we can specify a body if needed (we purposely kept this null since we don't need a body) and the fourth argument is the type of the expected Response. Since the expected type of the response has a type parameter, we need to use a ParameterizedTypeReference to represent the type. After that, we can make assertions on the returned ResponseEntity<Set<ProductDTO>> which contains all the HTTP response information. With that, I can show you the full integration test class and then explain (in list form) any potentially interesting or confusing details about it to you:

package io.john.amiscaray.productapi.controller;

import io.john.amiscaray.productapi.dao.ProductRepository;
import io.john.amiscaray.productapi.data.Product;
import io.john.amiscaray.productapi.dto.ProductDTO;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.client.TestRestTemplate;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpMethod;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.ActiveProfiles;

import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;


import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ActiveProfiles("test")
public class ProductControllerIT {

    @LocalServerPort
    private int port;
    @Autowired
    private TestRestTemplate restTemplate;
    @Autowired
    private ProductRepository productRepository;
    private List<Product> products;

    @BeforeEach
    public void setUpDataSet() {
        productRepository.deleteAll();
        products = List.of(
                new Product("Apples", "Fresh Apples", 6.0f),
                new Product("Oranges", "Fresh Oranges", 6.0f),
                new Product("Bananas", "Fresh Bananas", 6.0f),
                new Product("Apple Pie", "Fresh Apple Pie", 15.0f),
                new Product("Raspberries", "Fresh Raspberries", 6.0f),
                new Product("Apple Tarts", "Apple Pastries", 10.0f)
        );
        productRepository.saveAll(products);
    }

    @Test
    public void whenGetAllProductsExpectOkResponseWithListOfProducts() {
        var url = "http://localhost:" + port + "/products";

        ResponseEntity<Set<ProductDTO>> productsResponse = restTemplate.exchange(
                url,
                HttpMethod.GET,
                null,
                new ParameterizedTypeReference<Set<ProductDTO>>() {  }
        );

        assertThat(productsResponse.getStatusCode(), equalTo(HttpStatus.OK));
        assertThat(productsResponse.getBody(), equalTo(products.stream().map(ProductDTO::new).collect(Collectors.toSet())));
    }

    @Test
    public void whenGetProductByIdExpectOkResponseWithProduct() {
        var url = "http://localhost:" + port + "/products/" + products.getFirst().getId();

        ResponseEntity<ProductDTO> productResponse = restTemplate.getForEntity(url, ProductDTO.class);

        assertThat(productResponse.getStatusCode(), equalTo(HttpStatus.OK));
        assertThat(productResponse.getBody(), equalTo(new ProductDTO(products.getFirst())));
    }

    @Test
    public void whenGetAppleProductsExpectOkResponseWithProductsWithAppleSubstring() {
        var url = "http://localhost:" + port + "/products/apples";

        ResponseEntity<Set<ProductDTO>> productsResponse = restTemplate.exchange(
                url,
                HttpMethod.GET,
                null,
                new ParameterizedTypeReference<Set<ProductDTO>>() {  }
        );

        assertThat(productsResponse.getStatusCode(), equalTo(HttpStatus.OK));
        assertThat(productsResponse.getBody(), equalTo(products.stream()
                .filter(product -> product.getName().toLowerCase().contains("apple"))
                .map(ProductDTO::new)
                .collect(Collectors.toSet())));
    }

    @Test
    public void whenPostProductExpectCreatedResponseAndLocationHeader() {
        var url = "http://localhost:" + port + "/products";

        ResponseEntity<Void> saveResponse = restTemplate.exchange(
                url,
                HttpMethod.POST,
                new HttpEntity<>(new ProductDTO("Avocado", "Fresh Avocado", 12.0f)),
                Void.class
        );

        assertThat(saveResponse.getStatusCode(), equalTo(HttpStatus.CREATED));
        assertThat(saveResponse.getHeaders().getLocation().getPath(), matchesPattern("/products/[1-9][0-9]*"));
    }

    @Test
    public void whenDeleteProductExpectNoContentResponseAndProductDeleted() {
        var productID = products.getFirst().getId();
        var url = "http://localhost:" + port + "/products/" + productID;

        ResponseEntity<Void> saveResponse = restTemplate.exchange(
                url,
                HttpMethod.DELETE,
                null,
                Void.class
        );

        assertThat(saveResponse.getStatusCode(), equalTo(HttpStatus.NO_CONTENT));
        assertThat(productRepository.existsById(productID), is(false));
    }

}

1. In the whenGetProductByIdExpectOkResponseWithProduct we have an example of how you can use the TestRestTemplate#getForEntity method to send a simple GET request. We weren't able to use this for the whenGetAllProductsExpectOkResponseWithListOfProducts because of the need to specify a ParameterizedTypeReference
2. In the whenPostProductExpectCreatedResponseAndLocationHeader method you can see how we can make assertions on an HTTP header. I also made use of another Hamcrest assertion method I haven't shown yet to match a string against a regex pattern.
3. In whenDeleteProductExpectNoContentResponseAndProductDeleted, I asserted the state of the ProductRepository to verify the proper interaction between components as is done in integration tests.

Conclusion

With that, we have successfully unit and integration tested a REST API using Spring Framework, JUnit, Hamcrest, and Mockito. In doing so, we went over all the Spring application layers ensuring great test coverage while creating semantically sound tests that should be highly maintainable. I sincerely hope this guide helps you create awesome tests for your future Spring projects!

Prerequisites

To follow along with this tutorial, you should be able to follow along with our last guide on the basics of microservices and implementing them with Spring Cloud.

What is a Config Server

A config server is a microservice that other services use to import their configurations (in the case of Spring Cloud, settings that would go in your application.properties file). This is useful for sharing common configurations for your microservices and creating externalized configurations that can be updated without redeployment! The config server can read configuration files from one location (often a git repository) and then other services can read the configuration files from the config server and import them.

Implementation

Now that you understand the idea of a config server, let's implement it. To begin, we'll be starting from the code we left off from in the last guide which you can find here. From that starting point, we'll need to create a new ConfigServer module:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>io.john.amiscaray</groupId>
        <artifactId>MicroservicesClassroomExample</artifactId>
        <version>1.0-SNAPSHOT</version>
    </parent>

    <artifactId>ConfigServer</artifactId>

    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-config-server</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-consul-discovery</artifactId>
        </dependency>
    </dependencies>

    <properties>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

</project>

Notice the spring-cloud-config-server and spring-cloud-starter-consul-discovery dependencies. These will allow this microservice to act as a config server and register itself with Consul. Within this new Maven module, we need to add the following class to start up the Config Server:

@SpringBootApplication
@EnableConfigServer
public class ConfigServerApplication {

    public static void main(String[] args) {
        SpringApplication.run(ConfigServerApplication.class, args);
    }

}

From there, we need to create a Git repository with the configs we want to share and then have this config server retrieve it. To do this, first, create a new Git repository and host it on GitHub. Within the repository, we'll put a simple application.properties file in the root directory configuring a debug logging level for our testing purposes:

logging.level.root=DEBUG

Once you committed the configuration and pushed it to GitHub, you'll need to create a GitHub personal access token with private repository access. With it, your config service can authenticate with GitHub to fetch configurations from the repository. To generate the token, go to Settings > Developer settings > Personal access tokens > Tokens (classic) > Generate new token > Generate new token (classic). Select the following permissions for the access token:

Now that you have the token, you can configure your config service's application.properties to access the GitHub repo for the configs:

spring.application.name=config-server
server.port=8888
spring.cloud.config.server.git.uri=https://github.com/john-amiscaray/MicroservicesClassroomConfigs
# Set the GitHub username and access token using environment variables
spring.cloud.config.server.git.username=${GITHUB_USER}
spring.cloud.config.server.git.password=${GITHUB_REPO_TOKEN}

Now, we can configure our other microservices' application.properties to import configurations from our config server like so:

spring.config.import=configserver:
# Same as the spring.application.name property of our config service
spring.cloud.config.discovery.service-id=config-service
spring.cloud.config.discovery.enabled=true

Along with that, we also need to add a spring-cloud-starter-config dependency in our root pom.xml for all of our services:

 <dependency>
     <groupId>org.springframework.cloud</groupId>
     <artifactId>spring-cloud-starter-config</artifactId>
 </dependency>

Now, we can start up Consul and our microservices and see that our Student Service, API Gateway, and Course Service applications have debug-level logging in the console:

Importing Microservice-Specific Configs

With our config server and our Git repository, we can also import configs for specific microservices. Within the GitHub repository containing our configs, let’s create new course-service and student-service directories with course-service.properties and student-service.properties files for our Course Service and Student Service respectively. We name these property files the exact same as the names of the services (listed in their respective application.properties files) so that the config server will know what to do with them. After adding these new files, the directory structure would look like so:

Within each properties file, let’s configure the data sources for its respective microservice:

# student-service configuration
spring.datasource.url=jdbc:mysql://localhost:3306/student
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=password
spring.jpa.hibernate.ddl-auto=create-drop

# course-service configuration
spring.datasource.url=jdbc:mysql://localhost:3306/course
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=password
spring.jpa.hibernate.ddl-auto=create-drop

After that, we need to update the application.properties file in our config server. These new properties specify subdirectories in our GitHub repository where configs can be:

spring.cloud.config.server.git.search-paths[0]=course-service
spring.cloud.config.server.git.search-paths[1]=student-service

With all that, we can commit to our GitHub repository, remove the configuration from the microservices' resources/application.properties files, re-run the microservices, and see that these data source configurations are being imported from the config server!

Profile Specific Configuration

As you might expect, we can create Spring profile-specific configurations for all our microservices. As you'd normally do for a traditional Spring application, we can define property files with a "-dev" suffix for configurations to use in a dev profile:

# student-service-dev.properties

spring.datasource.url=jdbc:h2:mem:student
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.h2.console.enabled=true
spring.jpa.hibernate.ddl-auto=create-drop

# course-service-dev.properties

spring.datasource.url=jdbc:h2:mem:course
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=password
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.h2.console.enabled=true
spring.jpa.hibernate.ddl-auto=create-drop

# application-dev.properties

logging.level.root=DEBUG

# application.properties

logging.level.root=INFO

With these new files, our repository should look like so:

Once we run all our services with the dev profile, you'll see that our microservices have debug-level logging and connect to H2 databases.

Conclusion

In this guide, we learned about what config servers are in a microservice architecture and how we can implement them with Spring Cloud. To review the work we did, you can have a look at the final code on GitHub. Happy coding!

Prerequisites

To be able to follow this guide, you should have a solid understanding of the fundamentals needed to build a simple Spring Boot application. This includes:

• Dependency Injection,

• Spring MVC,

• Hibernate, and

• Spring Data JPA

This is important because I'll be skipping over parts of our application involving these things for the sake of brevity.

A Refresher on Microservices

To start, in case you don’t know, let’s first talk about the theory behind microservices. If you’re already aware, feel free to skip ahead to the project architecture.

First, let’s define what microservices are. The term “microservices” refers to an architecture where the application is split into a set of smaller service applications that communicate with each other. In the case of our application, the services will be Spring Boot applications communicating over HTTP. This set of “services” ("microservices") comprises the whole application by collectively providing the different services/features of it. Immediately, the first two problems that arise from this are: "How do we coordinate these services?" and "How can we expect the user/consumer to interact with all these services independently?" Well, I’m glad you asked! First, let’s talk about how we coordinate the services in a microservices application:

Service Registration and Service Discovery

In a microservices application, we typically have a service dedicated to the problem of registering our services (service registration) and helping them find each other (service discovery) so we can effectively coordinate them. In the context of our application, we will be using a service called Consul By HashiCorp as an out-of-the-box solution to start up a server for service registration and service discovery. When our other services start up, they will register themselves with Consul who will be responsible for keeping track of them. When one service wants to communicate with another service, Consul can help the service figure out the other’s hostname. The implication of this in our application, as you’ll see later, is that when we send an HTTP request from one service to another we can use a service name instead of the host in the URL. When we send this request, Consul will help our service resolve the actual hostname of the service we’re trying to talk to! This is great for having flexibility for where our services are deployed especially if we want to implement load balancing.

API Gateways

As for our second problem: "How can we expect the user/consumer to interact with all these services independently?", the quick answer is: we don't! Using an "API gateway", we can have the user use our application as though it were one unified entity. In the context of microservices, an API gateway is a service used as an interface for a user/consumer of our application to cleanly interact with our services. As you’ll see in our application, the API gateway will act as a facade for our microservices allowing the client to communicate with us as though we built our application as a traditional monolith (one single large application). In the implementation, the API Gateway will effectively forward HTTP requests to the appropriate service making it appear as though the HTTP request was handled by a single server (you’ll see more clearly what I mean later).

Pros And Cons of Microservices

Now, you may be asking, why bother with something complicated like this? To answer this, let’s analyze the pros and cons of this approach:

Pros:

• Flexibility and ease of development: this approach can make development easier in some ways. Here, we can split a large application into smaller, easier-to-understand services. Additionally, each of these services can be developed independently of each other, meaning we can have some flexibility on how each of them is developed, even allowing them to use different programming languages!

• Scalability: microservices can help with scalability in that service instances can be scaled independently of each other instead of having to scale the entire application (potentially saving resources). This can be helpful if one service requires additional load over others.

• Fault tolerance: microservices can help your application be more resilient to errors. If we are experiencing problems with one service, we could still have the other services up and running.

Cons:

• Communication overhead: as you can imagine, this architecture can add extra communication overhead because of the communication between services.

• Complexity: a distributed architecture clearly adds much more complexity to your application. This can make it more difficult for debugging, integration testing, and sometimes coding.

While it has its use cases for large-scale applications, as you can see, microservices aren't some magical be-all-end-all solution for everything.

Application Architecture

Before we begin implementing our application, let’s first discuss its architecture. To keep our application simple, we’ll create only three services: the API gateway, Student Service, and Course Service. Student Service will be for creating and retrieving students and the Course Service will do the same for courses. Below is a simple diagram of how the whole application will work:

As we talked about previously, the API Gateway will be the interface a client will use to communicate with our application. Every request to our application will first go to the gateway which will then forward them to the correct microservice. Moreover, you can see that each of our services registers themselves to Consul. This will happen automatically for us as soon as each Spring Boot application starts up, allowing our Student Service to leverage service discovery from Consul to communicate with the Course Service (as you’ll see later). Finally, the Course Service and the Student Service each have an independent database they will store data in. For simplicity, I had them connect to the same local MySQL database but using different schemas as though it were a distributed database.

Implementation

Now, let’s dive straight into the implementation. Let’s go through the process step-by-step so you can easily replicate this.

1. Starting Up a Local Consul Instance

First, download Consul here. Using your Consul executable run the following command:

consul agent -dev

Now, you should be able to see Consul running on localhost:8500:

2. Creating a Multi-module Maven Project

Now, let's start writing the code for our project. For my implementation, I chose to create a multi-module Maven project with a root pom.xml and submodules (in separate folders) for each microservice. The advantage of this approach is that we can aggregate common dependencies into the root pom and compile all the modules together. The final file structure of the project would look like this:

Here, we have a pom.xml file in the root folder of the project and one in each subfolder corresponding to a module. The following is the definition of the root pom.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>io.john.amiscaray</groupId>
    <artifactId>MicroservicesClassroomExample</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>pom</packaging>

    <properties>
        <maven.compiler.source>21</maven.compiler.source>
        <maven.compiler.target>21</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <java.version>21</java.version>
        <spring-boot.version>3.2.2</spring-boot.version>
        <spring-cloud.version>2023.0.0</spring-cloud.version>
    </properties>

    <modules>
        <module>APIGateway</module>
        <module>StudentService</module>
        <module>CourseService</module>
    </modules>

    <dependencies>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-consul-discovery</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-dependencies</artifactId>
                <version>${spring-boot.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <excludes>
                        <exclude>
                            <groupId>org.projectlombok</groupId>
                            <artifactId>lombok</artifactId>
                        </exclude>
                    </excludes>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

First, you’ll notice that we declare the names of our submodules in the modules declaration. To be able to specify these submodules, we need to set the packaging property in the pom.xml to “pom”. Next, within the dependencies declaration are common dependencies we’ll need for each microservice. Finally, within the dependencyManagement section we have dependencies that allow Maven to resolve the versions of our other dependencies based on our Spring Boot and Spring Cloud versions.

Note that we must add the spring-boot-starter-actuator dependency for health-checking functionality for our microservices that Consul makes use of.

3. Creating Our API Gateway

With that, we can start implementing our API Gateway. Begin by creating a new module within the multi-module Maven project. In IntelliJ, you can select New > Module… to add the new module setting the build tool to Maven and the parent module as the project specified in the root pom:

Note that I personally chose to not build these modules using the Spring Initializr since I found this easier. It seems that the Spring Initializer automatically set a parent module for these modules to be a Spring Boot Starter module.

In the module's pom.xml file, add the following dependency:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>

Then, within the application.properties file add the following properties:

spring.application.name=api-gateway
server.port=8080
spring.cloud.consul.host=localhost
spring.cloud.consul.port=8500
spring.cloud.consul.discovery.enabled=true
spring.cloud.consul.discovery.register=true

This will set the name of the service for when it registers (the spring.application.name property), the port it will run on, and allow the service to automatically register itself to Consul. Later, we will need to set additional properties for the HTTP request forwarding to our other services.

Now, all we need to do is add the following class:

@SpringBootApplication
@EnableDiscoveryClient
public class APIGatewayApplication {

    public static void main(String[] args) {
        SpringApplication.run(APIGatewayApplication.class, args);
    }

}

This is almost identical to the standard entry point for a Spring Boot application except with a @EnableDiscoveryClient annotation to signify that this will be registered to Consul.

4. Creating Our Student Service

Creating our Student Service should follow a similar process to our API Gateway and what you’re already familiar with in Spring Boot. First, create a new module for our Student Service as shown above. Then add the following dependencies for our database connectivity:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
    <groupId>com.mysql</groupId>
    <artifactId>mysql-connector-j</artifactId>
    <version>8.3.0</version>
</dependency>

Then, we’ll have a similar entry point class as what you’ve seen with the API Gateway:

@SpringBootApplication
@EnableDiscoveryClient
public class StudentServiceApplication {

    public static void main(String[] args) {
        SpringApplication.run(StudentServiceApplication.class, args);
    }

}

Now, for the application.properties file we’ll have the following:

spring.application.name=student-service
server.port=8081
spring.cloud.consul.host=localhost
spring.cloud.consul.port=8500
spring.cloud.consul.discovery.enabled=true
spring.cloud.consul.discovery.register=true
spring.datasource.url=jdbc:mysql://localhost:3306/student
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=password
spring.jpa.hibernate.ddl-auto=create-drop

For the sake of time, let’s skip over creating our ORM classes, repositories, DTOs, and services and show you this service’s controller (you can always see the final code in the GitHub repository):

@RestController
@RequestMapping("/student")
@AllArgsConstructor
public class StudentController {

    private StudentService studentService;
    private CourseServiceClient courseServiceClient;

    @GetMapping("")
    public ResponseEntity<StudentsView> getAllStudents() {
        var students = studentService.getAllStudents()
                .stream()
                .map(StudentDTO::new)
                .toList();

        return ResponseEntity.ok(new StudentsView(students));
    }

    @GetMapping("{id}")
    public ResponseEntity<StudentDTO> getStudentById(@PathVariable(value = "id") long id) {
        var student = studentService.getStudentById(id);

        return ResponseEntity.ok(student);
    }

    @PostMapping("")
    public ResponseEntity<Void> saveStudent(@RequestBody StudentDTO studentDTO) throws URISyntaxException {
        var studentID = studentService.saveStudent(studentDTO);

        return ResponseEntity.created(new URI("/student/" + studentID)).build();
    }

}

5. Configuring Our API Gateway For Our Student Service

Now, we need to configure our API gateway to forward requests to our Student Service. In the API gateway’s application.properties add the following:

spring.cloud.gateway.routes[0].id=student-service
spring.cloud.gateway.routes[0].uri=lb://student-service
spring.cloud.gateway.routes[0].predicates[0]=Path=/student/**

Here, we’re saying that the first route configured in the spring.cloud.gateway.routes array property is for any request to /student subpaths of our API gateway. These requests will be forwarded to our Student Service and can be load-balanced (the lb protocol in the URI) by Spring Cloud.

With that, our Student Service should now be fully functional and accept requests forwarded from our API gateway!

6. Implementing Our Course Service

Now for our Course Service, like our Student Service, we'll create a new submodule, add the same dependencies, and create a similar application.properties file:

spring.application.name=course-service
server.port=8082
spring.cloud.consul.host=localhost
spring.cloud.consul.port=8500
spring.cloud.consul.discovery.enabled=true
spring.cloud.consul.discovery.register=true
spring.datasource.url=jdbc:mysql://localhost:3306/course
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.username=root
spring.datasource.password=password
spring.jpa.hibernate.ddl-auto=create-drop

Now from there, we can implement our REST endpoints:

@RestController
@RequestMapping("/course")
@AllArgsConstructor
public class CourseController {

    private CourseService courseService;

    @GetMapping
    public ResponseEntity<CoursesView> getAllCourses() {
        var courses = courseService.getAllCourses();

        return ResponseEntity.ok(new CoursesView(courses));
    }

    @GetMapping("{id}")
    public ResponseEntity<CourseDTO> getCourseById(@PathVariable("id") Long id) {
        var course = courseService.getCourseById(id);

        return ResponseEntity.ok(course);
    }

    @PostMapping
    public ResponseEntity<Void> createCourse(@RequestBody CourseDTO courseDTO) throws URISyntaxException {
        var savedCourseID = courseService.saveCourse(courseDTO);

        return ResponseEntity.created(new URI("/course/" + savedCourseID)).build();
    }

}

7. Configuring Our API Gateway

Finally, we must update the API gateway's application.properties file with path mapping for our Course Service:

spring.cloud.gateway.routes[1].id=course-service
spring.cloud.gateway.routes[1].uri=lb://course-service
spring.cloud.gateway.routes[1].predicates[0]=Path=/course/**

Microservice Communication

With all that, our application should work fairly well now. The Student Service will manage student-related retrieval and creation endpoints and our Course Service will do the same for courses. However, one fundamental concept this application doesn't implement yet is communication between microservices. As I alluded to earlier in the project architecture, we'll also implement microservice communication from the Student Service to the Course Service.

The Motive for Microservice Communication in Our Project

First, I need to explain to you a motive for our microservices communicating with each other. In our Student Service, we use the following Student entity:

@Entity
@AllArgsConstructor
@NoArgsConstructor
@Getter
@Setter
public class Student {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    private String major;
    private Float gpa;

    public Student(StudentDTO studentDTO) {
        name = studentDTO.getName();
        major = studentDTO.getMajor();
        gpa = studentDTO.getGpa();
    }

}

Within our Course Service, we also replicate data (just the Student ID) from the corresponding Student table to maintain a foreign key relation between our Students and Courses:

@Entity
@AllArgsConstructor
@NoArgsConstructor
@Getter
@Setter
public class Student {

    @Id
    private Long id; // Keeping only the student IDs, the rest of the data will be in the Student Service
    @ManyToMany(mappedBy = "takenBy")
    private List<Course> courses;

    public Student(SaveStudentRequest student) {
        id = student.getId();
        courses = new ArrayList<>();
    }

}

@Entity
@NoArgsConstructor
@AllArgsConstructor
@Getter
@Setter
public class Course {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private Long id;
    private String name;
    @Enumerated(EnumType.ORDINAL)
    private CourseTerm term;
    @ManyToMany
    @JoinTable(
            name = "StudentCourse",
            joinColumns = {
                    @JoinColumn(name = "course_id", referencedColumnName = "id"),
            },
            inverseJoinColumns = {
                    @JoinColumn(name = "student_id", referencedColumnName = "id")
            }
    )
    private List<Student> takenBy;

    public Course(CourseDTO courseDTO) {
        name = courseDTO.getName();
        term = courseDTO.getTerm();
        takenBy = new ArrayList<>();
    }

}

The problem with this is that we need a way to insert data into the Course Service's Student table whenever there are new records in the Student table of the Student Service. While what we're about to do is flawed (both in security and possibly from a correctness standpoint) this will show you how to send a request from the Student Service to the Course Service to insert into its Student table. Like before, let's take this step-by-step:

1. Creating a New Endpoint in Our Course Service

To start, let's add a new endpoint in our Course Service for adding new Student records:

@AllArgsConstructor
@NoArgsConstructor
@Getter
public class SaveStudentRequest {

    private Long id;

}

@RestController
@RequestMapping("student")
@AllArgsConstructor
public class StudentController {

    private StudentService studentService;

    @PostMapping
    public ResponseEntity<Void> addStudent(@RequestBody SaveStudentRequest student) {
        studentService.saveStudent(student);

        return ResponseEntity.noContent().build();
    }

}

2. Creating a WebClient.Builder Bean

From our Student Service, we need to configure a bean for a WebClient.Builder class (provided by Spring Webflux) to send requests to the Course Service:

@SpringBootApplication
@EnableDiscoveryClient
public class StudentServiceApplication {

    public static void main(String[] args) {
        SpringApplication.run(StudentServiceApplication.class, args);
    }

    @Bean
    @LoadBalanced
    public WebClient.Builder webClientBuilder() {
        return WebClient.builder();
    }

}

Here, you'll notice the @LoadBalanced annotation added to the method declaration. This is required for Consul to properly resolve the Course Service's host (as you'll see soon).

3. Creating Our CourseServiceClient class

With that, we can create another class for using this WebClient.Builder to send a request to the new endpoint of our Course Service:

@Service
@AllArgsConstructor
public class CourseServiceClient {

    private WebClient.Builder webClientBuilder;

    public Mono<Void> saveNewStudent(CourseSaveStudentRequest saveStudentRequest) {
        return webClientBuilder.build()
                .post()
                .uri("http://course-service/student") // course-service will resolve to the correct host because of consul's service discovery
                .bodyValue(saveStudentRequest)
                .retrieve()
                .bodyToMono(Void.class);
    }

}

Here, we send a POST request to the new /student endpoint of our Course Service. Because of Consul's service discovery capabilities, the course-service part of the URI will be interpreted as the actual hostname of our Course Service! Also, because the WebClient.Builder is part of Spring Webflux (a module for reactive programming in a web context), we get the result back as a Mono. This acts as a wrapper around an asynchronously retrieved value which I believe should be similar in concept to Java's built-in CompleteableFuture.

4. Modifying Our POST Student Endpoint

Using the client, we can now update our endpoint for saving new students:

@RestController
@RequestMapping("/student")
@AllArgsConstructor
public class StudentController {

    private StudentService studentService;
    private CourseServiceClient courseServiceClient;

    // The other endpoints go here...

    @PostMapping("")
    public ResponseEntity<Void> saveStudent(@RequestBody StudentDTO studentDTO) throws URISyntaxException {
        var studentID = studentService.saveStudent(studentDTO);

        courseServiceClient.saveNewStudent(new CourseSaveStudentRequest(studentID))
                .then()
                .subscribe(); // Do nothing when the request goes through.

        return ResponseEntity.created(new URI("/student/" + studentID)).build();
    }

}

Here, whenever we save a new Student into the Student Service's Student table, we'll also send a request using the CourseServiceClient to update the Course Service's Student table.

Conclusion

With that, we successfully created a sample microservices application using an API gateway, communication between microservices, and a simple distributed database. While this application isn't perfect, this should give you a good taste of how you can create your first microservices project! In the future, you can expect follow-up blog posts on more things we can do with our microservices to improve this project. Happy coding!

Java with John is a blog I founded with the hopes of helping like-minded software developers, whether they are very new to coding or more advanced, advance their learning and careers. Using the experiences and knowledge I gained from personal learning, books, major projects, and professional experience, I hope to give you actional knowledge and advice you can use to better yourself and your future. While I plan for this blog to be centered around developing skills with Java, I also decided to keep my previously written soft skills content, the Soft Skills and Ramblings series, to help developers in a career and personal development direction.

Building a Personal Brand

Additionally, with this blog, I hope to develop a strong personal brand for myself as the Java Specialist I hope to be. By marketing myself and my skills, I hope to build a long-term impact on the community that can later help me find developer opportunities to use my skills for the betterment of an organization and the world.

Prerequisites

To follow along with this guide, you more or less just need intermediate Java knowledge. While this is more of an advanced concept, I tried to go over this thoroughly so intermediates can understand. Also, I believe that this shouldn’t be too hard to understand even without a strong Java background.

What is Metaprogramming In Java?

Wikipedia defines metaprogramming as “a computer programming technique in which computer programs have the ability to treat other programs as their data.” Yes, you read that correctly, you can write code that reads and processes other code! Java’s JDK implements this idea in the form of the Reflection API. This is a set of Java classes and functionalities used to read and process classes and potentially manipulate their members. This is how Java frameworks like Spring Framework do magical things behind the scenes that don’t otherwise look possible without somehow breaking the language. I was able to use it myself to build a couple of powerful Java Frameworks and I hope with what you learn today, you can be inspired and educated to do the same!

Diving Straight Into Metaprogramming

Now, let’s dive straight into a practical example. For our mini-project, let’s create a simple JSONSerializerService class for serializing Java objects into JSON. First, let’s discuss how our API will work for marking classes as serializable and controlling the JSON output.

Our Annotations

To mark classes as serializable and their fields with data on how to serialize them, we can create custom Java annotations. An annotation acts as a marker over classes, fields, parameters, or other Java constructs to give extra metadata about them. Typically, annotations are used for documentation purposes or, in our case, in conjunction with reflection. Probably the most well-known example of an annotation is the @Override annotation you’ll see on top of overridden methods to document that they are an overridden method.

Now that you know about annotations, let’s define one named @JSONSerializable to mark a class as serializable:

package io.john.amiscaray.annotation;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface JSONSerializable {


}

In the code above, we defined our annotation. We placed some other annotations above ours to define information to the Java compiler and about how ours will work. Using the @Target annotation, we defined that our annotation can be applied to classes/types (ElementType.TYPE). Then, using the @Retention annotation, we specify that this annotation should be accessible during runtime (i.e., we will be able to know if a class has this annotation at runtime using reflection). With that, let’s define other annotations to give information on a class’ fields for our JSON serializer:

package io.john.amiscaray.annotation;


import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.FIELD) // This annotation can be applied to fields
@Retention(RetentionPolicy.RUNTIME)
public @interface JSONIgnore {


}

package io.john.amiscaray.annotation;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Target(ElementType.FIELD)
@Retention(RetentionPolicy.RUNTIME)
public @interface JSONRename {

    String value() default ""; // An argument we can pass when we annotate a field.

}

Using the @JSONIgnore annotation, we will mark fields that the JSON serializer should ignore when serializing (for example, if we have some data we don’t want to make publically accessible in our JSON). Additionally, with our @JSONRename annotation, we will mark fields that we want to give a different name in our JSON body. This accepts a String argument for the name of the field that will appear in the JSON.

Implementing Our JSONSerializerService

Now, let’s create our JSONSerializerService class. We’ll be going through this process step-by-step to make sure everyone can understand this. Let’s first make this new JSONSerializerService class a singleton class. This is a design pattern, typically for global utility classes, to enforce a single global instance (you can choose to ignore this part if it confuses you):

package io.john.amiscaray.service;

public class JSONSerializerService {

    private static JSONSerializerService instance;

    private JSONSerializerService() { }

    public static JSONSerializerService getInstance() {
        if (instance == null) {
            instance = new JSONSerializerService();
        }
        return instance;
    }
}

Now let’s begin implementing a serialize method. This method will accept a single object we wish to serialize and return a String of the resulting JSON. First, we need to make sure that the object is a member of a class marked with our@JSONSerializable annotation:

public String serialize(Object object) {
    if (!object.getClass().isAnnotationPresent(JSONSerializable.class)) {
        throw new RuntimeException("The passed object is not serializable");
    }
    // The rest of the implementation goes here...
}

This is our first taste of the Reflection API. Using the Object#getClass method, we can get the runtime class of the passed object. Then, we can simply call the Class#isAnnotationPresent method to check if the class contains the @JSONSerializable annotation. From there, let’s define a StringBuilder which we will use to produce our output:

public String serialize(Object object) {
    if (!object.getClass().isAnnotationPresent(JSONSerializable.class)) {
        throw new RuntimeException("The passed object is not serializable");
    }

    var result = new StringBuilder("{\n"); // Initialize a StringBuilder with "{\n" at the start of the string
    // The rest of the implementation goes here...
}

Since we will be doing a lot of String appending operations to produce our JSON, it’s best we use a StringBuilder to make sure we do so optimally (since String appending in Java can be surprisingly costly). With that out of the way, we can get to the juicy part of reading the fields of the class to add to our JSON output. Using Java’s Reflection API, we can access the fields of the class like so:

 public String serialize(Object object) {
    if (!object.getClass().isAnnotationPresent(JSONSerializable.class)) {
        throw new RuntimeException("The passed object is not serializable");
    }

    var result = new StringBuilder("{\n"); // Initialize a StringBuilder with "{\n" at the start of the string
    var clazz = object.getClass(); // Naming this clazz to avoid keyword conflicts
    for (var field : clazz.getDeclaredFields()) {
        // Logic goes here
    }
    // The rest of the implementation goes here...
}

From there, let’s just see the rest of the implementation and go over some of the parts that are important to you:

public String serialize(Object object) {
    if (!object.getClass().isAnnotationPresent(JSONSerializable.class)) {
        throw new RuntimeException("The passed object is not serializable");
    }
    var result = new StringBuilder("{\n"); // Initialize a StringBuilder with "{\n" at the start of the string
    var clazz = object.getClass();
    for (var field : clazz.getDeclaredFields()) {
        if (field.isAnnotationPresent(JSONIgnore.class)) {
            continue;
        }
        field.setAccessible(true);
        var fieldName = field.getName();
        if (field.isAnnotationPresent(JSONRename.class)) {
            var annotation = field.getAnnotation(JSONRename.class);
            fieldName = annotation.value();
        }
        try {
            result
                .append("\t")
                .append("\"").append(fieldName).append("\"")
                .append(":")
                .append("\"").append(field.get(object)).append("\"")
                .append(",\n");
        } catch (IllegalAccessException e) {
            throw new RuntimeException(e);
        }
    }

    // Remove the trailing ,
    result = new StringBuilder(result.substring(0, result.length() - 2));

    return result.append("\n}").toString();
 }

So first, we check if the field has the @JSONIgnore annotation and skip the field if it is present. This way, we can exclude this field from our JSON output. Then, we need this line:field.setAccessible(true); for later so we can pry open the field values of the object passed to this method (even private fields!). From there, we can extract the name of the declared field using the Field#getName method. If the field has the @JSONRename annotation, we instead use the name passed to the annotation (fetched by calling the JSONRename#value method) in the JSON output. After that, we append the data to our JSON string. You’ll notice there we have a field.get(object) statement we are setting as the value of the field in the JSON. This fetches the corresponding field value from our object. So for example, if during the current iteration of the loop field corresponds to a String field named name, then that statement will fetch the value of that name field in the object.

Testing Our Project

Now that we implemented our JSON serializer, let’s test it out. Here’s a sample Student class we can serialize with our JSONSerializerService:

package io.john.amiscaray.domain;

import io.john.amiscaray.annotation.JSONIgnore;
import io.john.amiscaray.annotation.JSONRename;
import io.john.amiscaray.annotation.JSONSerializable;

@JSONSerializable
public class Student {

    private String name;
    private String major;
    @JSONRename("Term GPA") // Rename the field to "Term GPA" in the output
    private Float gpa;
    @JSONIgnore // Exclude this from the JSON output
    private Integer sinNumber;

    public Student(String name, String major, Float gpa, Integer sinNumber) {
        this.name = name;
        this.major = major;
        this.gpa = gpa;
        this.sinNumber = sinNumber;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getMajor() {
        return major;
    }

    public void setMajor(String major) {
        this.major = major;
    }

    public Float getGpa() {
        return gpa;
    }

    public void setGpa(Float gpa) {
        this.gpa = gpa;
    }

    public Integer getSinNumber() {
        return sinNumber;
    }

    public void setSinNumber(Integer sinNumber) {
        this.sinNumber = sinNumber;
    }
}

With that, we can print out a sample Student:

package io.john.amiscaray;

import io.john.amiscaray.domain.Student;
import io.john.amiscaray.service.JSONSerializerService;

public class Main {

    public static void main(String[] args) {
        var serializer = JSONSerializerService.getInstance();
        System.out.println(serializer.serialize(new Student("John", "Computer Science", 4.0f, 12345678)));
    }

}

The output to the console should look like so:

{
    "name":"John",
    "major":"Computer Science",
    "Term GPA":"4.0"
}

With that, we finally have our simple JSON serializer built with reflection and custom annotations!

One Caveat With Reflection

While the application works pretty well for our purposes, there’s one thing you might need to look out for in the future when it comes to reflection. If you are using Java’s JPMS (Java Package Management System) functionality to separate your project into different modules, you might have a bit of trouble doing stuff with reflection like this. This is because Java’s modules, by default, don’t allow other modules reflective access to private members. For example, if our JSONSerializerService class was in a module named my_module and we wanted to serialize a class in a other_module module, then we would get an exception when we try to access private fields of the class we wanted to serialize (something our JSONSerializerService is doing using reflection). To get around this, we would need to add a declaration to other.module's module declaration file (module-info.java):

module other_module {
    opens pack.containing.serializable.clazz to my_module;
}

Here, we are saying that for our module other_module, any class in a package named pack.containing.serializable.class allows reflective access to private fields for classes in the my.module module.

Conclusion

With that, I hope you got a good idea of what metaprogramming is in Java and how you could potentially use it to build cool things. Using the magic that is the Reflection API, you can expand the boundaries of what you could normally do with Java code and implement magic as you see from Java frameworks. Using your newfound powers, I encourage you to try it out yourself and build a cool and unique new project with it. Happy coding!

Why Hire Me?

For those interested in my expertise, I know I have what it takes to bring immense value to your organization using my highly effective technical skills and valuable soft skills. With years of passionate and directed Java learning under my belt, I have developed the talent that already brings value to thousands through my technical writing and professional experience. Outside of that, I used my expertise to build complex and robust passion projects using my knowledge and experience with advanced Java, Spring Framework, JUnit, Maven, Javascript, Angular, and Github actions. Soft skills wise, I use my drive and communication skills to take strong personal initiatives like this blog.

Do you think my expertise could be useful to you? You can view my resume on the navbar above and email me at john.amiscaray@torontomu.ca

Inspiration and Context

Oh boy! While writing out this blog post, I expected this section on the inspiration and context of Stir to be fairly short. I thought I would just give a fairly brief explanation of what brought me to work on this project and how you could draw inspiration for projects in a similar way. As it turns out, this ended up being a long but interesting story/rant on the exact details of how this idea came to be. Trust me though, you're in for a ride with this one and there are some interesting lessons here you could use (although I suppose adding in this extra paragraph as a disclaimer doesn't really help).

The inspiration I had for Stir came about through a perfect storm of circumstances and is yet very replicable in your journey. It was a really busy semester that term and so I didn't have much time to pursue my passion for building side projects or just coding creatively. Sure, as a computer science major, I did do a good amount of coding and there were some interesting assignments here and there, but it felt more like a necessity than the pursuit of a passion or a fun hobby. I guess to that extent it was like something was missing in my life at the time that I didn't necessarily pick up on. Sure, I was doing well, fairly happy, and pursuing other goals where I could but because of the workload I had it was like I wasn't able to use my fullest potential and fully be myself. Then comes along an assignment in my databases course where we got to develop a web application to interact with the database. At this point, I couldn't hold in my excitement to potentially build something nice and reminiscent of the types of large web projects I would enjoy building. We were deciding what tech stack to use here and were deciding between something in Java or PHP. At this point, I believe I was leaning more toward PHP because I felt that the way things are done in Spring with Hibernate (a Java ORM framework) and Spring Data JPA, would abstract too much of the SQL code we would use and defeat the purpose of the assignment in some aspects. However, I eventually found a way to instead use raw JDBC with Spring Framework which opened up the possibility of using my beloved backend framework for this project. I got extremely interested in taking this approach and started furiously coding a proof of concept during a lecture (which I probably should have listened to but oh well) and got something working. From there, I knew we just had to try to build the project this way and my team members agreed.

For the next week or so I started working hard on this assignment through my interest in learning new things here. As I start building off of my proof of concept, suddenly something insane and possibly masochistic comes to mind. What if ... I were to make my own mini version of Hibernate to both abstract my SQL commands into Java methods and take data from a database and convert it to simple classes. The idea blew my mind because I always found it almost magical how tools like Spring and Hibernate abstract complex logic into simple classes using annotations and the Java reflection API. If I could learn to do this myself imagine the type of possibilities it would open for me. Maybe I could build something pretty cool like my own backend framework. The thought of this deeply consumed me and I went into a full frenzy mode, coding hard for this project for long hours each day, learning more about reflection and annotation processing, and falling a bit behind on other schoolwork than I would have liked. Despite this, and the newfound stress from the deadlines and studying I would need to catch up with (which by the way, ended up being just fine 😉) I would say it was 100% worth it from all the new things I learned, and the joy of pursuing my passions. In the end, while my Hibernate clone was admittedly a little crusty and nowhere near as good as Hibernate itself, I learned a ton more about some of the more advanced Java features I didn't know about before and had a ton of fun doing this.

After the project was finished, it was a single question I asked myself, a simple yet powerful one that you too can use to draw inspiration, that led me to the cool and ambitious idea of building Stir. The question was: "What cool thing could I build using the knowledge I just acquired?". This question has been the source of many cool project ideas I've had and certainly hasn't failed me here when it comes to Stir. Through this question, various thoughts came to my head but the one that really resonated with me was the idea of using annotations in a class to represent HTML elements that would later be generated from objects. Part of this came from what I've seen going on with Angular components and also a bit of a dislike of Thymeleaf. Nevertheless, it was from here that the idea of Stir came to fruition.

How Stir Was Built

I might try to be a bit briefer and more surface-level after that full novel on the circumstances that led to Stir and how you should follow your passion (I hope that was as interesting to you as I felt it was in my head but if I lost you by now then oh well). I will, however, try my best to give you relevant info on the tools I used and the processes I followed that you could use for your next project. Whether you are a Java developer or not, there are some useful notes here you can take from me that could increase your productivity and potentially the quality of your project.

Design Philosophy

One of the first things I would like to mention was that a good deal of the thought process that was there when building Stir was: "how can I make the best possible experience for my target audience". Thankfully for me, the target audience was myself, a Java developer. From this, I thought hard about "what would this need for it to be something I would genuinely use myself and enjoy in the process?". The answer to this led to my emphasis on trying to make as intuitive of an API as I could with solid documentation, and features to reduce the required amount of code.

While that design philosophy had great impacts on the end project, another design philosophy I had (or lack thereof) has hindered me in some ways. Part of me just wanted to add as many cool features as I could out of interest, excitement, and potentially a fear that Stir wouldn't be good enough in its niche to justify being used. This led to me almost adding totally unrelated and out-of-scope features from the essence of the framework, and design choices that did come into fruition that I admittedly question to this day.

Developmental Practices And Tooling

Project Management

Much of how I organized myself to work on this project came from the agile methodology I learned from school and practiced in my most recent internship (although I honestly slacked off from it later on as the next semester started and I began setting my eyes back to my other goals). Much of this organization came from the use of Zenhub, a GitHub-compatible project management tool I learned from a previous course. Using Zenhub, I set up different sprints (2 week periods where I set on achieving different developmental goals and working on set GitHub issues), created epics (larger goals comprised of a set of related issues), and set estimates for the amount of time for each issue (if possible). From there, I would systematically get goal after goal completed by following this schedule, creating a new branch for every issue, merging the branch when completed, and closing the issue. Following this process would give me a clear direction and keep me going by seeing one issue close after another and progress slowly but surely being made.

Continuous Integration and Deployment

Outside of this, I used GitHub actions for my continuous integration and deployment pipeline. Using the Maven Jacoco plugin (Jacoco meaning Java code coverage) along with codecov.io, I was able to generate code coverage reports with a nice interface via a GitHub bot. Additionally, I used a GitHub action for continuous deployment to GitHub packages (and later Maven central) upon pushing to master or on creating a release, an action to push my Java docs to GitHub pages, and an action to run my tests upon the creation of a pull request. All of this was fairly easy to set up with just simple YAML files and a quick browse through the GitHub actions marketplace.

Testing

Lastly, I would like to talk about the practices I took regarding testing. Since the project is a library and not an application, it made it much more natural to incorporate test automation. This is because, unlike an application, I don't have anything to run but rather the end product is a set of classes, objects, and methods. The tools I used for testing were JUnit for the test setup and Mockito for creating mock objects. In general, I tried to keep my code coverage at around 80% for the classes that contained the core logic of my library. Meanwhile, for the classes meant to represent HTML elements, I was okay with a lot lower percentage because a lot of those classes were boilerplate/repetitive code like getters, setters, builders, etc that were generated by Lombok anyways. Still, I tried to add tests there for instances of the classes with all the builder fields set for future regression testing and to ensure that I did things correctly. Finally, for some of the more complex features I had to implement, I went about it using a test-driven development approach to try to simplify the process. By nature of test-driven development, I get to slowly build up functionality for that feature, test and ensure that functionality works, and iterate on it. That way, the process of implementing that feature becomes less overwhelming since I take it one step at a time and in the process get to build up my test suite.

What I learned and my Final Takeaways

Overall, there are a lot of lessons you and I can take from the experience I had with this project. On the design side of things, try to commit hard to the niche that you want for your project, not trying to overextend those boundaries and focusing more on fulfilling that niche as well as possible for your target audience. On the technical side of things, I talked about my agile processes, tools, and testing habits that are universal to any project. Lastly, on the philosophical or life skills side of things, I would say the main takeaway from this is to just pursue what you're really interested in and don't be afraid to be ambitious and passionate about things. I'm not sure how well you can relate to this but sometimes I feel like I'm not fully in tune with my ambitiousness and passion for my work because others may not relate to it, as well as just how hectic life can become. However, with this project, I was able to be more in tune with that part of me and was thus able to build something cool like this that I probably wouldn't imagine making before, all while embracing a good side of myself. By embracing this side of myself, I was able to live up to my fullest potential and achieve something pretty great which definitely made a positive impact on my confidence. With all that, I hope you could use my experiences to kickstart a super cool side project of your own and get the inspiration to follow your creative visions and passions.

If you like what you read, consider subscribing to my newsletter or following me on Hashnode to be notified of new blog posts. Also, check out my programming tutorials here.

Tip 1: Be persistent and persevere

This is the most important point to be made here. I can attribute much of my success to my determination to get things done which was the quality I was most recognized for. While you will likely be inexperienced and lacking the knowledge and intuition of a more seasoned developer, one thing you can always be on par with or better than others at is working damn hard. Furthermore, because of this lack of experience, how hard you hustle or how hungry you are to get things done is what I feel will have to stand out the most. Assuming you’re around the same level as me, you’ll most likely fumble here and there which is alright and expected from a newbie; worrying too much there can only make you second guess yourself too much and paralyze you. Instead, focus more on what you can control the most: how hard you work, and use it to pick yourself up and persevere through these challenges. Not only that, but you’ll certainly face some unholy bugs here and there that will be a test of your resolve (and also your sanity).

Tip 2: Learn to interpret existing code well

Another skill that helped me succeed was my ability to interpret existing code. Whether you’re debugging or adding enhancements to the application, this skill is critical, especially early on. When it comes to debugging, you’ll need to trace the underlying flow of effected parts of the program so being able to effectively understand it is critical. Even in adding enhancements, you need to understand how similar functionalities were implemented and the existing modules you could use for that feature. In fact, every task I was assigned required me to interpret existing code in one way or another.

Tip 3: Don’t be afraid to ask questions

While this may seem obvious, many people, including myself occasionally, need this lesson. It can feel a little scary at times doing this because of the perception that you’ll be annoying to them. However, as long as you have a legitimate question and are respectful of their time and boundaries, you’d be surprised by how willing others are to lend a hand. Of course, there is a balance to be struck where you need to be sure that it’s not something you can easily find on your own. Even still, you can’t deny yourself from using any resources at your disposal for the job. In fact, the experience of others would be among your most valuable resources, especially as an intern.

Tip 4: Hone in on your debugging skills

Debugging code is arguably the most important skill for a software developer. Other than death and taxes, software having bugs is an unavoidable part of life that you need to accept. No matter how good your QA team is or how thoroughly you test your code, the bad news is that something will go wrong. Hell, even while you are implementing a new feature, I’m willing to bet you’ll stumble upon a problem along the way (unless of course, it's a hello world program). Thus, your ability to debug code will make you a huge asset as a programmer. Moreover, based on what I’ve seen and heard, debugging code will be a larger responsibility for a junior developer so be sure to work on those skills. I could probably write a whole other blog post on debugging code (and maybe I will) but I’ll just leave you with some of the main points.

To start, you need to reproduce the bug as closely as possible yourself. Find the steps and conditions to reproduce the problem and ideally do so in the development environment where you’ll have more tools at your disposal to fix it. This is easily the most fundamental step because through this, you can start to understand where in the codebase the problem manifests and trace the related logic.

As context for those who don’t know, software teams will have separate environments for/copies of the application. Typically, you’ll hear of a development and production environment (there are more but this is for simplicity). The development environment represents an environment you use while developing code, and production environments are what the clients will see.

From there, as I already briefly touched upon, you need to trace through the code involved in this bug. In doing so, be sure to understand and appropriately use all the tools you have at your disposal, whether it be logs, documentation, debuggers, browser developer tools, etc. If you haven’t been able to reproduce the problem in the development environment, it can be impossible to trace it. In this case, I would analyze the problem and nature of the bug as closely as possible using whatever tools I have. With whatever findings I got, I would develop a theory on the possible cause of the problem and attempt a fix from there.

Tip 5: See the bigger picture

As a final and highly important tip, I leave you with the simple idea of seeing the bigger picture. This is something that I struggled with and was told about by my supervisor as being perhaps my biggest weakness. As a junior developer working on a new codebase larger than anything I’ve worked with, it was difficult to see how everything clicked and functioned together as a whole (and understandably so). This can be a problem in that you may end up making changes that affect unintended parts of the codebase which can lead to new bugs. In fact, I saw this happen firsthand with some code I wrote leading to bugs later down the line. The moral of the story is that you need to be careful and understand the scope of your changes before you commit to them. Try to look and see where else your changes might affect the code and if those changes won’t lead to unintended consequences. Always err on the side of caution when it comes to these things, avoiding any assumptions and trying to isolate your changes to only be for the problem at hand.

Conclusion

With that, I hope to have given you some key points on how you can be best equipped for your first tech internship. Using these ideas, I’ve managed to make a fairly large impact where I could, and through other ideas mentioned here, you can avoid some of the pitfalls I fell into. If you have any tips of your own that you want to share, feel free to mention them in the comments below. And with that, I wish you all the best in tackling that first internship. Have fun and give it your all!

If you like what you read, consider subscribing to my newsletter or following me on Hashnode to be notified of new blog posts. Also, check out my programming tutorials here.

For those interested it's a comedy show called The Good Place. Major-ish spoilers ahead for those who care.

In the finale, the main characters finally reached the show's version of heaven (The Good Place) but found a major flaw in it. Because everyone there was given their paradise to enjoy forever, they would be overstimulated for so long and it would eventually feel meaningless and not satisfying anymore. From all the stimulation, they grew weaker in mind and didn't find meaning in all this constant leisure. The angels in charge of the place couldn't find a way to fix the issue and fled, leaving the main characters to solve it for them. The angels tried everything they could to make things more satisfying for everyone by adding brand new things or improving what was already there. What was the solution the main characters found? It was to allow everyone to leave and fade out of existence once they feel satisfied with their stay at The Good Place. The episode subtly and sometimes explicitly gave a deep message surrounding death and life that was legitimately inspiring to be. I thought I would share with you my thoughts behind this idea and how you could apply this to be more productive in your day-to-day life not just as a software developer but as a human being.

WARNING potential existential crisis ahead

Our Time is Limited

Like the people in The Good Place, many of us go through life like our time here is unending. It's like we forget that we are not immortal. Because of this false belief, we often waste our time on things that might be incredibly pleasurable in the short term but don't give us long-term fulfillment or a real sense of purpose. As soon as death comes into our consciousness, we begin to work on the things that matter to us the most for the greater picture of our lives. The main characters, once they implemented death in the afterlife, started trying new hobbies and making great things out of their time because they wanted to make an impact before they inevitably are satisfied with all they experienced. The same can be true for us if we keep the idea of death in our consciousness. Why do you think we always hear about people who go through near-death experiences or are impacted by the death of loved ones trying to do meaningful things with their lives after they recovered? It's because death is the ultimate reminder to use your time wisely or you may never get it back.

Unlike the characters in the show, we can't even choose when we die; it could happen much sooner than we can imagine. The stoic philosopher Marcus Aurelius even noted that a significant amount of our years will be ones of old age where we are too weak or not as mentally sharp to do the things we want. Because of these things, each of us may have much less time on our hands than we act as we do on a day-to-day basis. Most of us (including myself) have probably already wasted it being anxious, afraid of failure, lazy, or even hedonistic at times. If we could be more conscious of death, we could get more done from this harsh reminder that we are slowly dying and can't get any of the experiences and opportunities of life back again. Do you have unfinished business? Could you be working harder towards your goals? If you were to die tomorrow would you be happy about how you spent your days? With these questions in mind, go on your journey as a human being and as a future or current developer, using your time as suited towards these goals.

The Reminder of Death Erases Fear

This may be going into Nihilistic territory but let's face it, once each of us die, probably much of the actions in our lives and their impacts will be worth very little in the grand scheme of history and the universe. Damn, that sounds painful and depressing but there is a good side to this and a practical reason to mention this. With this fact in mind, many of the things we fear and are anxious about may not be the end of the world as we know it. In this way, all this anxiety we face begins to almost seem self-centered. We see petty things like what is this person thinking of me and does this person like me as meaning the world to us when in the end, it won't be anything for the entire world. Do you have a relative that is doubting your career goals as a developer? Well, what harm does the thoughts of others, things you can neither control nor sense, do to you? What will it mean in the grand scheme of everything? Is this worth wasting your limited time and energy that you will never get back? In the words of Marcus Aurelius:

At all times, look at the things itself - the thing behind the appearance - and unpack it by analysis:

• cause
• purpose
• and the length of time it exists

With this and the inevitability of our death in mind, we can begin to judge things with clarity towards the purpose we seek for ourselves.

Furthermore, this mindset also allows us room to take more risks than we normally would be willing to take. With this new perspective of limited time and things being not nearly as fatal as we think, suddenly our limiting beliefs and fears seem silly. Do you want to waste your limited time and energy by being afraid of what others may think? Risk-taking is an important part of the learning process and achieving our goals in life and with this mindset, you can be more mentally prepared to do so. Again, in the words of our friend Marcus:

It is not death that a man should fear, but he should fear never beginning to live.

Don't fear letting your ego die but be afraid of that fear getting in the way of your mission in life. Don't even fear physical death because through it our lives get more meaning as I showed you. So go ahead and chase your dreams based on your own judgments and not that of others. Do more things out of your hopes and values than out of your fears.

But John... by this logic aren't my goals in life futile anyways? Can this not be an excuse to just do whatever the hell I want to do while ignoring the consequences? That doesn't sound right to me.

You do bring up a good point there. The way I see it is that while you alone are very limited in your impact, that isn't an excuse to not do whatever you can to make an impact according to your values. While you can have the depressing Nihilistic mindset that nothing matters and so you shouldn't try anyway, why chose to live with the dread of that thinking? You chose the meaning there is in life based on the values you hold. Value the things that help you make the most out of our short time here. Those values can be in being the best developer you can be, teaching others, creating a startup that improves the quality of life, or whatever suits you. In the end, while you may not be able to move mountains, your contributions will help the collective move them inch by inch. Who knows, maybe you can end up moving it much further than you could have imagined.

Conclusion

In short, if you want to maximize your time, you need to keep in your consciousness the futility of life. Through it, you are given the ultimate reminder that your time here is limited and if you misuse it, you may never get to experience what you truly want. Embrace the futility of it all, remembering that much of what you experience, the fears you thought were overwhelming, the good memories, and likely all of your accomplishments, will eventually be long gone. Yet, this isn't a reason for you to not care about anything. Enjoy what time you have to its fullest; explore, learn, enjoy, strive, accomplish, and don't take for granted your time, energy, belongings, and relationships. As a rule of thumb, think more often of what you would regret or look back to on your deathbed and use that to guide you in your endeavors. As the stoics would say: Memento Mori; remember death. Thanks for reading.

If you like what you read, consider subscribing to my newsletter or following me on Hashnode to be notified of new blog posts. Also, check out my programming tutorials here.

Why do we procrastinate

To start, it's important to understand the reasons why we would procrastinate in the first place. To me, I'd say that it boils down to two factors: being overwhelmed, and excessive instant dopamine.

Dealing with overwhelming problems

When you're overwhelmed by a problem, it's no wonder why you would struggle to tackle it; that's just our nature. Looking at the huge picture of a sizable problem/job for too long can be demoralizing or daunting. It’s as though someone showed you a mountain in your path you have to climb over to get home. You’ll start anxiously thinking about all the hours it will take to climb and the amount of physical effort you’ll bare. As a developer, I remember a long time ago procrastinating on refactoring a large portion of a side project I was working on. While I understood it was important and would make my code better, I held off on it for quite some time because of how daunting it was to tackle. Eventually, though, I ended up biting the bullet and overcame that procrastination using one or two of the tactics I’ll share later on.

Speaking of refactoring, here’s a side tangent you may find useful as a software developer (feel free to skip it if you only care about the main topic of this article). While you may want to be coding new cool features all the time, sometimes you have to learn to take a step back and look to improve your existing code and refactor it. While it may not be the most interesting work, you’ll thank yourself in the long term for making the code easier to read and thus easier to debug. That way, you don’t just code away all day and eventually end up with complete spaghetti code that you can’t easily debug (trust me, I’m still experiencing this to this day with a side project I’m working on).

Excessive dopamine

Now, let's look at the other side of the picture, excessive instant dopamine. While I’m no expert in hormones or biology, I'll be more or less paraphrasing ideas I've heard before so take this with a grain of salt.

Dopamine acts as a sort of reward mechanism for your body. You do good things (or whatever your body may think is good) and you get that good feeling. So naturally, if you give yourself a lot of dopamine through things like video games, food, social media, etc., you'll already be satisfied and not have the motivation to do things that will make you happier in the long term. I know from personal experience that this can be true. Sometimes I would try having break days where I would only do a few pomodoros of work and try to rest. These days, it was far harder for me to get my work done because I would spend my time being lazy, eating junk food, and playing video games a lot of the day before getting to that work. Through that, I would be jamming my brain with dopamine which would make me feel already satisfied and lazy. As another example, I noticed that if I meditate during my Pomodoro breaks instead of scrolling through Reddit, I would subtly produce more productive work sessions. I suspect here that because I'm not giving myself dopamine during the brief break before my work period, my work becomes more interesting. The instant dopamine from those Reddit posts is more satisfying than the delayed dopamine from work. Thus, I can get my brain into action faster if I meditate instead.

How we can apply this knowledge

Now that I’ve broken down the core areas I believe cause procrastination, let’s see how we can use this knowledge to overcome it.

Overcoming the insurmountable obstacle

Instead of looking at the large size of your task and crying that it will take you forever to achieve, you should break down your task step by step and focus your efforts on the step you are at. If you've been following this blog you would have read that I like to break down my workload into pomodoros. With that, I can just focus on tackling my work a few pomodoros a day, focusing on each of them at a time. Moreover, through this process, I only look at the larger scope of things when I need them for the sake of planning. As an analogy, while it may be overwhelming to think of building a brick wall, if you take it one brick at a time it must be completed eventually. Brick by brick, that wall will eventually be built through patience and following the right process. Moreover, if you can split up your large task into a bunch of smaller, easier to complete and reason about tasks, you can more easily tackle the whole problem. As a reference to computer science, think about a divide and conquer algorithm. With these types of algorithms, you break down your problems into a bunch of small problems, solving the elementary ones and combining them into the final solution. So is the same when you encounter large problems in life or work.

Stop yourself from dopamine overload

Now that you are aware of how excessive dopamine can lower productivity or even stop it altogether, snap out of it! Have a plan to avoid too many of these dopamine sources before you work. That is why I normally like to get my work out of the way first before letting myself rest. I’ve even noticed that if I allow myself to play video games before finishing all my pomodoros, I struggle to find the motivation to complete them.

But John… what if I’m already in that place of laziness from excessive dopamine? How can I get out of it?

Well, I’m glad you asked! From my experience, I have a couple of points of advice to give in that case:

Drink Ice Cold Water

I’m not sure what it is about drinking cold water but this ritual seems to work for me to get out of laziness. I suppose the coldness helps wake me up a bit and maybe the hydration helps too.

Take a Cold (or at least room temperature) shower

I find cold showers help wake me up and act as a quick exercise in getting out of my comfort zone. If you can bear freezing cold water pouring all over you, I’m sure you can bite the bullet and get to work. Look, I know this can be a lot easier said than done especially if you are in that lazy state. In which case, I’d say you should at least try not to take a warm shower and strike a middle ground. I find that for whatever reason warmer showers can make me tired and increase that lazy feeling. Thus, I’ve been trying to take colder showers more often and have gotten more used to it (although admittedly not fully and struggle to or not do so some days). It also has the added benefit of being good for your skin and immune system. It might take a bit to get used to but I would definitely recommend it.

Turn on some music and start that timer

I find that putting on music can help at times when I’m not feeling the motivation to work. I just put on my earbuds, open play my Spotify playlist, and start my Pomodoro timer. Having music on while I work can ease the pain of getting through it that I may be feeling at the time. As well, after committing to the Pomodoro method for so long, my brain is wired to get to focus mode as soon as I click the button to start my timer. Thus, I know that all I need to get to that productivity mode I’m seeking is the touch of a button. Then, as soon as I got that simple 25 minutes of work done, my brain has been warmed up and is more interested in this work and can more easily carry on and do more. The moral of the story here is to give yourself whatever small lifts you need to get something, anything, started and let the momentum carry you. Aside from just this, you can maybe do small chores or a quick workout to get the momentum going.

Make the process a habit

Finally, before I wrap up this blog post I want to give a final word of advice. Make the process of reaching your goals a habit. By turning things into habits, you don’t have to think about whether you should do something or not, you get straight to it! For me, I made working out every day (usually mornings) into a habit by seeing it as a part of my daily routine. Because of this, I’ve had an easy time getting myself to work out every day because it just feels like a normal part of my day like eating or brushing my teeth. As something more relatable to you as programmers or potentially computer science students like me, you can maybe study during your commute, work on Leetcode problems for a couple of Pomodoros each day, or read programming books/articles each day.

Conclusion

With that, I gave you my best advice when it comes to overcoming procrastination as a software developer. You saw that by being overwhelmed by your problems and having excessive dopamine you are putting yourself in an easy spot to procrastinate. From there, I gave my personal experience on how you can get yourself out of this trap and avoid it in the first place. I hope this helps you in your endeavors in becoming the best developer you can be. Do feel free to post your questions and/or comments in the comments section below. Happy coding!

If you like what you read, consider subscribing to my newsletter or following me on Hashnode to be notified of new blog posts. Also, check out my programming tutorials here.

Software engineering is much more of a social field than you would think

In software engineering, you might think that you'll just sit around and code isolated from a lot of human interaction. In reality, a lot of software engineering revolves around effective human interaction. From my software engineering course, I've learned that much of the battle is requirements gathering from clients who might not know what they want, are saying, or have different domain expertise than you. With this, a software engineering team must be careful not to play a game of broken telephone and clearly communicate with clients, make sure they are understood, and understand the technical lingo of the application domain they are building a product for. This way, you ensure your code specifications are complete and accurate so you don't end up writing the wrong functionality and waste a lot of time.

Not only that but during the coding phase of project development, a lot of communication will still be required. With other developers, you may need to train them, be trained by them, attend meetings with them, pitch your ideas, do code reviews, or even do some pair programming if your company follows extreme programming or parts of it. That's not even mentioning the amount of communication you may need between management and QA (quality assurance). With how critical human interaction is in software engineering, it would be a huge advantage for you to project the confidence to speak your mind and effectively communicate with others. This way, you can market yourself and ideas to other developers and management, deal with the potential problems that can arise with human egos, and display leadership in these settings to climb the corporate ladder. How could you market yourself and ideas if you don't have confidence in them in the first place?

A lack of confidence can sabotage your learning

This might seem quite hard to believe but hear me out here. You might be thinking that being confident here can hinder your ability to learn through a lack of humility and thinking you are much better than what you are. While these are valid points, that confidence is more so arrogance than real confidence. Here, I'm defining confidence as comfort and acceptance in yourself and your realistic abilities so that no matter what mistakes you make, you won't feel down about yourself. You'll just stay on course and do what needs to be done without personalizing things. The arrogance shown above is different in that it is trying to prove a better picture of yourself to the world and yourself that is not true.

With that disclaimer aside, in what ways does a lack of confidence sabotage your learning? Well, I like to see this as being similar to confirmation bias. If you are doubtful of your abilities and think you aren't good enough, your mind will be looking for and interpreting things as proof of this. How much harder will it be for you to achieve your goals if you are simultaneously looking for proof you can't achieve them? From my personal experience, I've sometimes found myself in doubt thinking: "does this person dislike me?" and interpreting little things as proof of it. Later on, I would learn what these little things really were and that I was too much in my head the whole time. As a more relevant example, when I first started technical writing, I was getting much more critiques from my peer reviewer on my first article. Throughout that whole revision process, I was thinking things like: "Am I not good enough?", "Is there something wrong with me?" or "How much longer will this review process be?" which was draining a lot of my mental space with negative energy. Because of this, I couldn't as effectively learn from constructive criticism or work with as much energy as I normally would. Had these thoughts gone too far, I even could have self-sabotaged myself by giving up and never becoming a technical writer. You probably wouldn't be reading this right now then. Contrarily, had I been more confident in myself and my abilities, I would have had more helpful thoughts such as: "I know I'm making a lot of mistakes right now but it's good my reviewer is showing them to me so I can learn and get better" or "I may not be the greatest right now but through more practice and exposure I'll get there. Keep it going". Overall, without a good baseline level of confidence, you will have a harder time learning through adversity and growing from mistakes. From confidence, you can then bear with failure and see it more for what it is: a learning experience that can't be easily replaced.

On the contrary, you may respond to a lack of confidence by trying too much to prove yourself worthy to others. You might try as hard as possible to show off how great you are and look perfect so that you won't have room for growth. How could you let yourself fail while simultaneously trying to look perfect to other people? Likewise, how much can you learn if you cover up your shortcomings and fail to accept them? While I don't have any strong personal examples of this, I can tell that I've struggled with this to a degree at times. At one point during a hackathon, I've made a flawed design decision of using Spring Boot instead of Firebase. Here, I realized it was because I didn't want to feel unimportant with how much firebase would make my role obsolete as a backend dev for that project. In hindsight I've realized that with the hackathon setting we were in, we were better off sticking with firebase which would have saved us loads of time and built us a better prototype. Had I had the confidence and security in myself to not need to prove my usefulness/worthiness to my team, we would have been better off there. I know that example isn't exactly in the context of learning but it just goes to show that a lack of confidence can draw you away from what's best.

Conclusion

With that, I hope you see how far confidence can go as an aspiring or practicing software developer. Through it, you can stop focusing on destructive thoughts and proving yourself to others and focus harder on the journey to great things. If you have any further thoughts on this idea or this article, feel free to drop them in the comments below. Happy learning!

If you like what you read, consider subscribing to my newsletter or following me on Hashnode to be notified of new blog posts. Also, check out my programming tutorials here.

Some general notes on side projects

Before I begin, I thought I'd share with you some general tips on side projects before you start pursuing one. To start, I'd say that you should aim to follow through with the project to the very end as much as possible. While this seems obvious, I've heard that it's a common phenomenon for developers to have many incomplete side projects on their hands and I certainly have been guilty of that. From my experience suffering this, I can say that I regret not following more projects all the way through. It gives me this weird feeling that I've done so many things but have little concrete code to show potential employers. As well, I’d stress planning out your code well and making sure you don’t write any code you’d regret writing later. Not only will the future you suffer from the pains of hard-to-maintain code, but if you intend to put your code out there to show potential employers, they might become disinterested after seeing poorly written code. As a final note about side projects, I tend to choose project ideas that would help me learn new skills I’m interested in in some way. Since actually coding is the most important thing you must do to get better at it, what better place is there to learn new things than here! To aid you in choosing a project to help with this, I’ll outline what my project suggestions could help you learn in the process.

Simple project Ideas

Text-based RPGs

In my opinion, this is an excellent project to help you better understand the concepts of object-oriented programming. Here, there is a lot of potential to have many large class hierarchies through different types of characters (ex. mages, warriors, archers), items (whether armor or consumable items), and more.

A portfolio site

While this may seem obvious to a lot of you, this is a great use of your time coding outside of school or work. For those who don’t know, this is more or less an online resume of your skills. Not only can you practice and show off your web design and HTML/CSS skills, but you can use it as an effective marketing tool for your skills as a developer. Moreover, as you progress as a programmer, you can further improve it with all the skills you’ve learned.

Stats viewer

With this project, you’ll simply be fetching some kind of stats from an external web API and displaying it in a nice format. This is a nice and easy way for you to learn how to interact with web APIs as you would in a real-world application. To find APIs to work with, I would recommend finding some here. You could even work with sample APIs if you are only interested in experimenting with the basics of HTTP and REST without committing to a full project.

Intermediate project ideas

Minesweeper

This is a project I’ve done maybe around a few years ago or so. I’d say it’s moderately challenging for intermediate developers and it can teach you quite a bit about GUI libraries and development. Logic-wise, it’s quite recursion-heavy if that’s something you need to practice with.

GUI RPG games

Here, you’ll be extending the functionality of your text-based RPG game to work on a GUI. This can help you learn more about concurrency since you’ll need to update the graphics on screen and run calculations simultaneously. Additionally, you can try adding more functionality you might not have added in the text-based version like save files or music.

Advanced project ideas

Algorithm visualizers

Out of all the side project ideas here, this is the only one I’ve never done myself. Despite this, I’m fairly convinced that this can be an incredibly valuable project for you to build. With this project, you’ll be using interactive visuals to demonstrate how famous algorithms work. So for example, you can show a maze and demonstrate how different path-finding algorithms go about finding the way through it, filling it out as it goes. Not only will you have a visually appealing application to show, but you will demonstrate your knowledge of algorithms which is a huge plus

Multiplayer games

This can be a very challenging app to build but at the same time completing it can feel incredibly rewarding. Through this, you can learn different protocols or strategies for real-time communication based on the technologies you choose to use. For example, if you are building off of web technologies you could use WebSockets for a traditional client-server model or WebRTC for a peer-to-peer connection. Additionally, while I’m not well versed in game engines, I know there is a Unity library called mirror which you can use to facilitate networking (if you are learning the game dev side of things).

Conclusion

With that, I hope this gives you some ideas for side projects based on what I’ve built and that you learn from the process of building them. At the end of the day, however, I’d say that you should just do whatever most interests you but keep your learning into consideration as well. That is how I ended up choosing all the side projects here and I found that that strategy has helped me to a large degree. Let me know what you think in the comments below and if you have side project ideas yourself that you would want to share.

If you like what you read, consider subscribing to my newsletter to be notified of new blog posts. Also, check out my programming tutorials here.

While someone could tell you an exact path of what to learn to get to where you want to be (which is just fine) I thoroughly believe mindset is a huge part of the battle too.

This got me thinking: shouldn't I also give you guys an exact path of where I think you should go to become the best web developer you can be? After all, that probably is the million-dollar question you all want to know. In this blog post, I'll be answering just that from the perspective of someone who just landed their first web development internship. If topics like these interest you, consider subscribing to my newsletter or following me on Hashnode so you don't miss my future blog posts.

How this guide will work

This guide on the path to being the best developer you can be will be agnostic of a particular type of web development. In other words, this guide will have use for you whether you want to enter backend development, frontend development, or maybe even UI/UX design (although likely more for backend or frontend development because I can't design stuff to save my life). Despite this, I'll try to give my best insights into where you can go to improve or refine your skills in each particular path. Moreover, I'll also be agnostic of any tech stack and will leave you to choose whichever frameworks interest you in any way. This shouldn't matter all that much in the end because whichever framework you choose, these concepts should still apply.

In terms of format, this guide is separated into different steps in your learning as if you knew nothing about web development. The steps laid out will be listed roughly in chronological order. This doesn't mean that you won't be learning things from the previous step simultaneously at some points, but this should give you a rough outline of what to learn first. You might move on to one step but realize you still need to deepen your knowledge of concepts from another step. In programming, there are always new things to learn and you might discover things you didn't quite know you needed to know before. As well, it's worth noting that the parts on frontend and backend development are more or less mutually exclusive depending on where you want to specialize. However, as you'll see later on, I recommend having some knowledge of both sides.

As a final note before we begin, I want to remind you that this guide is more so about what to learn and when than how to learn this stuff. Ultimately, how well you learn these things is up to you, your strategies, your motivation, and your values (as are most things in life). If you want to learn more about how to learn things, check out my previous blog post on this subject

Learning HTML and CSS

If I was starting over again, I'd say that you can't go wrong with HTML and CSS as a starting point. For you absolute beginners to web development, HTML is the language for making the bare bone structure of your pages whereas CSS is for styling them. Any web developer, whether frontend or backend, should at least have a decent amount of skills in this area (though for frontend even more than decent). It's quite visual to use, can be fun to play around with, and not too difficult to get the hang of. In particular, I'd start with learning at least learn the following:

• The basic syntax of the languages
• The commonly used HTML components
• The commonly used CSS properties
• The Box model
• Flexbox
• Positioning
• Display
• CSS animations
• Pseudo-classes
• Pseudo-elements

Learning Javascript

From there, naturally, the first general-purpose programming language to learn would be Javascript. Like HTML and CSS, if you're a web developer of any kind you should probably know how to use this. Javascript is the de-facto scripting language of the web. It allows you to make your web pages much more interactive by adding brains to them. There's almost no getting around learning some Javascript as a web developer so it is definitely a worthy investment. Even if you hate Javascript and end up not wanting to be a web developer, some of the things you learn from Javascript can be transferred to other languages. Here are the concepts I'd say you'd need to learn at this phase:

• Javascript syntax
• Commonly used functions and objects
• Event listeners
• Browser storage
• Control structures (i.e. for loops, if statements)
• Objects
• DOM manipulation
• Asynchronous javascript
• Fetch

Frontend vs backend development

From here, I'd start thinking more about whether I want to be a frontend or backend web developer. At this point, you'd be much more familiar with frontend web development. However, in general, I would use the following criteria to pick between these two: if I am more interested in the design of pages and the immediate user interaction, then frontend web development is likely for me. If I am very interested in the underlying logic and data flow of an application then backend development is for me. Again, this is only generally true as I understand that the frontend can contain a lot of the application logic too. Explore your options here and see what you like. Whichever path you choose to specialize in, however, I would highly recommend having some understanding of its counterpart as well. That is, a frontend developer should have some understanding of backend development and vice versa. In my opinion, this helps you best understand the entire picture of how a full application would work. This way, you can work more effectively with other disciplines and become a better overall developer. At the same time, it doesn't hurt to be a little more versatile this way. Just don't go overboard and try to learn every concept to be learned and never become a master or even competent at one of them. Specializing in one tech stack should work well enough.

Frontend web development

Let's say that you finally made your decision and went with frontend development. After all, you've grown comfortable with what you've been learning so far which has been, by definition, frontend development. At this point, I'd say it's more important for you to further your underlying skills than to go immediately to the shiny Javascript frameworks out there. At this point, I'd consider learning:

• Responsive design
• Bootstrap and/or other UI frameworks
• CSS preprocessors
• UI/UX design with Figma or other tools (more so if you are interested in the design side of things)
• Cross-browser compatibility

while practicing the fundamental HTML, CSS, and Javascript skills above. As a bonus, I would consider learning ThreeJS to stand out with some cool 3D animations!

Choosing a Javascript framework

After learning the above skills to a good proficiency, only then I would start thinking about learning a Javascript framework. There are many Javascript frameworks out there in the world but the three most popular are Angular, React, and Vue. In case you don't know what a Javascript framework is, these are a set of libraries and tools that help you quickly build larger frontend projects. They also tend to give you one particular streamlined way to build a working application. You shouldn't always need a framework to do things but for larger applications, they definitely should be used. They help give you organization and pre-built code so you don't have to re-invent the wheel every time. Moreover, don't be too scared about choosing the wrong framework since employers recognize that skills in one framework are transferable to another. From my experience getting my first internship recently, I got hired for a role that involves using a backend framework I never even heard of before! Because the framework is MVC based, as is my favorite backend framework, the interviewer saw that I had a useful connection there that would be of value.

As for my suggestions for frontend frameworks to start with, I don't have too many opinions on this area as someone more focused on backend development. However, from my experience, there are a couple of anecdotes I'll leave you with. The first Javascript framework I learned was Angular. This is because I've seen many people online use it with Spring (my backend framework of choice). As someone who started with it, I would not particularly recommend it for a beginner since it's known for having a steeper learning curve (for a good reason). When I started, it immediately threw at me a confusing project structure with a million things that you just couldn't learn right off the bat. On the other hand, I recently tried a bit of this other Javascript framework called Svelte which I found was quite nice. It's relatively intuitive to use and has some amazing interactive tutorials on the official web page for it. While it's not as popular as Angular, React, or Vue, I heard it's one of the most loved frameworks and I can certainly see why. Do give it a shot!

Backend web development

Ah yes, backend development, my personal favorite. Here, I have more experience since I've been on this path myself. I never knew I wanted to be a backend web developer to begin with, let alone the basic concept of backend vs frontend web development! I've tried new things not sure of where I wanted to go. I played around with HTML, CSS, and Javascript for a short while but wasn't super invested. Then eventually I discovered some basic backend web development with Java and got immediately intrigued. From there, I eventually learned more and grew into the developer I am today. Now, with the all-mighty power of hindsight, I'll give you my insights into what I'd tell my younger self to do.

After learning the fundamentals of HTML, CSS, and Javascript, I would start having a look at what backend frameworks there are to use.

But John! Why do I have to learn HTML, CSS, and Javascript first if those are frontend technologies? Can't I just delve straight into backend development?

Well, that's an interesting question. I can see why you might be thinking this way. The reason I recommended it was because much of what you're doing is to cover the data flow and business logic on behalf of a frontend and work closely with them. Your role is mainly to give them database access, process user actions, and/or provide mediums to communicate with others in real-time (a bit less so with WebRTC existing). Therefore, it helps to understand how a frontend might interact with your system so it's worth it to start from there. Additionally, I still find myself writing some HTML, CSS, and Javascript code from time to time as a backend developer. As a prime example, you might need to send emails from your backend so you would need to create email templates. Overall, don't try to cut corners here and take my word for it. You have to see the bigger picture first.

Choosing a backend framework

With that out of the way, let's talk about how you would choose a backend framework to work with. Naturally, since you already know a good amount of Javascript at this point, I think the most obvious choice you can't go wrong with is to use a NodeJS framework. For those who don't know, NodeJS is just Javascript but outside of the web browser. I don't use NodeJS myself but with the popularity of Javascript, there should be many options for frameworks out there. I personally use Spring Boot which is the most popular backend framework for the JVM (the Java virtual machine, i.e., the Java run-time environment). From my experience using it, it's very powerful and fairly intuitive. The major drawback however is that writing a backend with it while following best practices can take a lot of code. Some of that code even feels like boilerplate. You could use a tool like Lombok to aid that or code in the Kotlin programming language but even still, I find that it is a bit of an issue. However, I would still recommend it to anyone who enjoys object-oriented programming and is proficient with Java or Kotlin.

What to learn as a backend developer

From my experience, these are the main topics to learn to get a solid grasp of backend development to build cool stuff (roughly in this order):

• The idea of HTTP and REST
• How to build simple REST endpoints in a backend framework using the language of your choice
• Writing automated tests with your backend framework/language (technically optional but HIGHLY recommended)
• Connecting to and using a database
• How to build a simple CRUD app
• Authentication
• WebSockets
• Push notifications
• Hosting your backend

Data structures and algorithms

Now, we're starting to get into the concepts you need to become the very best rather than to get some footing. While you can code apps without a strong knowledge of data structures and algorithms (as I have), knowledge in this area can separate the mediocre from the very best. In case you don't know, data structures are different ways to store data, and algorithms are sets of instructions to perform a defined task (often involving data structures). With that being said, every programmer should at least have some knowledge in this area. Knowing the concepts in this field can teach you to optimize your code for better run time and less memory usage. Not only that but if you want to land an internship at one of the top tech companies, they will most certainly test you on these concepts. Overall, while this stuff may be pretty dry, it's more than worth the effort in learning (although I would suspect it's not necessary for you interested in UI/UX design). I would even say that you should learn a little bit about this subject while you are learning frontend and backend web development in the previous steps.

While I am no expert in this field by any means, I do know some of the fundamentals and will be learning more as I continue my journey. Here are the types of concepts I know you'll have to cover here:

• The various data structures that exist and their strengths and weaknesses
• The applications of different data structures
• Algorithmic analysis using Big-O notation
• Tail recursion and why it’s useful
• Known algorithms

Learning these concepts can deepen your toolbox as a developer giving you the tools to tackle complex problems you might have never solved before. Furthermore, as a couple of resources to practice or learn these skills, here's what I'd recommend:

• Cracking the Coding Interview by Gayle Laakmann. While I haven't read this book myself, this book is an absolute classic that everyone recommends for programmers to read.
• Leetcode. This is a very popular website for programmers to practice solving technical interview-type questions. It has easy to very difficult coding problems involving data structures and algorithms.

Writing clean and maintainable code

Let's face it, whether you like it or not, you will have to spend most of your time as a programmer debugging and maintaining existing code. Thus, being able to write clean and easily maintainable code in the first place is a highly valuable and sought-after skill. When I think about what separates a mediocre programmer from an excellent one, writing clean and maintainable code is at the top of that list. Believe it or not, writing clean code is often better these days than writing faster code. This is because many applications don't stress performance quite as much while hardware continues to get faster. Thus, being able to write good code that is not as vulnerable to bugs is perhaps the most valuable asset to have on a technical level. Mastering this field includes:

• Selecting good variable names
• Knowledge of software design patterns
• The SOLID object-oriented principles
• Avoiding code repetition
• Testing practices
• Knowledge of anti-patterns/code smells.

You might be wondering about that part about code smells. Did you just say my code stinks! Code smells are bad practices in software development that can often lead to hard-to-maintain systems. Here is one excellent resource my professor showed me on this topic. This resource goes through the types of bad programming practices, how to fix them, the benefits of fixing them, cases where you can ignore them, and more. Another resource on the topic of clean and maintainable code is the book Clean Code by Robert Martin. This is another classic book everyone recommends every programmer should read.

Conclusion

With that, you should have a clear picture of the full path I would recommend going as a new web developer. From my own experience going through this journey over the past few years, this is what I would tell a younger version of myself to do to reach where I want to go more effectively. While I'm still on the journey to becoming the greatest developer I can be to this day, I will possibly update this guide to reflect my deeper experience to come. I hope that through this knowledge I passed on, you can find a clearer path to achieve your dreams as a web developer. Happy coding!

If you like what you read, consider subscribing to my newsletter to be notified of new blog posts. Also, check out my programming tutorials here.

What is this book about?

Based on the title, you could probably assume this book is a complete guide on being a software developer (wow John I had no clue). When the title says it's a complete guide, it really means it. In this book, you can find both things you know you need to know and things you didn't know you needed to know. It covers a long range of topics from software development methodologies, types of jobs, marketing yourself as a developer, going up the corporate ladder, negotiating your salary, and much more. As a whole, it has a great mixture of technical knowledge and advice for improving your soft skills.

About the author

One really interesting thing about this book that sets it apart is its author, John Somnez. He certainly isn't the stereotypical nerdy, socially awkward software developer you'd think of. He has a wide variety of experiences from being a career software developer, an entrepreneur, an actor, a model, a freelancer, a speaker at software development conferences, a blogger, a marathon runner, a bodybuilder, and even a dating coach! While some of these experiences seem to be completely irrelevant, they surprisingly give him a unique and useful perspective that you would probably never find anywhere else. He also has a very pragmatic and sometimes inspirational vibe that has rubbed off on me while listening to the audiobook.

By the way, I would highly recommend getting the audio version of this book because it features the author's real voice and additional content he adds on the spot. These sections add a lot of personality and extra insights I wouldn't want you to miss.

Overall, the author has a very interesting character I admire and that inspired me in many areas of my life.

Who is this book for?

This book is meant for anyone pursuing or in the field of software development no matter the skill level. It's separated into 5 sections which are more or less in order of the stages of a career of a software developer. These include: Getting Started as a Software Developer, Getting a Job, What You Need to Know About Software Development, Working as a Developer, and Advancing your Career respectively. Because of this, you could start at the section that would be most applicable to you. The chapters are disjointed from each other so it shouldn't matter where you start anyways. Even still, I decided to listen to this book from start to end because I was interested in what the author had to say.

How the book is formatted

The book is formatted in a fairly engaging way. It gives a lot of good advice on many topics through relatively short chapters. While this book is very long, it's more so because of the sheer amount of topics it goes over. In total, there are 60 chapters, each on different topics (though some similar) with each around 15 to 30 minutes long. The author even states that each chapter was meant to be like its own blog post in a way. As you can imagine then, this book is a lot more breadth-first than depth-first in how it delivers its content. It gives you a great starting point on each of its topics and additional resources to look deeper. Despite this, I still found it gave me great insights and lessons in each of the ideas/scenarios it talked about that didn't feel rushed. With the wide variety of topics covered, you can even use this book as a life manual that you can refer to throughout your life and career as a developer.

The technical content of this book

As mentioned earlier, this book is a great mixture of technical and non-technical content related to being a software developer. While this book isn't meant to teach you deep technical skills in any area, it can certainly help you get a good starting point. In particular, it goes over a general overview of what source control is, the types of source control systems out there, types of testing, test-driven development, unit-testing, web architecture, continuous integration, and more.

Out of all the technical content, I most enjoyed the part on testing and test-driven development. He gave me some great extra insight/perspective on why you should unit-test, the true purpose of it, and much of what you need to understand about testing terminology and practices. Additionally, I found the section on software development methodologies to be quite informative. In this section, the author gives a wide overview of the common software development methodologies you'll see in the industry. He even had experience coaching teams to follow these methodologies and shares his pet peeves when it comes to doing so. Aside from this, he gives some book recommendations at the end for technical books he believes any software developer should read.

The non-technical content of this book

This book also features much of the soft-skills side of being a software developer that you might not have known you needed to know. These include climbing the corporate ladder, speaking at software development conferences, getting job security, dealing with your boss and co-workers, becoming a freelancer, building a business, and more. In particular, I think the chapters on starting a blog and work-life balance are great or maybe among the most applicable to you.

The chapter on work-life balance, my favorite one, preaches you to find a meaningful job that you can enjoy doing. This way you don't confine your life into the limited amounts of time outside of your work and therefore dread most of your life. It encourages you to make the most out of your time at work and enjoy every moment of it as much as you can since dreadfully waiting to go home for the weekend decreases the quality of your life. Overall, I found that chapter to be quite insightful and philosophical making it absolutely a good read.

As for the chapter on starting a blog, I imagine you reading this might have the idea of starting a tech blog in the back of your mind. This chapter gives you insights into why you should build a blog, how to start a blog, and how to grow a successful one. The main message in this chapter is to find a niche for you to stand out as one of the best in and to then consistently put out new content. He argues that he has never seen anyone who puts out consistent content not eventually have some success. Overall, this chapter is a nice starting point if you want to build a successful tech blog and has great advice you should consider.

The overall theme of this book

Before I wrap up this review, I wanted to share what I believe to be the underlying theme of much of the content of this book. Under the surface of much of the advice of this book is one central message: to achieve great things in your career and life as a developer, you need to go above and beyond, be persistent, try extraordinary things, and give great value to others. The section of the book that most emphasizes this is the section on getting a job. In this section, John talks about how the traditional approach of spamming job applications won't be the approach that yields you the greatest results. Instead, he argues to think outside the box, trying extraordinary things to show a potential employer that you can give them way more value than overhead and should therefore be hired. This includes begging to do some dirty work for cheap to prove your tenacity and dedication, networking with employees of companies you are interested in, taking the initiative to improve a company's product yourself, building a strong online presence to bring companies to you, and more. Additionally, in his section on starting a business, John emphasizes that if you want to be successful, you need to give your customers great value, even for free at some point to build trust. Overall, if there is one lesson to be learned from this book, this should be it.

Conclusion

While I could go on and on about how great I think this book is, I should start cutting myself off now before this blog post becomes longer than the book itself. Overall, I would highly recommend giving this book a shot and would say it has great lessons for anyone looking to be or already a software developer. This book can train you not just in how to get the technical knowledge you need but also in developing the intangibles that make a great software developer. Because of this, I believe this book has the potential to make a huge difference in your path to being the best software developer you can be.

If you like what you read, consider subscribing to my newsletter to be notified of new blog posts. Also, check out my programming tutorials here.