How to Migrate a Microservice from MySQL to MongoDB

0 Shares

In the previous Part 1 & Part 2 tutorial series, we decomposed a monolithic Spring Boot application into 3 different microservices using Spring Cloud. All the microservices have their own relational database. The user-auth-service is the core service that provides all the basic functionalities like user registration, authentication, and social login. In this tutorial, we are going to migrate the MySQL database of this service to MongoDB.

Migration from Relational to NoSQL Database

In the past, I have received a few requests from readers to implement this project using a NoSQL database. One of the main advantages of the microservice architecture is that we can implement each service with different technologies. Hence, we are gonna migrate the relational database to the NoSQL database for the user-auth-service leaving the other 2 services as they are with the MySQL database.

Basically, this tutorial may not provide all the details required to migrate from RDBMS to the NoSQL database in a real-world application. For instance, we had defined a bi-directional many-to-many association for user and role entities in the relational database. But, now we are going to embed the user role details directly into the user document in the MongoDB which may not be possible always. In that case, we might need to store them as separate documents and define the relationship between them. Consequently, when we work with multi-documents, we have to deal with atomicity and transaction management.

Nevertheless, for many scenarios, the denormalized data model (embedded documents and arrays) will continue to be optimal for your data and use cases. That is, for many scenarios, modeling your data appropriately will minimize the need for multi-document transactions.

Therefore, this tutorial is intended to give you a basic idea about migration. Before we dive deep into this topic, let’s first get a basic understanding of Relational vs NoSQL databases.

What is NoSQL Database?

NoSQL databases (aka “not only SQL”) are non-tabular databases and store data differently than relational tables. NoSQL databases come in a variety of types based on their data model. The main types are document, key-value, wide-column, and graph.

Relational Database vs NoSQL Database Consideration

Consider a NoSQL database If your data is unstructured/semi-structured, frequently changes, and/or relationships can be de-normalized data models.

Consider a relational database If your data is highly structured, requires referential integrity, and relationships are expressed through table joins on normalized data models.

You can refer here for more considerations on choosing the right database for your application.

What is MongoDB?

MongoDB is a document-oriented NoSQL database that provides support for JSON-like storage.

What you’ll do?

In a nutshell, we are going to perform the following steps in the user auth microservice.

Replace MySQL dependency and its configurations with MongoDB
Convert JPA Entities to POJO’s
Convert JPA Repositories to Mongo Repositories
Change the User ID type from String to Long in all the places

What you’ll need?

IntelliJ or any other IDE of your choice
JDK 11
MySQL Server 8
MongoDB
node.js

User Auth Service

Adding MongoDB Dependencies

pom.xml

Firstly, we need to replace the mysql-connector-java with spring-boot-starter-data-mongodb dependency. Also, we need to remove spring-boot-starter-data-jpa dependency since we no longer need JPA/Hibernate for connecting to the NoSQL database.

<dependency>
	<groupId>org.springframework.boot</groupId>
	<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>

Converting JPA Entities to MongoDB Documents

User.java

Removed all the JPA/Hibernate-specific annotations. @Document annotation is used to set the collection name that will be used by the model. MongoDB will create the collection If it doesn’t exist. It will create the collection based on the class name if the collection name is not specified.

package com.javachinna.model;

import com.fasterxml.jackson.annotation.JsonIgnore;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.springframework.data.mongodb.core.mapping.Document;

import java.io.Serializable;
import java.util.Date;
import java.util.Set;

/**
 * The persistent class for the user database table.
 * 
 */
@NoArgsConstructor
@Getter
@Setter
@Document("users")
public class User implements Serializable {

	/**
	 * 
	 */
	private static final long serialVersionUID = 65981149772133526L;

	private String id;

	private String providerUserId;

	private String email;

	private boolean enabled;

	private String displayName;

	protected Date createdDate;

	protected Date modifiedDate;

	private String password;

	private String provider;

	private boolean using2FA;

	private String secret;

	@JsonIgnore
	private Set<Role> roles;
}

Role.java

We will embed the role document within the user document which is generally known as the “denormalized” model and takes advantage of MongoDB’s rich documents. Embedded data models allow applications to store related pieces of information in the same database record. As a result, applications may need to issue fewer queries and updates to complete common operations. Hence, no more many-to-many relationships between the users and roles.

@Id annotation is used to specify the MongoDB document’s primary key _id.

If we don’t specify anything and if we have a field named id, then it will be mapped to the ‘_id‘ field.
Else, MongoDB will generate an _id field while creating the document.

As per the official Spring documentation, the following outlines what type of conversion, if any, will be done on the property mapped to the _id document field.

If a field named ‘id’ is declared as a String or BigInteger in the Java class, it will be converted to and stored as an ObjectId if possible. ObjectId as a field type is also valid. If you specify a value for ‘id’ in your application, the conversion to an ObjectId is detected by the MongoDB driver. If the specified ‘id’ value cannot be converted to an ObjectId, then the value will be stored as is in the document’s _id field.
If a field named ‘ id’ is not declared as a String, BigInteger, or ObjectID in the Java class then you should assign it a value in your application so it can be stored ‘as-is’ in the document’s _id field.
If no field named ‘id’ is present in the Java class then an implicit ‘_id‘ field will be generated by the driver but not mapped to a property or field of the Java class.

Here we have used the @Id annotation to specify the primary key field since the field name is roleId.

package com.javachinna.model;

import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import org.springframework.data.annotation.Id;

import java.io.Serializable;
import java.util.Set;

/**
 * The persistent class for the role database table.
 * 
 */
@Getter
@Setter
@NoArgsConstructor
public class Role implements Serializable {
	private static final long serialVersionUID = 1L;
	public static final String USER = "USER";
	public static final String ROLE_USER = "ROLE_USER";
	public static final String ROLE_ADMIN = "ROLE_ADMIN";
	public static final String ROLE_MODERATOR = "ROLE_MODERATOR";
	public static final String ROLE_PRE_VERIFICATION_USER = "ROLE_PRE_VERIFICATION_USER";

	@Id
	private String roleId;

	private String name;

	private Set<User> users;

	public Role(String name) {
		this.name = name;
	}

	@Override
	public int hashCode() {
		final int prime = 31;
		int result = 1;
		result = prime * result + ((name == null) ? 0 : name.hashCode());
		return result;
	}

	@Override
	public boolean equals(final Object obj) {
		if (this == obj) {
			return true;
		}
		if (obj == null) {
			return false;
		}
		if (getClass() != obj.getClass()) {
			return false;
		}
		final Role role = (Role) obj;
		if (!role.equals(role.name)) {
			return false;
		}
		return true;
	}

	@Override
	public String toString() {
		final StringBuilder builder = new StringBuilder();
		builder.append("Role [name=").append(name).append("]").append("[id=").append(roleId).append("]");
		return builder.toString();
	}
}

AbstractToken.java

Here, we have removed the @MappedSuperClass annotation since it is not required.

@NoArgsConstructor
@Getter
@Setter
public abstract class AbstractToken implements Serializable {
	/**
	 * 
	 */
	private static final long serialVersionUID = 1L;

	private static final int EXPIRATION = 60 * 24;

	private String id;

	private String token;

	private User user;

	private Date expiryDate;

VerificationToken.java

package com.javachinna.model;

import org.springframework.data.mongodb.core.mapping.Document;

@Document("tokens")
public class VerificationToken extends AbstractToken {

	private static final long serialVersionUID = -6551160985498051566L;

	public VerificationToken(final String token, final User user) {
		super(token, user);
	}
}

Creating Mongo Repositories

Now we need to modify the repositories to extend the MongoRepository interface instead of JpaRepository. Also, we need to modify the ID type from Long to String

UserRepository.java

package com.javachinna.repo;

import com.javachinna.model.User;
import org.springframework.data.mongodb.repository.MongoRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface UserRepository extends MongoRepository<User, String> {

	User findByEmail(String email);

	boolean existsByEmail(String email);
}

VerificationTokenRepository.java

package com.javachinna.repo;

import com.javachinna.model.VerificationToken;
import org.springframework.data.mongodb.repository.MongoRepository;

public interface VerificationTokenRepository extends MongoRepository<VerificationToken, String> {

	VerificationToken findByToken(String token);
}

Modifying Service Layer to Access the Repositories

UserService.java

Modified the ID type from Long to String

Optional<User> findUserById(String id);

UserServiceImpl.java

Here, we have removed the logic that fetches the role from the database since we are going to embed the user roles within the user document itself. Also, modified the ID type from Long to String.

@Service
@RequiredArgsConstructor
public class UserServiceImpl implements UserService {

	private final UserRepository userRepository;

	private final VerificationTokenRepository tokenRepository;

	private final PasswordEncoder passwordEncoder;

	private final SecretGenerator secretGenerator;

	private final MailService mailService;
	
	@Override
	@Transactional(value = "transactionManager")
	public User registerNewUser(final SignUpRequest signUpRequest) throws UserAlreadyExistAuthenticationException {
		if (signUpRequest.getUserID() != null && userRepository.existsById(signUpRequest.getUserID())) {
			throw new UserAlreadyExistAuthenticationException("User with User id " + signUpRequest.getUserID() + " already exist");
		} else if (userRepository.existsByEmail(signUpRequest.getEmail())) {
			throw new UserAlreadyExistAuthenticationException("User with email id " + signUpRequest.getEmail() + " already exist");
		}
		User user = buildUser(signUpRequest);
		Date now = Calendar.getInstance().getTime();
		user.setCreatedDate(now);
		user.setModifiedDate(now);
		user = userRepository.save(user);
		return user;
	}

	private User buildUser(final SignUpRequest formDTO) {
		User user = new User();
		user.setDisplayName(formDTO.getDisplayName());
		user.setEmail(formDTO.getEmail());
		user.setPassword(passwordEncoder.encode(formDTO.getPassword()));
		final HashSet<Role> roles = new HashSet();
		roles.add(new Role(Role.ROLE_USER));
		user.setRoles(roles);
		user.setProvider(formDTO.getSocialProvider().getProviderType());
		user.setEnabled(false);
		user.setProviderUserId(formDTO.getProviderUserId());
		if (formDTO.isUsing2FA()) {
			user.setUsing2FA(true);
			user.setSecret(secretGenerator.generate());
		}
		return user;
	}

	@Override
	public User findUserByEmail(final String email) {
		return userRepository.findByEmail(email);
	}

	@Override
	@Transactional
	public LocalUser processUserRegistration(String registrationId, Map<String, Object> attributes, OidcIdToken idToken, OidcUserInfo userInfo) {
		OAuth2UserInfo oAuth2UserInfo = OAuth2UserInfoFactory.getOAuth2UserInfo(registrationId, attributes);
		if (!StringUtils.hasText(oAuth2UserInfo.getName())) {
			throw new OAuth2AuthenticationProcessingException("Name not found from OAuth2 provider");
		} else if (!StringUtils.hasText(oAuth2UserInfo.getEmail())) {
			throw new OAuth2AuthenticationProcessingException("Email not found from OAuth2 provider");
		}
		SignUpRequest userDetails = toUserRegistrationObject(registrationId, oAuth2UserInfo);
		User user = findUserByEmail(oAuth2UserInfo.getEmail());
		if (user != null) {
			if (!user.getProvider().equals(registrationId) && !user.getProvider().equals(SocialProvider.LOCAL.getProviderType())) {
				throw new OAuth2AuthenticationProcessingException(
						"Looks like you're signed up with " + user.getProvider() + " account. Please use your " + user.getProvider() + " account to login.");
			}
			user = updateExistingUser(user, oAuth2UserInfo);
		} else {
			user = registerNewUser(userDetails);
		}

		return LocalUser.create(user, attributes, idToken, userInfo);
	}

	private User updateExistingUser(User existingUser, OAuth2UserInfo oAuth2UserInfo) {
		existingUser.setDisplayName(oAuth2UserInfo.getName());
		return userRepository.save(existingUser);
	}

	private SignUpRequest toUserRegistrationObject(String registrationId, OAuth2UserInfo oAuth2UserInfo) {
		return SignUpRequest.getBuilder().addProviderUserID(oAuth2UserInfo.getId()).addDisplayName(oAuth2UserInfo.getName()).addEmail(oAuth2UserInfo.getEmail())
				.addSocialProvider(GeneralUtils.toSocialProvider(registrationId)).addPassword("changeit").build();
	}

	@Override
	public Optional<User> findUserById(String id) {
		return userRepository.findById(id);
	}

Modifying Spring UserDetailsService

LocalUserDetailService.java

Modified the ID type from Long to String in the loadUserById method

@RequiredArgsConstructor
@Service("localUserDetailService")
public class LocalUserDetailService implements UserDetailsService {

	private final UserService userService;

	@Override
	@Transactional
	public LocalUser loadUserByUsername(final String email) throws UsernameNotFoundException {
		User user = userService.findUserByEmail(email);
		if (user == null) {
			throw new UsernameNotFoundException("User " + email + " was not found in the database");
		}
		return createLocalUser(user);
	}

	@Transactional
	public LocalUser loadUserById(String id) {
		User user = userService.findUserById(id).orElseThrow(() -> new ResourceNotFoundException("User", "id", id));
		return createLocalUser(user);
	}

	/**
	 * @param user
	 * @return
	 */
	private LocalUser createLocalUser(User user) {
		return new LocalUser(user.getEmail(), user.getPassword(), user.isEnabled(), true, true, true, GeneralUtils.buildSimpleGrantedAuthorities(user.getRoles()), user);
	}
}

Modifying Token Provider and Filter

Modified to remove the parsing of the User ID string to Long.

TokenAuthenticationFilter.java

@Override
	protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
		try {
			String jwt = getJwtFromRequest(request);

			if (StringUtils.hasText(jwt) && tokenProvider.validateToken(jwt)) {
				String userId = tokenProvider.getUserIdFromToken(jwt);

				UserDetails userDetails = customUserDetailsService.loadUserById(userId);
				Collection<? extends GrantedAuthority> authorities = tokenProvider.isAuthenticated(jwt)
						? userDetails.getAuthorities()
						: List.of(new SimpleGrantedAuthority(Role.ROLE_PRE_VERIFICATION_USER));
				UsernamePasswordAuthenticationToken authentication = new UsernamePasswordAuthenticationToken(userDetails, null, authorities);
				authentication.setDetails(new WebAuthenticationDetailsSource().buildDetails(request));

				SecurityContextHolder.getContext().setAuthentication(authentication);
			}
		} catch (Exception ex) {
			logger.error("Could not set user authentication in security context", ex);
		}

		filterChain.doFilter(request, response);
	}

TokenProvider.java

public String createToken(LocalUser userPrincipal, boolean authenticated) {
		Date now = new Date();
		Date expiryDate = new Date(now.getTime()
				+ (authenticated ? appProperties.getAuth().getTokenExpirationMsec() : TEMP_TOKEN_VALIDITY_IN_MILLIS));
		String roles = userPrincipal.getAuthorities().stream().map(item -> item.getAuthority())
				.collect(Collectors.joining(","));
		return Jwts.builder().setSubject(userPrincipal.getUser().getId()).claim(ROLES, roles)
				.claim(AUTHENTICATED, authenticated).setIssuedAt(new Date()).setExpiration(expiryDate).signWith(key)
				.compact();
	}

	public String getUserIdFromToken(String token) {
		Claims claims = parseClaims(token).getBody();
		return claims.getSubject();
	}

Note: We will remove these JWT related custom providers and filters from the code in our next tutorial since Spring itself has in-built support for token based authentication.

Modifying DTOs

SignUpRequest.java

Modified userID type from Long to String

@Data
@PasswordMatches
public class SignUpRequest {

	private String userID;

	private String providerUserId;

	@NotEmpty
	private String displayName;

Creating Users on Application Startup

We no longer need to persist roles separately since we embed role documents within user documents. Hence, we have removed the RoleRepository and its related code.

SetupDataLoader.java

package com.javachinna.config;

import com.javachinna.dto.SocialProvider;
import com.javachinna.model.Role;
import com.javachinna.model.User;
import com.javachinna.repo.UserRepository;
import lombok.RequiredArgsConstructor;
import org.springframework.context.ApplicationListener;
import org.springframework.context.event.ContextRefreshedEvent;
import org.springframework.security.crypto.password.PasswordEncoder;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Transactional;

import java.util.Calendar;
import java.util.Date;
import java.util.Set;

@Component
@RequiredArgsConstructor
public class SetupDataLoader implements ApplicationListener<ContextRefreshedEvent> {

	private boolean alreadySetup;

	private final UserRepository userRepository;
	private final PasswordEncoder passwordEncoder;

	@Override
	@Transactional
	public void onApplicationEvent(final ContextRefreshedEvent event) {
		if (alreadySetup) {
			return;
		}
		// Create initial roles
		Role userRole = new Role(Role.ROLE_USER);
		Role adminRole = new Role(Role.ROLE_ADMIN);
		Role modRole = new Role(Role.ROLE_MODERATOR);
		createUserIfNotFound("[email protected]", Set.of(userRole, adminRole, modRole));
		alreadySetup = true;
	}

	private User createUserIfNotFound(final String email, Set<Role> roles) {
		User user = userRepository.findByEmail(email);
		if (user == null) {
			user = new User();
			user.setDisplayName("Admin");
			user.setEmail(email);
			user.setPassword(passwordEncoder.encode("admin@"));
			user.setRoles(roles);
			user.setProvider(SocialProvider.LOCAL.getProviderType());
			user.setEnabled(true);
			Date now = Calendar.getInstance().getTime();
			user.setCreatedDate(now);
			user.setModifiedDate(now);
			user = userRepository.save(user);
		}
		return user;
	}
}

Modifying Main Application Class

UserAuthServiceApplication.java

Since we are not using JPA/Hibernate, we can remove the @EnableJpaRepositories and @EnableTransactionManagement annotations.

package com.javachinna;

import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;
import org.springframework.cloud.netflix.eureka.EnableEurekaClient;

@SpringBootApplication(scanBasePackages = "com.javachinna")
@EnableEurekaClient
public class UserAuthServiceApplication extends SpringBootServletInitializer {

	public static void main(String[] args) {
		SpringApplicationBuilder app = new SpringApplicationBuilder(UserAuthServiceApplication.class);
		app.run();
	}

	@Override
	protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
		return application.sources(UserAuthServiceApplication.class);
	}
}

Modifying Application Properties

Remove MySQL/JPA/Hibernate-related properties and add MongoDB URI

application.properties

server.port=8084
spring.application.name=user-auth-service
#spring.profiles.active=dev
spring.config.import=configserver:http://localhost:8888
# Database configuration props
spring.data.mongodb.uri=mongodb://localhost:27017/demo

Testing

You can follow the instructions here to run and test this application with MongoDB.

Source Code

Spring Cloud Microservices Demo

Conclusion

That’s all folks. In this article, we have migrated our microservice from the MySQL database to MongoDB.

Thank you for reading.

0 Shares