Organize evolutions

Play Framework
1

Hy!

We have an “old” project with more than 350 evolutions. Is there a way to organize these files to multiple directories? Like 1-50/ 51-100/ etc? It is really hard at this point to find out what is the last “magic number” and also not really user-friendly in most editors to list 300-500 files… I don’t want to modify them, just organize to subdirs.

1 Like
2

Not sure about sub-foldering. As an alternative, you might consider consolidating evolutions 1-100, 101-200 and so on. This will reduce the number of files, and make the setup of new database quicker.

3

This question is a perfect use case for a custom implementation of an EvolutionsReader (which still needs documentation):
I copied an example from an existing project of mine (also have a look at the code comments with links to similiar resources):

package my.organisation.evolutions;

import java.io.IOException;
import java.io.InputStream;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.Optional;

import javax.inject.Inject;
import javax.inject.Singleton;

import com.typesafe.config.Config;

import play.Environment;
import play.Logger;
import play.api.db.evolutions.Evolutions;
import play.api.db.evolutions.ResourceEvolutionsReader;
import scala.Option;
import scala.compat.java8.OptionConverters;

/**
 * From
 * https://github.com/playframework/playframework/blob/2.8.0/persistence/play-jdbc-evolutions/src/main/scala/play/api/db/evolutions/EvolutionsApi.scala#L522-L558
 *
 * <p>Also see: https://github.com/playframework/playframework/issues/2813
 * https://gist.github.com/mkurz/061d413715f004c8c963
 */
@Singleton
public class MyEvolutionsReader extends ResourceEvolutionsReader {

  private final Environment environment;
  private final Config config;

  @Inject
  public MyEvolutionsReader(final Environment environment, final Config config) {
    this.environment = environment;
    this.config = config;
  }

  @Override
  public Option<InputStream> loadResource(final String db, final int revision) {
    // here you need to implement your logic like if revision 1-50, load the script from the the "50" folder, etc.
  }
}

You need to override the loadResource method, which looks up the evolution file and returns it as an input stream. IMHO that is exactly what you need.

Then you have to put that into a module:

package my.organisation.evolutions;

import com.typesafe.config.Config;
import play.Environment;
import play.api.db.evolutions.*;
import play.inject.Binding;
import play.inject.Module;

import java.util.List;

public class EvolutionsModule extends Module {

  /**
   * See
   * https://github.com/playframework/playframework/blob/2.8.0/persistence/play-jdbc-evolutions/src/main/scala/play/api/db/evolutions/EvolutionsModule.scala#L20-L23
   */
  @Override
  public List<Binding<?>> bindings(Environment environment, Config config) {
    return List.of(
        bindClass(EvolutionsConfig.class).toProvider(DefaultEvolutionsConfigParser.class),
        bindClass(EvolutionsReader.class).to(MyEvolutionsReader.class),
        bindClass(EvolutionsApi.class).to(DefaultEvolutionsApi.class),
        bindClass(ApplicationEvolutions.class)
            .toProvider(ApplicationEvolutionsProvider.class)
            .eagerly());
  }
}

And enable your custom module and disable Play’s default one:

play.modules.disabled += "play.api.db.evolutions.EvolutionsModule"
play.modules.enabled += "my.organisation.evolutions.EvolutionsModule"

I hope that helps.

4

Nice!

For the lazy ones (most code copypasted from the original reader);


import play.api.db.evolutions.{Evolutions, ResourceEvolutionsReader}
import play.api.{Environment, Logger}

import java.io.InputStream
import java.net.URI
import javax.inject.{Inject, Singleton}
import scala.annotation.tailrec

@Singleton
class DirectoryBatchedEvolutionsReader @Inject() (environment: Environment) extends ResourceEvolutionsReader {
  private val logger = Logger(this.getClass)

  def bucketRevision(rev: Int): String = {
    val b = (rev - 1) / 50
    (b * 50 + 1) + "-" + ((b + 1) * 50)
  }

  def fileNameGen(db: String, revisionS: String, revision: Int): String =
    s"${Evolutions.directoryName(db)}/${bucketRevision(revision)}/${revisionS}.sql"
  def resourceNameGen(db: String, revisionS: String, revision: Int): String =
    s"evolutions/${db}/${bucketRevision(revision)}/${revisionS}.sql"

  def loadResource(db: String, revision: Int): Option[InputStream] = {
    @tailrec def findPaddedRevisionResource(paddedRevision: String, uri: Option[URI]): Option[InputStream] = {
      if(paddedRevision.length > 15) {
        uri.map(u => u.toURL().openStream()) // Revision string has reached max padding
      } else {
        val evolution = {
          // First try a file on the filesystem
          val filename = fileNameGen(db, paddedRevision, revision)
          environment.getExistingFile(filename).map(_.toURI)
        }.orElse {
          // If file was not found, try a resource on the classpath
          val resourceName = resourceNameGen(db, paddedRevision, revision)
          environment.resource(resourceName).map(url => url.toURI)
        }.orElse {
          // fallback to old filesystem mode
          val filename = Evolutions.fileName(db, paddedRevision)
          environment.getExistingFile(filename).map(_.toURI)
        }.orElse {
          // fallback to old classpath mode
          val resourceName = Evolutions.resourceName(db, paddedRevision)
          environment.resource(resourceName).map(url => url.toURI)
        }

        for {
          u <- uri
          e <- evolution
        } yield logger.warn(
          s"Ignoring evolution script ${e.toString.substring(e.toString.lastIndexOf('/') + 1)}, using ${u.toString
            .substring(u.toString.lastIndexOf('/') + 1)} instead already"
        )
        findPaddedRevisionResource("0" + paddedRevision, uri.orElse(evolution))
      }
    }
    findPaddedRevisionResource(revision.toString, None)
  }
}
1 Like