Blended ZIO Core

Functionality that is required by all blended containers.

Configuring Blended Containers#

As outlined here all modules that require configuration should be able to use external configuration files containing place holders to specify lookups from environment variables or resolve encrypted values.

For example, the configuration for an LDAP service might be:

{
  url             : "ldaps://ldap.$[[env]].$[[country]]:4712"
  systemUser"     : "admin"
  systemPassword" : "$[(encrypted)[5c4e48e1920836f68f1abbaf60e9b026]]"
  userBase"       : "o=employee"
  userAttribute"  : "uid"
  groupBase"      : "ou=sib,ou=apps,o=global"
  groupAttribute" : "cn"
  groupSearch"    : "(member={0})"
}

The ZIO ecosystem has a library called zio-config which supports different sources such as property files, HOCON, YAML or even the command line. At the core of the library are ConfigDescriptors which can be used to read the config information into config case classes. The descriptors are also used to generate documentation for the available config options or reports over the configuration values used within the application.

Following the advice from the zio-config library author on discord, we introduce a LazyConfigString as follows:

Descriptor

sealed abstract case class LazyConfigString(value: String)

object LazyConfigString {

  final case class Raw(raw: String) {
    def evaluate(
      ctxt: Map[String, String]
    ): ZIO[StringEvaluator.StringEvaluator, EvaluatorException, LazyConfigString] = for {
      se  <- ZIO.service[StringEvaluator.Service]
      res <- se.resolveString(raw, ctxt)
    } yield (new LazyConfigString(res) {})
  }

  val configString: ConfigDescriptor[Raw]               =
    string(Raw.apply, Raw.unapply)
  def configString(path: String): ConfigDescriptor[Raw] = nested(path)(configString)

}

Essentially we define a class LazyConfigString, which instances will eventually hold the resolved config value. Making the class sealed and abstract ensures that new instances can only be created from within the companion object.

Within the companion object the case class Raw can be instantiated with Strings read from the config sources. Also, within this class the evaluate method holds the effect describing the resolution of the raw config string to a real value. Essentially we are deferring the resolution to a StringEvaluator service.

At last we need to provide a config descriptor for LazyConfigStrings, so that the generated documentation will reflect that the config values are subject to lazy evaluation.

Using the LazyConfigString, we can define the LDAPConfig as follows:

Sample Config Descriptor

object LDAPConfig {

  import LazyConfigString._

  def desc: ConfigDescriptor[LDAPConfig] = (
    configString("url") ?? "The url to connect to the LDAP server" |@|
      configString("systemUser") |@|
      configString("systemPassword") |@|
      configString("userBase") |@|
      configString("userAttribute") |@|
      configString("groupBase") |@|
      configString("groupAttribute") |@|
      configString("groupSearch")
  )(LDAPConfig.apply, LDAPConfig.unapply)
}

case class LDAPConfig(
  url: LazyConfigString.Raw,
  systemUser: LazyConfigString.Raw,
  systemPassword: LazyConfigString.Raw,
  userBase: LazyConfigString.Raw,
  userAttribute: LazyConfigString.Raw,
  groupBase: LazyConfigString.Raw,
  groupAttribute: LazyConfigString.Raw,
  groupSearch: LazyConfigString.Raw
)

To access a config value, a layer with a StringEvaluator must be referenced:

Access Config

  private val desc: ConfigDescriptor[LDAPConfig] = LDAPConfig.desc
  private val ctxt: Map[String, String]          = Map("user" -> "ADMIN", "env" -> "dev", "country" -> "es")

  private def simpleEval(src: ConfigSource) = testM("Evaluate a simple config map")(
    (for {
      cfg <- ZIO.fromEither(read(desc.from(src)))
      pwd <- cfg.systemPassword.evaluate(ctxt)
    } yield assert(pwd.value)(equalTo("blended"))).provideLayer(evalLayer)
  )

With the code above, zio-config will generate the following report in markdown format:

note

Field Descriptions#

FieldName	Format	Description
url	primitive	lazily evaluated config string, The url to connect to the LDAP server
systemUser	primitive	lazily evaluated config string
systemPassword	primitive	lazily evaluated config string
userBase	primitive	lazily evaluated config string
userAttribute	primitive	lazily evaluated config string
groupBase	primitive	lazily evaluated config string
groupAttribute	primitive	lazily evaluated config string
groupSearch	primitive	lazily evaluated config string

Evaluate simple string expressions#

Lazy evaluated string expressions are simple expressions as defined here:

sealed trait StringExpression
case class SimpleExpression(value: String)                                      extends StringExpression
case class SequencedExpression(parts: Seq[StringExpression])                    extends StringExpression
case class ModifierExpression(modStrings: Seq[String], inner: StringExpression) extends StringExpression

The notable piece here is the ModifierExpression, which has the form

$[modifier*[StringExpression]]

A modifier expression contains an inner expression and evaluation will be from the innermost expression outwards. After resolving the inner expression, zero or more modifiers will be applied to the resolved value for a given context. The context is a simple Map[String, String] and the normal resolution simply maps the resolved expression to the corresponding value in the map.

For example, the expression $[[foo]] with the context map Map("foo" -> "bar") will yield "bar".

Modifiers will be applied to the value resolved from the context map, for example with the context map from above

$[(upper)[foo]] => "BAR"
$[(left:2)[foo]] => "ba"

Modifiers are specified as:

Modifier

trait Modifier {
  def name: String
  def op(s: String, p: String): ZIO[Any, Throwable, String]

  def lookup: Boolean = true

  final def modifier(
    s: String,
    p: String
  ): ZIO[Any, ModifierException, String] = (for {
    input <- ZIO.fromOption(Option(s))
    param  = Option(p).getOrElse("")
    mod   <- op(input, param)
  } yield mod).mapError {
    case me: ModifierException => me
    case t: Throwable          => new ModifierException(this, s, p, t.getMessage)
    case _                     => new ModifierException(this, "", p, "Segment can't be null")
  }
}

note

A modifier implementation can override lookup to avoid that the value resolved from the inner expression will be used to look up the final value from the context map.

The EncryptModifier does that, so that the decryption will be applied to the string resolved from the inner expression.

Simple crypto service#

The EncryptModifier is defined as

Decryption Modifier

object EncryptedModifier {

  def create: ZIO[CryptoSupport.CryptoSupport, Nothing, Modifier] = for {
    cs <- ZIO.service[CryptoSupport.Service]
    mod = new Modifier {
            override def name: String = "encrypted"

            override def lookup = false

            override def op(s: String, p: String): ZIO[Any, Throwable, String] = (for {
              res <- cs.decrypt(s)
            } yield (res))
          }
  } yield mod
}

It relies on a crypto service available within the ZIO environment and simply delegates the resolution to the decrypt method of that service.

The crypto service is defined as

Simple Crypto Service

  trait Service {
    def encrypt(plain: String): ZIO[Any, CryptoException, String]
    def decrypt(encrypted: String): ZIO[Any, CryptoException, String]
  }

The default implementation can be instantiated with a password, for convenience the code also contains a default password. The password can also be provided via a file. Essentially, the provided password is used to generate a key that is then used to create an instance of a CryptoService which simply wraps some Crypto methods from Java:

Simple Crypto Service Implementation

final private class DefaultCryptoSupport(key: Key, alg: String) {

  def decrypt(encrypted: String): ZIO[Any, CryptoException, String] = for {
    bytes     <- string2Byte(encrypted)
    ciph      <- cipher(Cipher.DECRYPT_MODE)
    decrypted <- ZIO.effect(ciph.doFinal(bytes.toArray)).refineOrDie { case t => new CryptoFrameworkException(t) }
  } yield (new String(decrypted))

  def encrypt(plain: String): ZIO[Any, CryptoException, String] = for {
    ciph  <- cipher(Cipher.ENCRYPT_MODE)
    bytes <- ZIO.effect(ciph.doFinal(plain.getBytes())).refineOrDie { case t => new CryptoFrameworkException(t) }
    res   <- byte2String(bytes.toSeq)
  } yield res

  private def cipher(mode: Int): ZIO[Any, CryptoException, Cipher] =
    ZIO.effect { val res = Cipher.getInstance(alg); res.init(mode, key); res }.refineOrDie { case t =>
      new CryptoFrameworkException(t)
    }

  private def byte2String(a: Seq[Byte]): ZIO[Any, Nothing, String] =
    ZIO.collectPar(a)(b => ZIO.succeed(Integer.toHexString(b & 0xff | 0x100).substring(1))).map(_.mkString)

  private def string2Byte(s: String, orig: Option[String] = None): ZIO[Any, CryptoException, Seq[Byte]] = {
    val radix: Int = 16

    (s match {
      case ""                               => ZIO.succeed(Seq.empty)
      case single if (single.length() == 1) =>
        ZIO.effect(Seq(Integer.parseInt(single, radix).toByte)).refineOrDie { case _: NumberFormatException =>
          new InvalidInputException(orig.getOrElse(s))
        }
      case s                                =>
        string2Byte(s.substring(2), Some(orig.getOrElse(s)))
          .map(rest => Seq(Integer.parseInt(s.substring(0, 2), radix).toByte) ++ rest)
    })
  }
}

Using the services#

To use the services resolving config string, a layer with all required services must be provided:

Layer Provisioning

  private val logSlf4j = Slf4jLogger.make((_, message) => message)

  private val cryptoDefault: ZLayer[Any, Nothing, CryptoSupport.CryptoSupport] =
    CryptoSupport.default.orDie

  private val mods: ZIO[Any, Nothing, Seq[Modifier]] = EncryptedModifier.create.provideLayer(cryptoDefault).map { em =>
    Seq(UpperModifier, LowerModifier, em)
  }

  private val evalLayer: ZLayer[Any, Nothing, StringEvaluator.StringEvaluator] =
    logSlf4j >>> StringEvaluator.fromMods(mods)

This layer can be provided to an effect by the means of provideLayer

Layer Access

  private val desc: ConfigDescriptor[LDAPConfig] = LDAPConfig.desc
  private val ctxt: Map[String, String]          = Map("user" -> "ADMIN", "env" -> "dev", "country" -> "es")

  private def simpleEval(src: ConfigSource) = testM("Evaluate a simple config map")(
    (for {
      cfg <- ZIO.fromEither(read(desc.from(src)))
      pwd <- cfg.systemPassword.evaluate(ctxt)
    } yield assert(pwd.value)(equalTo("blended"))).provideLayer(evalLayer)
  )

Finally, the config can be resolved from a config source created from a Map with ConfigSource.fromMap:

Sample config class

object LDAPConfig {

  import LazyConfigString._

  def desc: ConfigDescriptor[LDAPConfig] = (
    configString("url") ?? "The url to connect to the LDAP server" |@|
      configString("systemUser") |@|
      configString("systemPassword") |@|
      configString("userBase") |@|
      configString("userAttribute") |@|
      configString("groupBase") |@|
      configString("groupAttribute") |@|
      configString("groupSearch")
  )(LDAPConfig.apply, LDAPConfig.unapply)
}

case class LDAPConfig(
  url: LazyConfigString.Raw,
  systemUser: LazyConfigString.Raw,
  systemPassword: LazyConfigString.Raw,
  userBase: LazyConfigString.Raw,
  userAttribute: LazyConfigString.Raw,
  groupBase: LazyConfigString.Raw,
  groupAttribute: LazyConfigString.Raw,
  groupSearch: LazyConfigString.Raw
)