Zookeeper notes

Zookeeper handles coordination tasks between multiple activities, cooperation and contention, sharing state for distributed application. Before same problems were solved via DB primary.

It is just a remote tree (with such called znodes). Znodes can have data associated with, binary format. No partial writes are allowed, whole data can be added/replaced/deleted only.

Types of tasks to solve:

  • master election
  • master/worker crash detection
  • membership/clastering
  • common data storage (small amount)

Operations allowed:

  • create znode (with optional data)
  • delete znode
  • check if znode exists
  • get/set data on existing node
  • get znode’s children

Kinds of nodes:

  • persistent – can be deleted via delete operation
  • ephemerial – can deleted via delete operation or automatically, if client created the node, crashed or simply closed connection. No children allowed here.
  • sequential – assigned with unique increasing integer.

Tracking node state changing:

  • polling
  • notification: set a “watch” – one-time trigger, reacts on change of some znode. Should be reset each time it fires, if znode should be constantly monitored.

Znode has a version associated with it, it allows conditional operation (delete and setData), means optimistic lock.

Zookeeper operates as standalone server or quorum multi-server (ensemble), clients should establish and keep connection session with server.

Quorum (odd number) – amount of running servers to run an ensemble, or amount of servers to store data (to avoid brain-split problem)

Session can be moved to another server within ensemble, transparently to client. Client’s commands within same session are strongly FIFO executed on server.


Can be JMX monitored.


Java Spring Cloud notes


  • configurations
  • service registration
  • service discovery
  • load balancing
  • circuit breakers
  • messaging

Change of configuration setting are not pulled by clients for updates, but it can be turned-on (@Sheduled, @RefreshScope annotation) or even Spring Cloud Bus can be used (EnvironmentChangeEvent).

Ribbon – client side load balancing, @LoadBalanced for RestTemplate and for WebClient. Retry logic can be enabled.

Configuration usually locates at GIT (or other SCM), for debug purposes it can be stored locally as file. Vault backend can be used. JDBC backend for configuration storing. Config client retry.

Discovery server heartbeats clients

Discovery Client – @EnableDiscoveryClient: for Eureka, Consul, Zookeeper

Hystrix – circuit breaker

Hystrix Dashboard – allows to track Hystrix clients individually or through accumulated stream of multiple Hystrix dashboards via Turbine.

Feign – REST client

Archaius – external configurations

Spectator, Servo, Atlas – metrics

Cloud Stream – application communicates with external world through input and output channel

Binder for Kafka and RabbitMQ

Consumer group (like AWS’s target group) – set of competing members, only on is given with particular message. But all groups, subscribed for same source of messages get a copy of data. If no group specified, each service is considered a member of anonymous single-item consumer group.

Reactive streams a supported as well: RxJava, Reactor

Aggregation – connection inputs and outputs together, to avoid load on broker.

Binder – connection to particular Broker (RabbitMQ or Kafka)

Schema based message converters (out-of-the-box is Avro only supported for the moment), Schema registry stored schemas.

Spring Cloud Bus – delivering configuration changes or infrastructure management instructions to microservices (RabbitMQ or Kafka).

Sleuth – distributed tracing. Span – request+response (usually it is HTTP request+response, but can be written manually, like transaction, or in form of annotation, assigned on Runnable interface). Set of spans – tree-like structure, span and all its children. Zipkin – trace visualisation.

Consul – Service Discovery, Control Bus and Configuration.

Spring Cloud Contract – like customer written acceptance tests, to make sure that any service fits its microservice environment.

Spring Cloud Vault Config – stores configs and secrets for microservice application (HashiCorp Vault).


Java Spring Boot notes

Important maven artifacts:

  • spring-boot-starter-parent (versions)
  • spring-boot-maven-plugin (executable jar)


Main features:

  • BOM, bills of materials (spring-boot-starter-parent or spring-boot-dependencies)
  • Autoconfiguration (@EnableAutoConfiguration, @ComponentScan, @Configuration, or @SpringBootApplication), with exclude and redefine config, from annotation or yaml-config.
  • creating own configurations
  • embedded servlet (version 3.1) containers: Tomcat, Jetty, UnderTow, Netty
  • eeveloper tools (can be automatically removed while running fully packaged application): caching, automatic restart/reload, remote automatic restart/reload
  • starters
  • actuators
  • CommandLineRunner
  • custom health checks and info
  • custom metrics


Auto-configuration for:

  • caching function returned cached values (it is possible to update or evict)
  • messaging
  • WebClient is better than RestTemplate (at least it is reactive)
  • bean validation
  • metrics
  • http tracing (last 100 calls)

Java framework: Spring Core

Each bean has an ID in scope of context (by default it is modified class name, can be specified explicitly)

Container implementations:

  • bean factories (simple)
  • application context (advanced)
    • internationalization
    • event publication
    • resource management
    • life-cycle events

Popular application contexts:

  • AnnotationConfigApplicationContext – java based application context
  • AnnotationConfigWebApplicationContext – java based web context
  • ClassPathXmlApplicationContext
  • FileSystemXmlApplicationContext
  • XmlWebApplicationContext

Bean lifecycle

  • instantiate
  • set properties
  • setBeanName()
  • setBeanFactory()
  • setApplicationContext()
  • postProcessBeforeInitialization() (from BeanPostProcessor interface)
  • afterPropertiesSet() (from InitializingBean interface)
  • custom init method
  • postProcessAfterInitialization() (from BeanPostProcessor interface)
  • destroy() (from DisposableBean interface)
  • custom destroy method

Bean configuration:

  • explicit XML
  • explicit Java config
  • implicit (component scanning and autowiring)


  • @Component/@Named – marks bean (more specific @Service, @Repository, @Controller)
  • @ComponentScan – turns on component scanning (XML: <context:component-scan>)
  • @Configuration – marks java configuration class – current package and subpackages will be scanned for @Component marked classes
  • @Profile/@ActiveProfiles/@IfProfileValue
  • @Conditional – if condition is true – bean gets created
  • @Autowired/@Inject – marks a destination for bean wiring (if no beans wired, exception is thrown – default, of null reference left – can be enabled explicitly, if more than one bean found – exception is thrown)

Destination for wired beans:

  • constructor
  • setter
  • method

What can be wired:

  • references on other objects/beans
  • literals
  • collections (list, map, set)

Mitigating ambiguity:

  • @Primary – marked bean will have precedence over other matching beans
  • @Qaulifier – additional level of matching between @Component and @Autowired

Scoping @Scope bean annotation, instantiation mode:

  • singleton (default) – only one instance is created
  • prototype – each time new instance
  • session – WEB specific
  • request – WEB specific
  • global session – for portlet-based WEB application
  • thread
  • custom

WEB specific beans are wired through proxy (in case of interface, java dynamic proxy creation) or through inheritance (CGLib)

Spring contexts can be nested to properly separate logic domains, beans from child context can refer beans from parent context.

Spring bean’s method can be replaced (via xml configuration) by other bean’s method.

Beans can have multiple names (but single ID), and aliases to names/id.

SBT extract


  1. Short, concise DSL, can be extended by pure Scala code
  2. Interactivity
  3. Background execution
  4. Default parallel execution (restriction on CPU, network and disk can be specified)
  5. Scala REPL integration
  6. Incremental compilation
  7. Default folder structure (can be adjusted)
  8. Defined workflow (can be adjusted or redefined)
  9. Type safety
  10. Direct dataflow between tasks is supported
  11. Simple script entities hierarchy, just tasks and settings, some already defined, but it is easy to add custom
  12. Crossbuild (for several Scala versions in parallel)
  13. Plugin extensible

Folder structure

  • <root>/project/plugins.sbt
  • <root>/project/build.properties
  • <root>/build.sbt

SBT tasks, executing items, can depend on other tasks (use other task return value inside body), can accept a user input.

  • Declare key: val keyName = taskKey[keyType](“key description”)
  • Assign value: keyName := …
  • Get value: keyName.value

SBT setting – just a named value, can dependent only no literal or value of other setting. The exact value is determined during starting script up. It cannot depend on some task return value.

  • settingName := settingValue – for assign (redefine, if already defined)
  • settingName += settingValue – for append single value to Seq
  • settingName ++= settingValue – for append Seq to Seq


  • project
  • configurations – namespaces for keys (default: Compile,Test, Runtime, IntegrationTest)
  • task
  • global – default, if not specified

Multiproject – can be declared as single or multiple (own for each project) sbt file. Abstract parent project can have common settings, added or redefined by concrete child projects. dependsOn – defines dependency.

Sources (compile/test configurations):

  • location settings: javaSource, resourceDirectory, scalaSource.
  • filtering: includeFilter, excludeFilter.
  • Managed: autogenerated by SBT or added explicitly into build.
  • Unmanaged: created outside of SBT, written by coder.

Dependencies (compile/test/runtime):

  • internal (between projects) or external (on some lib outside – maven / ivy)
  • external can be: managed (maven / ivy) or unmanaged (jars from lib folder)
  • resolvers – setting that can be added with additional maven/ivy external repositories.

Dependency format:  ModuleID – “groupID/organisation” % or %% “artifactID/product” % “version” (optional: “test”, “provided”)

  • exclude – specified dependency will be omitted (additionaly rules can be applied)
  • classifier – additional parameters, like JDK version
  • intransitive or notTransitive – do not load dependencies
  • withSources
  • withJavadoc
  • externalPom
  • externalIvy

Forking – execution Test or Run in separate JVM, custom settings can be applied

Session – memory mapped SBT configuration, will be lost after reload, can be saved as SBT file.

SBT script troubleshooting: streams.value.log

Extending SBT: commands and plugins

Publishing artifact: publishTo


Functional approach specifics

Pure functions – no side effects, nothing else than transforming input parameters into returning value. State is kept outside, it can be passed into function and returned from it.


  • First-class citizen functions
  • High order functions
  • Anonymous functions
  • Closures
  • Currying
  • Lazy evaluation

Referential transparency – function call can be replaced by its returning value

Higher-order-function – function that accepts as an input other function

Partial function application (currying) – transform function with two parameters into function with one parameter, giving the dropped parameter a some value

Functional data structures – are immutable.

Instead of exceptions -> Option, Either, Result – returning values

Laziness – evaluating parameter when it used, but not declared

Memoization – remembering some resource-consuming operation’s result, and returning for later calls (some kind of caching)

Strict functions – all parameters are evaluated before function call, non-strict functions – with lazy parameters

Recursive functions consumes a data, corecursive – produces, both need terminating condition.

Monoid – set of some type, binary operation that takes two instances of the type and returns a new instance of the type, and empty “identity” element


  • computation context for other value
  • container, with two (or three) function defined: unit (constructor) and flatMap OR unit and compose OR unit, map, join

It is always possible to transform impure function into pure one, with two side-effects, one producing the pure function input, other consumes pure function output.

Effect – an object that contains an operation that produces side-effect, it has a method that executes a side-effect

Side-effect – is actually non-functional operation

Docker notes

Terms and definitions

  • Image – particular set of layers
  • Repository – set if images, fully identified by <host name>/<user name>/<repository name>
  • Tag – particular image name (docker tag command)
  • Index/registry – catalog of repositories

Docker architecture

  • Docker daemon – does all work, has REST interface (can be exposed out from PC)
  • Docker client – connects docker-daemon
  • Each container has its own PIDs’ tree
  • Possible state of container: running, paused, restarting, exited

Create an image

  1. Altering manually some running container and committing the changes
  2. Docker-file
  3. Docker-file and some external configuration tool
  4. Docker-file and TAR-file (that contains all files from existing PC) enrolled over zero-image


  • “docker run” command
    • docker run -it <image name> – run container interactively, with connected console
    • docker run -d <image name> – run detached
    • Giving name to container –name <name>
    • –read-only – filesystem of container can be read only
    • –restart =<policy: always, no, on-failure>
    • –link <imagename:containername> binds containers via exposed port
    • –device – maps hosts devices into container
    • –ipc – shares ipc items between containers
    • –cpuset-cpus – limits CPU cores used by container
    • -c/–cpu-shares – percentage of CPU allowed for container
    • -m/–memory – limits amount of memory, accessible for container
  • Port mapping -p <host port number:container port number> <image name>, or -P – map all exposed ports
    • –icc=false – disables network communication between containers
    • –expose <port number>
    • –hostname <name>
    • –dns <ip addresses’ array>
    • –add-host <host name>:<ip address>
    • –link – connects containers by name, since before container starts, no ip address are known
  • Docker network archetypes:
    • closed (only loopback), –net none
    • bridged (can communicate one to another, but have to be explicitly configured to access external network), –net bridge
    • joined (different containers shares the same network stack), –net container
    • opened (connected directly to external network), –net host
  • Adding environment variable —env/-e <name>=<value>
  • Restarting containers –restart with options: never (default), always, on failure (with optional delay)
  • Volumes mapping – map host file system to container’s file system. Mapped folders are not committed, mapped folders hides existing container’s folder with the same name
    • -v/–volume <host path>:<container path> – for mounted host’s paths, can be mounted as read-only
    • -v/–volume <container path> – for docker-managed volume
    • Data only container (no need to run ever), can map some host folder and other containers can just reference it, to obtain the mapped folder
    • –volumes-from <container name>
  • –rm – removes container after it is exited
  • “docker inspect” – Return metadata about some image, it is JSON formatted, fancy filtering syntax
  • “docker kill” – kills container
  • “docker stop” – stops container (gracefully)
  • “docker build” – creates new image from docker-file, –no-cache – build all commands, otherwise only changed docker file instructions will be built, non-changed will be taken from store cached at previous build
  • “docker tag” – gives a name to particular image
  • “docker commit” – creates an image from running container, only filesystem’s changes are preserved.
  • “docker exec” – execute a command over running container (basic- synchronously, daemon- background, interactive)
  • “docker search” – search image at registry
  • “docker history” – lists commands executed in order to build the image specified
  • “docker help” <command name>
  • “docker ps” – list containers run
  • “docker logs” <container name> – shows an output (stdout stderr), -f option allows autoscroll output logs
  • “docker restart” <container name> – restarts container
  • “docker rename” <new container name> <old container name>
  • “docker ps” (-a) – list of running (and other states) containers
  • “docker create” – creates, but doesn’t starts container (exited state)
  • “docker start” <container name> – start exited container
  • “docker top” <container name> – lists all processes running inside container
  • “docker rm” <container name> – remove exited container
  • “docker login/logout” – access a registry
  • “docker rmi” <repository name> – remove local repository
  • “docker rm” <container name> – remove container, -v – remove/decrement reference on docker managed volumes
  • “docker save” – saves an image as a file
  • “docker load” – loads image from file
  • “docker diff” <container name> – shows filesystem differences between container and its image
  • “docker export” – saves a container as a tar-archive
  • “docker import” – loads a container from a tar-archive
  • Docker machine – turns some PC (virtual or real) into host for running containers, runs instance of “machine” – the process is a platform for running containers. It is command line utility with a several commands: create, ls, stop, start, restart, rm, kill, inspect, config, ip, url, upgrade.

Docker-file – script that adjust an image before running its as a container

  • FROM – existing image tag
  • MAINTAINER – author’s mail
  • ONBUILD – executes a command specified at build step
  • RUN – command to run
  • USER – sets user and group
  • WORKDIR – sets current directory
  • EXPOSE – port erxposed
  • ADD – add files into container from some host, unpacks a tar-file
  • COPY – like ADD, but with no unpacking
  • CMD – commands that executed as container’s main process
  • ENTRYPOINT – like CMD but with no parameters (expected provided via run commnad)

Compose – run an application, represented by a set of containers, represented by yaml-file

  • “docker-compose up” – run a dicker yaml file with definitions
  • “docker-compose ps” – all containers run by yaml-file
  • “docker-compose rm” – remove all containers represented by yaml-file
  • “docker-compose stop/kill” – like “docker”
  • “docker-compose logs” – like “docker”
  • “docker-compose build” – like “docker”
  • “docker-compose scale” – alter a number of instances of containers

Docker-machine – represents driver that allows to run a docker-daemon on different hosts

docker-swarm – cluster of machines to run a container, it can balance on basis of accessible resources (Spread algorithm) as well as on basis of custom filters (affinity, health, constraint, port, dependency), with builtin service discovery