Kotlin notes for Java/Scala developers

Main features

  • type inference
  • nullable types
  • function types
  • interactive shell
  • online playground
  • java-to-kotlin converter

 

Language feature

  • smart casts, (type check + cast)
  • fun (function) can be declared on level of file (no need to put it into file), no static utility classes are required. For java callers such functions will be placed as static functions into auto-generated class (named after kotlin file). Class name can be redefined, via @file:JvmName(“StringFunctions”).
  • return type can be skipped for expression-body function
  • string templates/interpolation (${expression} or $variable)
  • val/var – like java’s final/not-final
  • public by default
  • properties (val – only getter, var – getter and setter), round braces can be omitted, even for javabean objects
  • “package” and “import” directives – like Java’s
  • classes, class members, functions and properties can be imported
  • it is possible to have folder hierarchy not like packages hierarchy (unlike Java), but it is good practice to have follow java way in case of mixed project java+kotin
  • good practice to have multiple classes within same file (since classes are small)
  • enum classes
  • semicolon is used only to separate enum constants from enum methods (if defined any)
  • “when” instead of “switch”
  • “is” like “instaceof” but no additional cast is required afterwards (within underlying block)
  • last expression at block its result
  • while/do-while like java’s, “for” like java’s for-each
  • “in” (“!in”) keyword checks if an item is inside some collection
  • closed range – <start>..<end>
  • java’s “try-with-resources” is implemented as library function
  • “”” – triple quoting string – nothing need be escaped
  • local functions
  • Any class (open) – like java’s Object

 

Functions

  • creating collections:
    • setOf
    • hashSetOf
    • arrayListOf
    • hashMapOf (key to val)
  • named parameters (but not for java methods)
  • default parameters
  • @JvmOverloads – makes from kotlin function with default parameters a set of overloaded java functions, for each default parameter
  • extension functions and properties (adding methods to other’s class), can access public methods and properties of class receiver. From java side extension methods seen as static members of class named after kotlin file name. Could not be overridden, called by variable declared type, i.e. statically resolved. Member function is always takes precedence over extension function.
  • varargs – “vararg”, but arrays should be unpacked, “*” – spread operator
  • infix call – “infix” keyword before function definition, like making a pair: “1 to 2”
  • destruction declarations: val (one, two) = 1 to 2
  • “inline” – marked functions are not called as a functions but embedded into caller’s body (can spare time for call and size for object that internally represents a lambda, but increase a size of byte-code). Inline functions called from java as usual functions, with no in-linement. Type information for template parameters are reified (not erased) at runtime for such functions. it is better to use for functions with lambda parameters, in other cases compiler is smart enough, can injects itself:
    • not every lambda (passed as a parameter to in-lined function) can be inlined (if is is not called, but only stored as local variable for instance)
    • to prevent inlining some lambda parameter of in-lined function, “noninline” specification can be used for such parameter
  • “withLock” – multithread protect
  • “use” – alternative to java’s try-with-resources

 

Imports

  • aliasing
  • class, method, property from file or static members of class can be imported
  • * – like java’s

 

Classes

  • “super” – calls parent parent’s method, “super<type>” – to avoid ambiguity in case few parents with the same method, unlike java, it is not mandatory to have it as first method call
  • “open” – class or/and method, means not final,  should be explicitly specified, for both class and method.
  • “final” – like java’s class/method, could not be inherited/overridden, but unlike java, it is by default, if omitted any modifier. Used if opened function was overridden (overwriting keeps opened state) but further overwriting should be prohibited (function should be closed)
  • “abstract” – class/method that should be overwritten, opened by default, methods do not have a body.
  • visibility (public by default):
    • public
    • protected – only visible by subclass, no package-wide like java
    • internal – module-wide visibility (becomes public in java)
    • private – like java, but with exception, that outer class doesnt see private members of inner or nested class.
  • nested class – doesnt depend on outer class, “static” specified at java, but default for kotlin
  • inner class – depend on outer class, default for java, but “inner” specification should be used with kotlin.
  • sealed – defines nested classes of some sealed base class or at the same kotlin file where base sealed class defined. Opened by default. Used to avoid default branch at “when” construct.
  • main constructor is defined outside class body (“init” block and “constructor” keyword together are used if default logic should be customized)
  • class ClassName private constructor() {} – with private constructor
  • constructor(params): super/this(params) {} – secondary constructor
  • visibility of properties can be changed
  • compiler generated (can be overridden, inherited from Any)
    • toString(), like java
    • equals – called while ==, compares content, not references by default, === in koltin for reference equality
    • hashCode – like java’s, equal objects should have equal hash codes, but not vice versa
    • copy – to support immutability
  • data class – actual toString, equals and hashCode are generated on basis of properties declared only at primary constructor.
  • by – kotlin support for decorator design pattern
  • overloading operations (“operator fun” as a member or extension function), no “commutativity” (left<->right):
    • binary: times, div, mod, plus, minus, <op name>Assign (for compound operations), it conflicts with not compound, better use only one, or use val and var parameter to separate them
    • unary: unaryPlus, unaryMinus, not, inc, dec
    • equals – override from Any, not an operator
    • compareTo – overide from Any, not an operator, mapped to <, >, <=, >=, even an object has come from java
    • index operator: get/set
    • range belongings operator: “in” – contains
    • rаnge generation operator: “..” – rangeTo
    • loops iterating: “in” – iterator
    • destructing declaration: componentX()
  • delegated properties: “by” on property level, an access to property is delegated to corresponding property of another object (which implements getValue/setValue methods)
  • lazy property initialization “by lazy <lambda>”
  • storing property values at map, i.e. dynamic properties
  • <class name>::class.java – gets kotlin class for specified java class.

 

Object

  • singleton – can inherit classes and interfaces
  • companion – defined with a class and can call private class members, its members can be accessed as outer class members
  • anonymous inner object – can implement a several interfaces

 

Lambdas

  • blocks of code within curvy brackets
  • it – current element
  • syntax : var_name: var_type, … -> body
  • if lambda is a last parameter of function it can be placed after closing round bracket after parameter list
  • if lambda is a single argument, and it is placed outside of parameter list, empty round brackets can be omitted.
  • clojures not final variables
  • reference to class function ClassName/InstanceName::FunctionName, or to top-level function ::FunctionName, or constructor ::ClassName
  • kotlin lambda can be passed to java method with lambda parameter, if no variables captured by lambda – same lambda instance is used all the time (otherwise, each time new lambda is created)
  • java’s SAM != kotlin lambda, SAM constructor, named as corresponding functional interface
  • “with” – parameter can be used implicitly at body of lambda, return result is a last expression of lambda
  • “apply” – called as an extension, always returns an object that it has been called on, which can be used implicitly within a lambda expression, as with “with”. Useful for object initializations.
  • local return from lambda: “return@label”, jumps execution flow after a block of code marked with “label”, function name can be used instead of “label”
  • anonymous functions, it is just  form of lambda expression, “fun (params) {body }” – if return specified, it performs local return (no label required), and as a general rule: “return” returns from closest “fun”.

 

Interfaces

  • can contain property declaration, abstract methods, non-abstract methods (like Java 8), but no state allowed.
  • override modifier in front of function (like java’s @override) is mandatory

 

Collections

  • same “map”, “filter”, “find”, “any”, “groupBy”, “flatMap” (map+flatten) operations like at java
  • lazy operations: sequences, like Java, but could not be paralleled, can be created from a collection, or be generated (initial value+lambda)
  • create an immutable collection “<collection type>Of” or “<mutable collection type>Of”: list, arrayList, set, hashSet, linkedSet, sortedSet, map, hashMap, linkedMap, sortedMap, arrayOf
  • mutable and immutable collections
  • primitive types arrays: IntArray, CharArray, BooleanArray, ByteArray, etc

 

Nullability

  • nullable types – <type_name>?
  • java’s functionality @Nullable and @NotNull
  • if nullable variable is checked it is not a null, it becomes non-nullable within a context of check being performed
  • safe call operator ?. – calls a method or return a null as a result
  • elvis operator ?: – provides default value if null (or do something, like throwing exception)
  • safe cast as? – return null if cast is failed, otherwise keeps  variable casted with a context
  • !! – assertion that checks nullable variable is not null, and returns it
  • “let” function accepts lambda and in case of non-null executes body ad provides “it” with a value of object it is called on, otherwise nothing happens
  • “lateinit” – for non-null properties, that should be left not initialized within constructor. Always “var”.
  • extension function can be called on null values (of nullable types)
  • template parameter is nullable by default, <T: Any> – to specify a template type as non-nullable
  • java’s (platform, means no nullability information provided) types – type_name! – can be used for both: nullable and non-nullable. Kotlin recognizes java’s annotations @Nullable, @NotNull, etc. But in case no annotations are specified, java’s type is treated as “platform” type.

 

Generics

  • by default generic parameter considered as nullable
  • no row type, like java
  • like java, inclduding type boundaries, just “:” instead of java’s “extends”
  • reified, like “inline fun <reified T>” – to preserve type information at runtime, some (all) parameters can be reified, such functions cannot be called from java code;
    • can be used for: “is”, “!is”, “as”, “as?”, “::class”, “::class.java”, as a type argument
    • cannot be used for: create a new instance of T, call T’s companion methods, pass not-reified parameter to function that expects reified parameter
  • variance (specified on class level):
    • co-variance, <out T>, means T can be used only on “out” positions, applies only to public, protected or internal class members.
    • contra-variance, <in  T>, means T can be used only on “in” positions
    • invariant – no relations, T at any position
    • constructor parameters are neither “in” nor “out”
    • iser-side variance specification, like at java boundaries extends/super, when variance is specified on each T occurrence, means at each method
  • type projections – restrictions of user side variance on calling some type’s methods that can broke variance

 

Annotations

  • @Deprecated – allows to specify new version of deprecated construction
  • can have a parameters of the following types:
    • primitive types
    • strings
    • enum
    • class references (<class name>::)
    • annotation classes
    • arrays (“arrayOf”)
  • “const” modifier for compile-time constant – can be used at annotations, should be placed at top-level file or object, only primitive types and String are allowed.
  • since kotlin’s property corresponds to java’s field, getter and/or setter, it should be specified what exactly (target) is expected to be decorated with the annotation, like @<target name>:<annotation name>, possible targets:
    • property – not for java
    • field
    • get/set
    • receiver
    • param – constructor’s parameter
    • setparam
    • delegate – delegator instance
    • file – needs to be placed on top of file even before “package” instruction
  • annotations can be applied almost to every code line
  • to customize byte-code generation for java:
    • @JvmName – changes method of field name
    • @JvmStatic – turns object’s method into java static
    • @JvmOverloads – generates overloads methods for java from kotlin method with default values
    • @JvmField turns kotlin property into java’s field (with no getters or setters)
  • declaring annotations: “annotation class <class name>(<parameters list>)”, but no body is allowed
  • meta-annotations – annotation that applied to annotation classes, like @Target (defines retention)

 

Reflection

  • java’s – java.lang.reflect
  • kotlin’s – kotlin.reflect
  • “::<parameter>” – gets an reflection instance of parameter

 

DSL, it is hard, as usually…

  • invoke – callable objects
  • lambdas with receivers

 

Advertisements

Real Estate marketplace

A friend of mine created his own project, it is in essense a real estate marketplace – http://metr.ua
Assisted with some issues related to Spring, Hibernate, Amason S3 during development.

Main difference from other marketplaces, among other things is that there are no real estate agency fees or comissions – sellers, buyers and tentants can contact each other directly (buyers can save thousands of dollars on fees).
Pretty good design and usability!

Apache Kafka notes

Apache Kafka Java example, topic, producer, consumer:

https://github.com/DmitryKrinitsyn/pets2/blob/master/kafkacluster2/src/test/java/edu/dkrinitsyn/kafka/KafkaTest.java

Apache Kafka and Zookeeper clusters docker:

https://github.com/DmitryKrinitsyn/pets2/blob/master/kafkacluster2/src/main/resources/docker-compose.yml

Purpose of Kafka, mostly:

  • messaging systems
  • streaming data

Message – data unit

Batch – collection of messages flushed to disk together for performance optimisation (after some amount is collected, or some time passed).

Schema – message serialization by producer and deserialization by consumer, popular – avro. Schema can be stored within Kafka separately from actual messages.

Topic – group of partitions (default number of partition and replication factor: 1)

Partition – messages are appended at the end, or deleted from beginning. Message ordering is preserved only within partition.

Producers send messages to Kafka, of no partition key specified, messages evenly distributed over partition within topic specified. If a message need to be written to specific partition, message key should be specified or custom partitioner can be written.

Consumers – consume messages from a a topic, as part of consumer group. Several consumer groups can subscribe for topic. Only one consumer from a group can read messages from specific partition.

Kafka server – one or more brokers.

Controller – one elected broker from cluster, for administration tasks, like assigning partition to brokers and monitoring brokers failures.

Each partition is read/written by single broker, called leader for this partition (for consistency). Partition can be bound to multiple brokers, but all other brokers beside leader for this partition just replicate (duplicate) it for failover reasons.

Messages within partitions of some topic are preserved (retention) for:

  • some time (default is 7 days)
  • partition length (default 1 GB)

MirrorMaker makes cluster duplication between datacenters, for failover reasons.

Broker important settings:

  • id
  • port
  • zookeeper
  • log directory

Topic important settings:

  • number of partition created automatically
  • retention time
  • retention size

Brokers are stateless regarding number of consumed messages, recent offset should be stored and provided by consumer.

Log compaction – relates to messages with same key – most recent message is preserved, while previous can be deleted, as less interested for consumer.

Message compression – group of messages (batch), is compressed:

  • on producer, to minimize network traffic
  • on leader, it is decompressed, offset is assigned for each message and messages get compressed again to minimize storage size and speedup potential cross-cluster replication.

Replication – copying messages from leader to followers, and notifying producer about success/failure:

  • sync – producer is acknowledged by leader about successful write, after all followers write the message to disk.
  • async – producer is acknowledged by leader about successful write, after message is flushed to disk on leader node only, but before all followers write the message to disk.

Writing producers:

  • connects to random broker from cluster, gets information about leaders, and connects second type for sending actual messages to leader broker for topic specified.
  • messages can be sent:
    • synchronously, per message
    • asynchronously, per message or in batches (flushed by count of messages pre-collected, or by time limit for collecting)
    • fire-and-forget

Producer important settings:

  • list of brokers
  • key and value serializers
  • type (sync, async)
  • acks count
  • partitioner class
  • compression codec (gzip, snappy, none)
  • batch size or time to wait for async batching mode

Reading consumers:

  • connects to random broker from cluster, gets information about leaders, and connects second type for reading actual messages to leader broker for topic specified.
  • high and low level API

Consumer groups:

  • name is global across cluster
  • adding new consumer to group causes rebalancing consumers between brokers, what can cause message deliver inconsistency. To mitigate: shutdown all consumers and start them up.

Consumed messages, but not committed, can be consumed once again (by another consumer) after rebalancing, if:

  • consumer which consumed some amount of messages crashed before commit
  • rebalancing happened for another reason, like adding new consumer into group, altering cluster, adding brokers, changing replication factor

Committing offset:

  • auto (default every 5 seconds), or next poll(), commits offset of previous poll()
  • manually, commitSync() or commitAsync(), commits all messages from last poll() called.
  • manually, particular offset (sync or async)

Rebalancing can have special logic assositated with, by implementing ConsumerRebalanceListener interface.

Exiting consumer loop: consumer.wakeup()

High level consumer API:

  • current offset is stored within Zookeeper, it is handled automatically underneath.

Low level (simple) consumer API:

  • offset is stored by consumer itself
  • contacts any broker to find a leader broker for topic of interest

Multithreaded consumers should map: one partition – one thread – one consumer.

Consumer can subscribe for topic or to specific partitions.

Consumer important settings:

  • group id
  • id
  • zookeeper, with timeouts
  • client id (kafka client identification)
  • offset autocomits timeouts
  • initial offset (latest, smalest, ?)
  • time interval to wait for incoming messages before exception

Administration, tools:

  • tool for graceful shutdown of broker (another leader will be selected proactively or shutdown operation failed if there is no new leader, to minimize downtime, milliseconds, and avoid data loss)
  • tool for rebalancing of leads between available brokers
  • tools for populating newly added broker with partitions (or replacing existing partitions from a broker planned for decommission), since after new broker is added, it is empty.
  • tool for increasing number of partition at some topic
  • tool to list all topics, partitions, all other details for specific cluster.

Apache Zookeeper notes

Zookeeper handles coordination tasks between multiple activities, cooperation and contention, sharing state for distributed application. Before same problems were solved via DB primary.

It is just a remote tree (with such called znodes). Znodes can have data associated with, binary format. No partial writes are allowed, whole data can be added/replaced/deleted only.

Types of tasks to solve:

  • master election
  • master/worker crash detection
  • membership/clastering
  • common data storage (small amount)

Operations allowed:

  • create znode (with optional data)
  • delete znode
  • check if znode exists
  • get/set data on existing node
  • get znode’s children

Kinds of nodes:

  • persistent – can be deleted via delete operation
  • ephemerial – can deleted via delete operation or automatically, if client created the node, crashed or simply closed connection. No children allowed here.
  • sequential – assigned with unique increasing integer.

Tracking node state changing:

  • polling
  • notification: set a “watch” – one-time trigger, reacts on change of some znode. Should be reset each time it fires, if znode should be constantly monitored.

Znode has a version associated with it, it allows conditional operation (delete and setData), means optimistic lock.

Zookeeper operates as standalone server or quorum multi-server (ensemble), clients should establish and keep connection session with server.

Quorum (odd number) – amount of running servers to run an ensemble, or amount of servers to store data (to avoid brain-split problem)

Session can be moved to another server within ensemble, transparently to client. Client’s commands within same session are strongly FIFO executed on server.

Session’s states: NOT_CONNECTED, CONNECTING, CONNECTED, CLOSED.

Can be JMX monitored.

Java Spring Cloud notes

Features:

  • configurations
  • service registration
  • service discovery
  • load balancing
  • circuit breakers
  • messaging

Change of configuration setting are not pulled by clients for updates, but it can be turned-on (@Sheduled, @RefreshScope annotation) or even Spring Cloud Bus can be used (EnvironmentChangeEvent).

Ribbon – client side load balancing, @LoadBalanced for RestTemplate and for WebClient. Retry logic can be enabled.

Configuration usually locates at GIT (or other SCM), for debug purposes it can be stored locally as file. Vault backend can be used. JDBC backend for configuration storing. Config client retry.

Discovery server heartbeats clients

Discovery Client – @EnableDiscoveryClient: for Eureka, Consul, Zookeeper

Hystrix – circuit breaker

Hystrix Dashboard – allows to track Hystrix clients individually or through accumulated stream of multiple Hystrix dashboards via Turbine.

Feign – REST client

Archaius – external configurations

Spectator, Servo, Atlas – metrics

Cloud Stream – application communicates with external world through input and output channel

Binder for Kafka and RabbitMQ

Consumer group (like AWS’s target group) – set of competing members, only on is given with particular message. But all groups, subscribed for same source of messages get a copy of data. If no group specified, each service is considered a member of anonymous single-item consumer group.

Reactive streams a supported as well: RxJava, Reactor

Aggregation – connection inputs and outputs together, to avoid load on broker.

Binder – connection to particular Broker (RabbitMQ or Kafka)

Schema based message converters (out-of-the-box is Avro only supported for the moment), Schema registry stored schemas.

Spring Cloud Bus – delivering configuration changes or infrastructure management instructions to microservices (RabbitMQ or Kafka).

Sleuth – distributed tracing. Span – request+response (usually it is HTTP request+response, but can be written manually, like transaction, or in form of annotation, assigned on Runnable interface). Set of spans – tree-like structure, span and all its children. Zipkin – trace visualisation.

Consul – Service Discovery, Control Bus and Configuration.

Spring Cloud Contract – like customer written acceptance tests, to make sure that any service fits its microservice environment.

Spring Cloud Vault Config – stores configs and secrets for microservice application (HashiCorp Vault).

 

Java Spring Boot notes

Important maven artifacts:

  • spring-boot-starter-parent (versions)
  • spring-boot-maven-plugin (executable jar)

 

Main features:

  • BOM, bills of materials (spring-boot-starter-parent or spring-boot-dependencies)
  • Autoconfiguration (@EnableAutoConfiguration, @ComponentScan, @Configuration, or @SpringBootApplication), with exclude and redefine config, from annotation or yaml-config.
  • creating own configurations
  • embedded servlet (version 3.1) containers: Tomcat, Jetty, UnderTow, Netty
  • eeveloper tools (can be automatically removed while running fully packaged application): caching, automatic restart/reload, remote automatic restart/reload
  • starters
  • actuators
  • CommandLineRunner
  • custom health checks and info
  • custom metrics

 

Auto-configuration for:

  • caching function returned cached values (it is possible to update or evict)
  • messaging
  • WebClient is better than RestTemplate (at least it is reactive)
  • bean validation
  • metrics
  • http tracing (last 100 calls)

Java framework: Spring Core

Each bean has an ID in scope of context (by default it is modified class name, can be specified explicitly)

Container implementations:

  • bean factories (simple)
  • application context (advanced)
    • internationalization
    • event publication
    • resource management
    • life-cycle events

Popular application contexts:

  • AnnotationConfigApplicationContext – java based application context
  • AnnotationConfigWebApplicationContext – java based web context
  • ClassPathXmlApplicationContext
  • FileSystemXmlApplicationContext
  • XmlWebApplicationContext

Bean lifecycle

  • instantiate
  • set properties
  • setBeanName()
  • setBeanFactory()
  • setApplicationContext()
  • postProcessBeforeInitialization() (from BeanPostProcessor interface)
  • afterPropertiesSet() (from InitializingBean interface)
  • custom init method
  • postProcessAfterInitialization() (from BeanPostProcessor interface)
  • USAGE
  • destroy() (from DisposableBean interface)
  • custom destroy method

Bean configuration:

  • explicit XML
  • explicit Java config
  • implicit (component scanning and autowiring)

Annotations:

  • @Component/@Named – marks bean (more specific @Service, @Repository, @Controller)
  • @ComponentScan – turns on component scanning (XML: <context:component-scan>)
  • @Configuration – marks java configuration class – current package and subpackages will be scanned for @Component marked classes
  • @Profile/@ActiveProfiles/@IfProfileValue
  • @Conditional – if condition is true – bean gets created
  • @Autowired/@Inject – marks a destination for bean wiring (if no beans wired, exception is thrown – default, of null reference left – can be enabled explicitly, if more than one bean found – exception is thrown)

Destination for wired beans:

  • constructor
  • setter
  • method

What can be wired:

  • references on other objects/beans
  • literals
  • collections (list, map, set)

Mitigating ambiguity:

  • @Primary – marked bean will have precedence over other matching beans
  • @Qaulifier – additional level of matching between @Component and @Autowired

Scoping @Scope bean annotation, instantiation mode:

  • singleton (default) – only one instance is created
  • prototype – each time new instance
  • session – WEB specific
  • request – WEB specific
  • global session – for portlet-based WEB application
  • thread
  • custom

WEB specific beans are wired through proxy (in case of interface, java dynamic proxy creation) or through inheritance (CGLib)

Spring contexts can be nested to properly separate logic domains, beans from child context can refer beans from parent context.

Spring bean’s method can be replaced (via xml configuration) by other bean’s method.

Beans can have multiple names (but single ID), and aliases to names/id.